• INN 2.x FAQ (2/3)

    From Russ Allbery@21:1/5 to All on Sat Nov 21 08:01:04 2020
    [continued from previous message]

    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, the default cc requires -O be given when -Kalloca is
    given (which is added by default by configure since the parsers generated
    by bison need it). However, there appears to be a bug in the compiler
    that causes it to miscompile nnrpd/commands.c under -O, generating the
    error:

    Assembler: commands.c
    aline 1505 : Syntax error

    I had to get around this by cd'ing into nnrpd, running:

    make COPT=-g commands.o

    and then cd'ing back to the top level and running make again. On
    OpenServer, to build with cc, you have to use:

    env CC=cc CFLAGS=-O ./configure --with-sendmail=/usr/lib/sendmail

    Building under gcc is cleaner, but of course if you want to use
    --with-perl you want to build with the same compiler that you built Perl
    with.

    It's also worth noting that with a shared Perl library, Perl on this
    platform doesn't apparently generate the right link magic to include the
    path to the dynamic Perl libraries. You need to either set LD_RUN_PATH
    before building or LD_LIBRARY_PATH before running any binaries so that
    they can find the Perl libraries. (The former is preferred, since then
    the path is encoded into the binaries and you don't have to remember to
    set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on Mac OS X

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration, so this problem may be
    fixed with newer builds of Mac OS X.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles.

    In case you wish to only feed articles arrived on the old server
    between two dates, you can adapt the previous commands. For instance,
    the following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref headers. Current versions of INN
    will always add an Xref header, but very old versions (earlier 1.x
    versions) will only add an Xref header to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles
    to add an Xref header.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups: header
    is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using the tradspool storage method. The newsgroup of an article is stored in the Newsgroups header and Xref header of the article as stored on disk (and possibly in Followup-To), as well as determining where the overview
    information is stored, and in the case of tradspool is also encoded in the article's storage token. To rename a newsgroup in tradspool, stop the
    server, move the directory containing all of the articles to its
    appropriate new location in the news spool, edit every article to change
    the old name to the new name in Newsgroups, Followup-To, and Xref, create
    the new newsgroup with ctlinnd newgroup, and then rebuild history and
    overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups headers didn't break those links, you may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To headers of the articles in that newsgroup and then rebuild history and
    overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To headers
    to rename the newsgroup, and then create a file containing paths to all of
    the articles, one per line. You can then use that file as input to
    innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message IDs generated by INN will use the domain of the
    local system for the right-hand-side of the message ID. In some cases,
    this isn't desirable for various reasons (the server may have an internal
    name that doesn't make sense on Usenet at large, or one may not want to
    expose the name of the server).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will use
    that domain to generate message IDs. So if you need to change the domain
    used to generate message IDs for every local post from your server, just
    add virtualhost: and domain: keys to every access stanza in readers.conf.

    This is really overkill for this option, and eventually the domain:
    parameter in inn.conf will probably be changed to allow this to be
    modified for the whole server. (Right now, domain: in inn.conf means
    something completely different.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------

    Subject: 6.8. Generate MRTG graphs for INN

    INN's CNFS storage system has direct support for producing information
    suitable for MRTG graphs on the usage of the CNFS cycbuffs. Running
    cnfsstat -m <cycbuf> will generate output suitable for MRTG, and running cnfsstat -p will generate sample MRTG configuration fragments for each
    cycbuff.

    To generate MRTG graphs of the usage of the buffindexed overview system,
    try the following configuration fragment:

    Target[overview-BUFF]: `/usr/local/etc/mrtg/overview.sh`
    MaxBytes[overview-BUFF]: 100
    Title[overview-BUFF]: BUFF1 Usage
    Options[overview-BUFF]: growright gauge
    YLegend[overview-BUFF]: Overview Buffers
    ShortLegend[overview-BUFF]: %
    PageTop[overview-BUFF]: <H1>Usage of Overview Buffers</H1>
    <BR><TT>overview</TT>

    where the overview.sh script is:

    #!/bin/sh
    echo "100"
    <pathbin in inn.conf>/inndf -o | awk '{print $1}'
    echo "0"
    echo "overview"

    This sample configuration is from Basil Kruglov. Note that you can
    instead use -n (for total count of articles); in that case, you'll want to remove the MaxBytes setting above or change it to be some sensible limit
    on the total number of articles you receive. You'll also want to change a
    few of the other labels in the MRTG configuration.

    I'm not aware of any packaged solutions for generating MRTG data from
    other things, such as incoming or outcoming news flows. If anyone has any pointers, let me know.

    ------------------------------

    Subject: 6.9. Hide the junk and control groups from users

    The junk, control, and control.cancel groups must exist in the active file
    for the proper operation of INN, so you can't remove the groups entirely.
    You can, however, hide them completely from users.

    To do this, edit readers.conf, and for each user access group where you
    want to hide the junk and control groups, add "!junk,!control,!control.*"
    to the newsgroups pattern. In other words, if you have a line like:

    newsgroups: *

    just change that to:

    newsgroups: *,!junk,!control,!control.*

    If you use read and post patterns instead, do the same for each of them individually. The groups will then no longer show up on the server for
    users to which that access group applies; it will be as if they do not
    exist.

    ------------------------------

    Subject: 6.10. Modify the body of posts made through my server

    You can't without either making code changes to INN or putting your own software in the path of incoming posts. This is intentional.

    Some sites like to try to append a standard signature to all posts through their service, generally as advertising. This creates the appearance of
    users saying things that they didn't, runs the risk of corrupting messages
    by appending text without regard to what's in the message, and can
    possibly modify messages that arrive via a suck/rpost connection. It also
    adds advertising in an obnoxious location, rather than in the Organization header which is more widely used for that purpose. Accordingly, INN does
    not support this, or any other modification of the body of a message from inside the news server.

    If you only want to do this for a private hierarchy, the easiest way to do
    this (as well as any other modifications and internal filtering that you
    want to perform) is to mark all of the groups as moderated and route all submissions through a script that makes whatever modifications you want
    and then posts the messages with an Approved header.

    If you want to do this in order to advertise your service, please
    reconsider. You can add your advertisements to the headers, like many
    other news service providers.

    ------------------------------

    Subject: 6.11. Hide the Injection-Info header

    There is no built-in support for suppressing generation of the
    Injection-Info header. You can, however, remove it from inside a Perl
    posting filter. Try using a posting filter like this:

    sub filter_post {
    $modify_headers = 1;
    delete $hdr{'Injection-Info'};
    return '';
    }

    Note that you have to set $modify_headers to make changes to the article
    header effective in the actual posted article. Instead of removing the
    header, you can also alter it if you modify $hdr{'Injection-Info'}. If you only want to alter the host name used in Injection-Info, see the
    virtualhost: and domain: parameters in readers.conf.

    ------------------------------

    Subject: 6.12. Run innd and nnrpd on separate ports


    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Mon Dec 21 08:01:02 2020
    [continued from previous message]

    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, the default cc requires -O be given when -Kalloca is
    given (which is added by default by configure since the parsers generated
    by bison need it). However, there appears to be a bug in the compiler
    that causes it to miscompile nnrpd/commands.c under -O, generating the
    error:

    Assembler: commands.c
    aline 1505 : Syntax error

    I had to get around this by cd'ing into nnrpd, running:

    make COPT=-g commands.o

    and then cd'ing back to the top level and running make again. On
    OpenServer, to build with cc, you have to use:

    env CC=cc CFLAGS=-O ./configure --with-sendmail=/usr/lib/sendmail

    Building under gcc is cleaner, but of course if you want to use
    --with-perl you want to build with the same compiler that you built Perl
    with.

    It's also worth noting that with a shared Perl library, Perl on this
    platform doesn't apparently generate the right link magic to include the
    path to the dynamic Perl libraries. You need to either set LD_RUN_PATH
    before building or LD_LIBRARY_PATH before running any binaries so that
    they can find the Perl libraries. (The former is preferred, since then
    the path is encoded into the binaries and you don't have to remember to
    set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles.

    In case you wish to only feed articles arrived on the old server
    between two dates, you can adapt the previous commands. For instance,
    the following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref headers. Current versions of INN
    will always add an Xref header, but very old versions (earlier 1.x
    versions) will only add an Xref header to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles
    to add an Xref header.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups: header
    is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using the tradspool storage method. The newsgroup of an article is stored in the Newsgroups header and Xref header of the article as stored on disk (and possibly in Followup-To), as well as determining where the overview
    information is stored, and in the case of tradspool is also encoded in the article's storage token. To rename a newsgroup in tradspool, stop the
    server, move the directory containing all of the articles to its
    appropriate new location in the news spool, edit every article to change
    the old name to the new name in Newsgroups, Followup-To, and Xref, create
    the new newsgroup with ctlinnd newgroup, and then rebuild history and
    overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups headers didn't break those links, you may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To headers of the articles in that newsgroup and then rebuild history and
    overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To headers
    to rename the newsgroup, and then create a file containing paths to all of
    the articles, one per line. You can then use that file as input to
    innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message IDs generated by INN will use the domain of the
    local system for the right-hand-side of the message ID. In some cases,
    this isn't desirable for various reasons (the server may have an internal
    name that doesn't make sense on Usenet at large, or one may not want to
    expose the name of the server).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will use
    that domain to generate message IDs. So if you need to change the domain
    used to generate message IDs for every local post from your server, just
    add virtualhost: and domain: keys to every access stanza in readers.conf.

    This is really overkill for this option, and eventually the domain:
    parameter in inn.conf will probably be changed to allow this to be
    modified for the whole server. (Right now, domain: in inn.conf means
    something completely different.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------

    Subject: 6.8. Generate MRTG graphs for INN

    INN's CNFS storage system has direct support for producing information
    suitable for MRTG graphs on the usage of the CNFS cycbuffs. Running
    cnfsstat -m <cycbuf> will generate output suitable for MRTG, and running cnfsstat -p will generate sample MRTG configuration fragments for each
    cycbuff.

    To generate MRTG graphs of the usage of the buffindexed overview system,
    try the following configuration fragment:

    Target[overview-BUFF]: `/usr/local/etc/mrtg/overview.sh`
    MaxBytes[overview-BUFF]: 100
    Title[overview-BUFF]: BUFF1 Usage
    Options[overview-BUFF]: growright gauge
    YLegend[overview-BUFF]: Overview Buffers
    ShortLegend[overview-BUFF]: %
    PageTop[overview-BUFF]: <H1>Usage of Overview Buffers</H1>
    <BR><TT>overview</TT>

    where the overview.sh script is:

    #!/bin/sh
    echo "100"
    <pathbin in inn.conf>/inndf -o | awk '{print $1}'
    echo "0"
    echo "overview"

    This sample configuration is from Basil Kruglov. Note that you can
    instead use -n (for total count of articles); in that case, you'll want to remove the MaxBytes setting above or change it to be some sensible limit
    on the total number of articles you receive. You'll also want to change a
    few of the other labels in the MRTG configuration.

    I'm not aware of any packaged solutions for generating MRTG data from
    other things, such as incoming or outcoming news flows. If anyone has any pointers, let me know.

    ------------------------------

    Subject: 6.9. Hide the junk and control groups from users

    The junk, control, and control.cancel groups must exist in the active file
    for the proper operation of INN, so you can't remove the groups entirely.
    You can, however, hide them completely from users.

    To do this, edit readers.conf, and for each user access group where you
    want to hide the junk and control groups, add "!junk,!control,!control.*"
    to the newsgroups pattern. In other words, if you have a line like:

    newsgroups: *

    just change that to:

    newsgroups: *,!junk,!control,!control.*

    If you use read and post patterns instead, do the same for each of them individually. The groups will then no longer show up on the server for
    users to which that access group applies; it will be as if they do not
    exist.

    ------------------------------

    Subject: 6.10. Modify the body of posts made through my server

    You can't without either making code changes to INN or putting your own software in the path of incoming posts. This is intentional.

    Some sites like to try to append a standard signature to all posts through their service, generally as advertising. This creates the appearance of
    users saying things that they didn't, runs the risk of corrupting messages
    by appending text without regard to what's in the message, and can
    possibly modify messages that arrive via a suck/rpost connection. It also
    adds advertising in an obnoxious location, rather than in the Organization header which is more widely used for that purpose. Accordingly, INN does
    not support this, or any other modification of the body of a message from inside the news server.

    If you only want to do this for a private hierarchy, the easiest way to do
    this (as well as any other modifications and internal filtering that you
    want to perform) is to mark all of the groups as moderated and route all submissions through a script that makes whatever modifications you want
    and then posts the messages with an Approved header.

    If you want to do this in order to advertise your service, please
    reconsider. You can add your advertisements to the headers, like many
    other news service providers.

    ------------------------------

    Subject: 6.11. Hide the Injection-Info header

    There is no built-in support for suppressing generation of the
    Injection-Info header. You can, however, remove it from inside a Perl
    posting filter. Try using a posting filter like this:

    sub filter_post {
    $modify_headers = 1;
    delete $hdr{'Injection-Info'};
    return '';
    }

    Note that you have to set $modify_headers to make changes to the article
    header effective in the actual posted article. Instead of removing the
    header, you can also alter it if you modify $hdr{'Injection-Info'}. If you only want to alter the host name used in Injection-Info, see the
    virtualhost: and domain: parameters in readers.conf.

    ------------------------------

    Subject: 6.12. Run innd and nnrpd on separate ports

    Originally, innd was designed to handle all incoming connections and hand

    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Thu Jan 21 08:01:03 2021
    [continued from previous message]

    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports and to mail messages to moderators, and it assumes that you
    have a program installed as /usr/sbin/sendmail or /usr/lib/sendmail that
    it can use to do this. It does not speak SMTP, nor is it likely to ever
    speak SMTP; it's hard enough maintaining a package to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, the default cc requires -O be given when -Kalloca is
    given (which is added by default by configure since the parsers generated
    by bison need it). However, there appears to be a bug in the compiler
    that causes it to miscompile nnrpd/commands.c under -O, generating the
    error:

    Assembler: commands.c
    aline 1505 : Syntax error

    I had to get around this by cd'ing into nnrpd, running:

    make COPT=-g commands.o

    and then cd'ing back to the top level and running make again. On
    OpenServer, to build with cc, you have to use:

    env CC=cc CFLAGS=-O ./configure --with-sendmail=/usr/lib/sendmail

    Building under gcc is cleaner, but of course if you want to use
    --with-perl you want to build with the same compiler that you built Perl
    with.

    It's also worth noting that with a shared Perl library, Perl on this
    platform doesn't apparently generate the right link magic to include the
    path to the dynamic Perl libraries. You need to either set LD_RUN_PATH
    before building or LD_LIBRARY_PATH before running any binaries so that
    they can find the Perl libraries. (The former is preferred, since then
    the path is encoded into the binaries and you don't have to remember to
    set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles.

    In case you wish to only feed articles arrived on the old server
    between two dates, you can adapt the previous commands. For instance,
    the following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref headers. Current versions of INN
    will always add an Xref header, but very old versions (earlier 1.x
    versions) will only add an Xref header to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles
    to add an Xref header.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups: header
    is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using the tradspool storage method. The newsgroup of an article is stored in the Newsgroups header and Xref header of the article as stored on disk (and possibly in Followup-To), as well as determining where the overview
    information is stored, and in the case of tradspool is also encoded in the article's storage token. To rename a newsgroup in tradspool, stop the
    server, move the directory containing all of the articles to its
    appropriate new location in the news spool, edit every article to change
    the old name to the new name in Newsgroups, Followup-To, and Xref, create
    the new newsgroup with ctlinnd newgroup, and then rebuild history and
    overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups headers didn't break those links, you may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To headers of the articles in that newsgroup and then rebuild history and
    overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To headers
    to rename the newsgroup, and then create a file containing paths to all of
    the articles, one per line. You can then use that file as input to
    innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message IDs generated by INN will use the domain of the
    local system for the right-hand-side of the message ID. In some cases,
    this isn't desirable for various reasons (the server may have an internal
    name that doesn't make sense on Usenet at large, or one may not want to
    expose the name of the server).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will use
    that domain to generate message IDs. So if you need to change the domain
    used to generate message IDs for every local post from your server, just
    add virtualhost: and domain: keys to every access stanza in readers.conf.

    This is really overkill for this option, and eventually the domain:
    parameter in inn.conf will probably be changed to allow this to be
    modified for the whole server. (Right now, domain: in inn.conf means
    something completely different.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------

    Subject: 6.8. Generate MRTG graphs for INN

    INN's CNFS storage system has direct support for producing information
    suitable for MRTG graphs on the usage of the CNFS cycbuffs. Running
    cnfsstat -m <cycbuf> will generate output suitable for MRTG, and running cnfsstat -p will generate sample MRTG configuration fragments for each
    cycbuff.

    To generate MRTG graphs of the usage of the buffindexed overview system,
    try the following configuration fragment:

    Target[overview-BUFF]: `/usr/local/etc/mrtg/overview.sh`
    MaxBytes[overview-BUFF]: 100
    Title[overview-BUFF]: BUFF1 Usage
    Options[overview-BUFF]: growright gauge
    YLegend[overview-BUFF]: Overview Buffers
    ShortLegend[overview-BUFF]: %
    PageTop[overview-BUFF]: <H1>Usage of Overview Buffers</H1>
    <BR><TT>overview</TT>

    where the overview.sh script is:

    #!/bin/sh
    echo "100"
    <pathbin in inn.conf>/inndf -o | awk '{print $1}'
    echo "0"
    echo "overview"

    This sample configuration is from Basil Kruglov. Note that you can
    instead use -n (for total count of articles); in that case, you'll want to remove the MaxBytes setting above or change it to be some sensible limit
    on the total number of articles you receive. You'll also want to change a
    few of the other labels in the MRTG configuration.

    I'm not aware of any packaged solutions for generating MRTG data from
    other things, such as incoming or outcoming news flows. If anyone has any pointers, let me know.

    ------------------------------

    Subject: 6.9. Hide the junk and control groups from users

    The junk, control, and control.cancel groups must exist in the active file
    for the proper operation of INN, so you can't remove the groups entirely.
    You can, however, hide them completely from users.

    To do this, edit readers.conf, and for each user access group where you
    want to hide the junk and control groups, add "!junk,!control,!control.*"
    to the newsgroups pattern. In other words, if you have a line like:

    newsgroups: *

    just change that to:

    newsgroups: *,!junk,!control,!control.*

    If you use read and post patterns instead, do the same for each of them individually. The groups will then no longer show up on the server for
    users to which that access group applies; it will be as if they do not
    exist.

    ------------------------------

    Subject: 6.10. Modify the body of posts made through my server

    You can't without either making code changes to INN or putting your own software in the path of incoming posts. This is intentional.


    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Sun Feb 21 08:01:02 2021
    [continued from previous message]

    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports and to mail messages to moderators, and it assumes that you
    have a program installed as /usr/sbin/sendmail or /usr/lib/sendmail that
    it can use to do this. It does not speak SMTP, nor is it likely to ever
    speak SMTP; it's hard enough maintaining a package to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, the default cc requires -O be given when -Kalloca is
    given (which is added by default by configure since the parsers generated
    by bison need it). However, there appears to be a bug in the compiler
    that causes it to miscompile nnrpd/commands.c under -O, generating the
    error:

    Assembler: commands.c
    aline 1505 : Syntax error

    I had to get around this by cd'ing into nnrpd, running:

    make COPT=-g commands.o

    and then cd'ing back to the top level and running make again. On
    OpenServer, to build with cc, you have to use:

    env CC=cc CFLAGS=-O ./configure --with-sendmail=/usr/lib/sendmail

    Building under gcc is cleaner, but of course if you want to use
    --with-perl you want to build with the same compiler that you built Perl
    with.

    It's also worth noting that with a shared Perl library, Perl on this
    platform doesn't apparently generate the right link magic to include the
    path to the dynamic Perl libraries. You need to either set LD_RUN_PATH
    before building or LD_LIBRARY_PATH before running any binaries so that
    they can find the Perl libraries. (The former is preferred, since then
    the path is encoded into the binaries and you don't have to remember to
    set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles.

    In case you wish to only feed articles arrived on the old server
    between two dates, you can adapt the previous commands. For instance,
    the following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref headers. Current versions of INN
    will always add an Xref header, but very old versions (earlier 1.x
    versions) will only add an Xref header to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles
    to add an Xref header.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups: header
    is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using the tradspool storage method. The newsgroup of an article is stored in the Newsgroups header and Xref header of the article as stored on disk (and possibly in Followup-To), as well as determining where the overview
    information is stored, and in the case of tradspool is also encoded in the article's storage token. To rename a newsgroup in tradspool, stop the
    server, move the directory containing all of the articles to its
    appropriate new location in the news spool, edit every article to change
    the old name to the new name in Newsgroups, Followup-To, and Xref, create
    the new newsgroup with ctlinnd newgroup, and then rebuild history and
    overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups headers didn't break those links, you may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To headers of the articles in that newsgroup and then rebuild history and
    overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To headers
    to rename the newsgroup, and then create a file containing paths to all of
    the articles, one per line. You can then use that file as input to
    innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message IDs generated by INN will use the domain of the
    local system for the right-hand-side of the message ID. In some cases,
    this isn't desirable for various reasons (the server may have an internal
    name that doesn't make sense on Usenet at large, or one may not want to
    expose the name of the server).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will use
    that domain to generate message IDs. So if you need to change the domain
    used to generate message IDs for every local post from your server, just
    add virtualhost: and domain: keys to every access stanza in readers.conf.

    This is really overkill for this option, and eventually the domain:
    parameter in inn.conf will probably be changed to allow this to be
    modified for the whole server. (Right now, domain: in inn.conf means
    something completely different.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------

    Subject: 6.8. Generate MRTG graphs for INN

    INN's CNFS storage system has direct support for producing information
    suitable for MRTG graphs on the usage of the CNFS cycbuffs. Running
    cnfsstat -m <cycbuf> will generate output suitable for MRTG, and running cnfsstat -p will generate sample MRTG configuration fragments for each
    cycbuff.

    To generate MRTG graphs of the usage of the buffindexed overview system,
    try the following configuration fragment:

    Target[overview-BUFF]: `/usr/local/etc/mrtg/overview.sh`
    MaxBytes[overview-BUFF]: 100
    Title[overview-BUFF]: BUFF1 Usage
    Options[overview-BUFF]: growright gauge
    YLegend[overview-BUFF]: Overview Buffers
    ShortLegend[overview-BUFF]: %
    PageTop[overview-BUFF]: <H1>Usage of Overview Buffers</H1>
    <BR><TT>overview</TT>

    where the overview.sh script is:

    #!/bin/sh
    echo "100"
    <pathbin in inn.conf>/inndf -o | awk '{print $1}'
    echo "0"
    echo "overview"

    This sample configuration is from Basil Kruglov. Note that you can
    instead use -n (for total count of articles); in that case, you'll want to remove the MaxBytes setting above or change it to be some sensible limit
    on the total number of articles you receive. You'll also want to change a
    few of the other labels in the MRTG configuration.

    I'm not aware of any packaged solutions for generating MRTG data from
    other things, such as incoming or outcoming news flows. If anyone has any pointers, let me know.

    ------------------------------

    Subject: 6.9. Hide the junk and control groups from users

    The junk, control, and control.cancel groups must exist in the active file
    for the proper operation of INN, so you can't remove the groups entirely.
    You can, however, hide them completely from users.

    To do this, edit readers.conf, and for each user access group where you
    want to hide the junk and control groups, add "!junk,!control,!control.*"
    to the newsgroups pattern. In other words, if you have a line like:

    newsgroups: *

    just change that to:

    newsgroups: *,!junk,!control,!control.*

    If you use read and post patterns instead, do the same for each of them individually. The groups will then no longer show up on the server for
    users to which that access group applies; it will be as if they do not
    exist.

    ------------------------------

    Subject: 6.10. Modify the body of posts made through my server

    You can't without either making code changes to INN or putting your own software in the path of incoming posts. This is intentional.


    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Sun Mar 21 07:01:03 2021
    [continued from previous message]

    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports and to mail messages to moderators, and it assumes that you
    have a program installed as /usr/sbin/sendmail or /usr/lib/sendmail that
    it can use to do this. It does not speak SMTP, nor is it likely to ever
    speak SMTP; it's hard enough maintaining a package to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, the default cc requires -O be given when -Kalloca is
    given (which is added by default by configure since the parsers generated
    by bison need it). However, there appears to be a bug in the compiler
    that causes it to miscompile nnrpd/commands.c under -O, generating the
    error:

    Assembler: commands.c
    aline 1505 : Syntax error

    I had to get around this by cd'ing into nnrpd, running:

    make COPT=-g commands.o

    and then cd'ing back to the top level and running make again. On
    OpenServer, to build with cc, you have to use:

    env CC=cc CFLAGS=-O ./configure --with-sendmail=/usr/lib/sendmail

    Building under gcc is cleaner, but of course if you want to use
    --with-perl you want to build with the same compiler that you built Perl
    with.

    It's also worth noting that with a shared Perl library, Perl on this
    platform doesn't apparently generate the right link magic to include the
    path to the dynamic Perl libraries. You need to either set LD_RUN_PATH
    before building or LD_LIBRARY_PATH before running any binaries so that
    they can find the Perl libraries. (The former is preferred, since then
    the path is encoded into the binaries and you don't have to remember to
    set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles.

    In case you wish to only feed articles arrived on the old server
    between two dates, you can adapt the previous commands. For instance,
    the following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref headers. Current versions of INN
    will always add an Xref header, but very old versions (earlier 1.x
    versions) will only add an Xref header to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles
    to add an Xref header.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups: header
    is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using the tradspool storage method. The newsgroup of an article is stored in the Newsgroups header and Xref header of the article as stored on disk (and possibly in Followup-To), as well as determining where the overview
    information is stored, and in the case of tradspool is also encoded in the article's storage token. To rename a newsgroup in tradspool, stop the
    server, move the directory containing all of the articles to its
    appropriate new location in the news spool, edit every article to change
    the old name to the new name in Newsgroups, Followup-To, and Xref, create
    the new newsgroup with ctlinnd newgroup, and then rebuild history and
    overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups headers didn't break those links, you may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To headers of the articles in that newsgroup and then rebuild history and
    overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To headers
    to rename the newsgroup, and then create a file containing paths to all of
    the articles, one per line. You can then use that file as input to
    innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message IDs generated by INN will use the domain of the
    local system for the right-hand-side of the message ID. In some cases,
    this isn't desirable for various reasons (the server may have an internal
    name that doesn't make sense on Usenet at large, or one may not want to
    expose the name of the server).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will use
    that domain to generate message IDs. So if you need to change the domain
    used to generate message IDs for every local post from your server, just
    add virtualhost: and domain: keys to every access stanza in readers.conf.

    This is really overkill for this option, and eventually the domain:
    parameter in inn.conf will probably be changed to allow this to be
    modified for the whole server. (Right now, domain: in inn.conf means
    something completely different.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------

    Subject: 6.8. Generate MRTG graphs for INN

    INN's CNFS storage system has direct support for producing information
    suitable for MRTG graphs on the usage of the CNFS cycbuffs. Running
    cnfsstat -m <cycbuf> will generate output suitable for MRTG, and running cnfsstat -p will generate sample MRTG configuration fragments for each
    cycbuff.

    To generate MRTG graphs of the usage of the buffindexed overview system,
    try the following configuration fragment:

    Target[overview-BUFF]: `/usr/local/etc/mrtg/overview.sh`
    MaxBytes[overview-BUFF]: 100
    Title[overview-BUFF]: BUFF1 Usage
    Options[overview-BUFF]: growright gauge
    YLegend[overview-BUFF]: Overview Buffers
    ShortLegend[overview-BUFF]: %
    PageTop[overview-BUFF]: <H1>Usage of Overview Buffers</H1>
    <BR><TT>overview</TT>

    where the overview.sh script is:

    #!/bin/sh
    echo "100"
    <pathbin in inn.conf>/inndf -o | awk '{print $1}'
    echo "0"
    echo "overview"

    This sample configuration is from Basil Kruglov. Note that you can
    instead use -n (for total count of articles); in that case, you'll want to remove the MaxBytes setting above or change it to be some sensible limit
    on the total number of articles you receive. You'll also want to change a
    few of the other labels in the MRTG configuration.

    I'm not aware of any packaged solutions for generating MRTG data from
    other things, such as incoming or outcoming news flows. If anyone has any pointers, let me know.

    ------------------------------

    Subject: 6.9. Hide the junk and control groups from users

    The junk, control, and control.cancel groups must exist in the active file
    for the proper operation of INN, so you can't remove the groups entirely.
    You can, however, hide them completely from users.

    To do this, edit readers.conf, and for each user access group where you
    want to hide the junk and control groups, add "!junk,!control,!control.*"
    to the newsgroups pattern. In other words, if you have a line like:

    newsgroups: *

    just change that to:

    newsgroups: *,!junk,!control,!control.*

    If you use read and post patterns instead, do the same for each of them individually. The groups will then no longer show up on the server for
    users to which that access group applies; it will be as if they do not
    exist.

    ------------------------------

    Subject: 6.10. Modify the body of posts made through my server

    You can't without either making code changes to INN or putting your own software in the path of incoming posts. This is intentional.


    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Wed Apr 21 07:01:04 2021
    [continued from previous message]

    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports and to mail messages to moderators, and it assumes that you
    have a program installed as /usr/sbin/sendmail or /usr/lib/sendmail that
    it can use to do this. It does not speak SMTP, nor is it likely to ever
    speak SMTP; it's hard enough maintaining a package to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, the default cc requires -O be given when -Kalloca is
    given (which is added by default by configure since the parsers generated
    by bison need it). However, there appears to be a bug in the compiler
    that causes it to miscompile nnrpd/commands.c under -O, generating the
    error:

    Assembler: commands.c
    aline 1505 : Syntax error

    I had to get around this by cd'ing into nnrpd, running:

    make COPT=-g commands.o

    and then cd'ing back to the top level and running make again. On
    OpenServer, to build with cc, you have to use:

    env CC=cc CFLAGS=-O ./configure --with-sendmail=/usr/lib/sendmail

    Building under gcc is cleaner, but of course if you want to use
    --with-perl you want to build with the same compiler that you built Perl
    with.

    It's also worth noting that with a shared Perl library, Perl on this
    platform doesn't apparently generate the right link magic to include the
    path to the dynamic Perl libraries. You need to either set LD_RUN_PATH
    before building or LD_LIBRARY_PATH before running any binaries so that
    they can find the Perl libraries. (The former is preferred, since then
    the path is encoded into the binaries and you don't have to remember to
    set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles.

    In case you wish to only feed articles arrived on the old server
    between two dates, you can adapt the previous commands. For instance,
    the following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref headers. Current versions of INN
    will always add an Xref header, but very old versions (earlier 1.x
    versions) will only add an Xref header to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles
    to add an Xref header.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups: header
    is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using the tradspool storage method. The newsgroup of an article is stored in the Newsgroups header and Xref header of the article as stored on disk (and possibly in Followup-To), as well as determining where the overview
    information is stored, and in the case of tradspool is also encoded in the article's storage token. To rename a newsgroup in tradspool, stop the
    server, move the directory containing all of the articles to its
    appropriate new location in the news spool, edit every article to change
    the old name to the new name in Newsgroups, Followup-To, and Xref, create
    the new newsgroup with ctlinnd newgroup, and then rebuild history and
    overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups headers didn't break those links, you may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To headers of the articles in that newsgroup and then rebuild history and
    overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To headers
    to rename the newsgroup, and then create a file containing paths to all of
    the articles, one per line. You can then use that file as input to
    innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message IDs generated by INN will use the domain of the
    local system for the right-hand-side of the message ID. In some cases,
    this isn't desirable for various reasons (the server may have an internal
    name that doesn't make sense on Usenet at large, or one may not want to
    expose the name of the server).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will use
    that domain to generate message IDs. So if you need to change the domain
    used to generate message IDs for every local post from your server, just
    add virtualhost: and domain: keys to every access stanza in readers.conf.

    This is really overkill for this option, and eventually the domain:
    parameter in inn.conf will probably be changed to allow this to be
    modified for the whole server. (Right now, domain: in inn.conf means
    something completely different.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------

    Subject: 6.8. Generate MRTG graphs for INN

    INN's CNFS storage system has direct support for producing information
    suitable for MRTG graphs on the usage of the CNFS cycbuffs. Running
    cnfsstat -m <cycbuf> will generate output suitable for MRTG, and running cnfsstat -p will generate sample MRTG configuration fragments for each
    cycbuff.

    To generate MRTG graphs of the usage of the buffindexed overview system,
    try the following configuration fragment:

    Target[overview-BUFF]: `/usr/local/etc/mrtg/overview.sh`
    MaxBytes[overview-BUFF]: 100
    Title[overview-BUFF]: BUFF1 Usage
    Options[overview-BUFF]: growright gauge
    YLegend[overview-BUFF]: Overview Buffers
    ShortLegend[overview-BUFF]: %
    PageTop[overview-BUFF]: <H1>Usage of Overview Buffers</H1>
    <BR><TT>overview</TT>

    where the overview.sh script is:

    #!/bin/sh
    echo "100"
    <pathbin in inn.conf>/inndf -o | awk '{print $1}'
    echo "0"
    echo "overview"

    This sample configuration is from Basil Kruglov. Note that you can
    instead use -n (for total count of articles); in that case, you'll want to remove the MaxBytes setting above or change it to be some sensible limit
    on the total number of articles you receive. You'll also want to change a
    few of the other labels in the MRTG configuration.

    I'm not aware of any packaged solutions for generating MRTG data from
    other things, such as incoming or outcoming news flows. If anyone has any pointers, let me know.

    ------------------------------

    Subject: 6.9. Hide the junk and control groups from users

    The junk, control, and control.cancel groups must exist in the active file
    for the proper operation of INN, so you can't remove the groups entirely.
    You can, however, hide them completely from users.

    To do this, edit readers.conf, and for each user access group where you
    want to hide the junk and control groups, add "!junk,!control,!control.*"
    to the newsgroups pattern. In other words, if you have a line like:

    newsgroups: *

    just change that to:

    newsgroups: *,!junk,!control,!control.*

    If you use read and post patterns instead, do the same for each of them individually. The groups will then no longer show up on the server for
    users to which that access group applies; it will be as if they do not
    exist.

    ------------------------------

    Subject: 6.10. Modify the body of posts made through my server

    You can't without either making code changes to INN or putting your own software in the path of incoming posts. This is intentional.


    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Fri May 21 07:01:04 2021
    [continued from previous message]

    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports and to mail messages to moderators, and it assumes that you
    have a program installed as /usr/sbin/sendmail or /usr/lib/sendmail that
    it can use to do this. It does not speak SMTP, nor is it likely to ever
    speak SMTP; it's hard enough maintaining a package to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, the default cc requires -O be given when -Kalloca is
    given (which is added by default by configure since the parsers generated
    by bison need it). However, there appears to be a bug in the compiler
    that causes it to miscompile nnrpd/commands.c under -O, generating the
    error:

    Assembler: commands.c
    aline 1505 : Syntax error

    I had to get around this by cd'ing into nnrpd, running:

    make COPT=-g commands.o

    and then cd'ing back to the top level and running make again. On
    OpenServer, to build with cc, you have to use:

    env CC=cc CFLAGS=-O ./configure --with-sendmail=/usr/lib/sendmail

    Building under gcc is cleaner, but of course if you want to use
    --with-perl you want to build with the same compiler that you built Perl
    with.

    It's also worth noting that with a shared Perl library, Perl on this
    platform doesn't apparently generate the right link magic to include the
    path to the dynamic Perl libraries. You need to either set LD_RUN_PATH
    before building or LD_LIBRARY_PATH before running any binaries so that
    they can find the Perl libraries. (The former is preferred, since then
    the path is encoded into the binaries and you don't have to remember to
    set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles.

    In case you wish to only feed articles arrived on the old server
    between two dates, you can adapt the previous commands. For instance,
    the following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref headers. Current versions of INN
    will always add an Xref header, but very old versions (earlier 1.x
    versions) will only add an Xref header to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles
    to add an Xref header.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups: header
    is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using the tradspool storage method. The newsgroup of an article is stored in the Newsgroups header and Xref header of the article as stored on disk (and possibly in Followup-To), as well as determining where the overview
    information is stored, and in the case of tradspool is also encoded in the article's storage token. To rename a newsgroup in tradspool, stop the
    server, move the directory containing all of the articles to its
    appropriate new location in the news spool, edit every article to change
    the old name to the new name in Newsgroups, Followup-To, and Xref, create
    the new newsgroup with ctlinnd newgroup, and then rebuild history and
    overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups headers didn't break those links, you may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To headers of the articles in that newsgroup and then rebuild history and
    overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To headers
    to rename the newsgroup, and then create a file containing paths to all of
    the articles, one per line. You can then use that file as input to
    innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message IDs generated by INN will use the domain of the
    local system for the right-hand-side of the message ID. In some cases,
    this isn't desirable for various reasons (the server may have an internal
    name that doesn't make sense on Usenet at large, or one may not want to
    expose the name of the server).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will use
    that domain to generate message IDs. So if you need to change the domain
    used to generate message IDs for every local post from your server, just
    add virtualhost: and domain: keys to every access stanza in readers.conf.

    This is really overkill for this option, and eventually the domain:
    parameter in inn.conf will probably be changed to allow this to be
    modified for the whole server. (Right now, domain: in inn.conf means
    something completely different.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------

    Subject: 6.8. Generate MRTG graphs for INN

    INN's CNFS storage system has direct support for producing information
    suitable for MRTG graphs on the usage of the CNFS cycbuffs. Running
    cnfsstat -m <cycbuf> will generate output suitable for MRTG, and running cnfsstat -p will generate sample MRTG configuration fragments for each
    cycbuff.

    To generate MRTG graphs of the usage of the buffindexed overview system,
    try the following configuration fragment:

    Target[overview-BUFF]: `/usr/local/etc/mrtg/overview.sh`
    MaxBytes[overview-BUFF]: 100
    Title[overview-BUFF]: BUFF1 Usage
    Options[overview-BUFF]: growright gauge
    YLegend[overview-BUFF]: Overview Buffers
    ShortLegend[overview-BUFF]: %
    PageTop[overview-BUFF]: <H1>Usage of Overview Buffers</H1>
    <BR><TT>overview</TT>

    where the overview.sh script is:

    #!/bin/sh
    echo "100"
    <pathbin in inn.conf>/inndf -o | awk '{print $1}'
    echo "0"
    echo "overview"

    This sample configuration is from Basil Kruglov. Note that you can
    instead use -n (for total count of articles); in that case, you'll want to remove the MaxBytes setting above or change it to be some sensible limit
    on the total number of articles you receive. You'll also want to change a
    few of the other labels in the MRTG configuration.

    I'm not aware of any packaged solutions for generating MRTG data from
    other things, such as incoming or outcoming news flows. If anyone has any pointers, let me know.

    ------------------------------

    Subject: 6.9. Hide the junk and control groups from users

    The junk, control, and control.cancel groups must exist in the active file
    for the proper operation of INN, so you can't remove the groups entirely.
    You can, however, hide them completely from users.

    To do this, edit readers.conf, and for each user access group where you
    want to hide the junk and control groups, add "!junk,!control,!control.*"
    to the newsgroups pattern. In other words, if you have a line like:

    newsgroups: *

    just change that to:

    newsgroups: *,!junk,!control,!control.*

    If you use read and post patterns instead, do the same for each of them individually. The groups will then no longer show up on the server for
    users to which that access group applies; it will be as if they do not
    exist.

    ------------------------------

    Subject: 6.10. Modify the body of posts made through my server

    You can't without either making code changes to INN or putting your own software in the path of incoming posts. This is intentional.


    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Mon Jun 21 07:01:04 2021
    [continued from previous message]

    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports and to mail messages to moderators, and it assumes that you
    have a program installed as /usr/sbin/sendmail or /usr/lib/sendmail that
    it can use to do this. It does not speak SMTP, nor is it likely to ever
    speak SMTP; it's hard enough maintaining a package to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, the default cc requires -O be given when -Kalloca is
    given (which is added by default by configure since the parsers generated
    by bison need it). However, there appears to be a bug in the compiler
    that causes it to miscompile nnrpd/commands.c under -O, generating the
    error:

    Assembler: commands.c
    aline 1505 : Syntax error

    I had to get around this by cd'ing into nnrpd, running:

    make COPT=-g commands.o

    and then cd'ing back to the top level and running make again. On
    OpenServer, to build with cc, you have to use:

    env CC=cc CFLAGS=-O ./configure --with-sendmail=/usr/lib/sendmail

    Building under gcc is cleaner, but of course if you want to use
    --with-perl you want to build with the same compiler that you built Perl
    with.

    It's also worth noting that with a shared Perl library, Perl on this
    platform doesn't apparently generate the right link magic to include the
    path to the dynamic Perl libraries. You need to either set LD_RUN_PATH
    before building or LD_LIBRARY_PATH before running any binaries so that
    they can find the Perl libraries. (The former is preferred, since then
    the path is encoded into the binaries and you don't have to remember to
    set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    Note that rejected articles are remembered during the number of days
    specified by the /remember/ line in expire.ctl; so, in case you forgot
    to change the above parameters, you'll have to wait that number of
    days before being able to inject them again. You can of course set
    /remember/ to 0, run the expire process (for instance via news.daily),
    undo the change in expire.ctl and then start the feed again.

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles.

    In case you wish to only feed articles arrived on the old server
    between two dates, you can adapt the previous commands. For instance,
    the following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref headers. Current versions of INN
    will always add an Xref header, but very old versions (earlier 1.x
    versions) will only add an Xref header to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles
    to add an Xref header.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups: header
    is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using the tradspool storage method. The newsgroup of an article is stored in the Newsgroups header and Xref header of the article as stored on disk (and possibly in Followup-To), as well as determining where the overview
    information is stored, and in the case of tradspool is also encoded in the article's storage token. To rename a newsgroup in tradspool, stop the
    server, move the directory containing all of the articles to its
    appropriate new location in the news spool, edit every article to change
    the old name to the new name in Newsgroups, Followup-To, and Xref, create
    the new newsgroup with ctlinnd newgroup, and then rebuild history and
    overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups headers didn't break those links, you may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To headers of the articles in that newsgroup and then rebuild history and
    overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To headers
    to rename the newsgroup, and then create a file containing paths to all of
    the articles, one per line. You can then use that file as input to
    innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message IDs generated by INN will use the domain of the
    local system for the right-hand-side of the message ID. In some cases,
    this isn't desirable for various reasons (the server may have an internal
    name that doesn't make sense on Usenet at large, or one may not want to
    expose the name of the server).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will use
    that domain to generate message IDs. So if you need to change the domain
    used to generate message IDs for every local post from your server, just
    add virtualhost: and domain: keys to every access stanza in readers.conf.

    This is really overkill for this option, and eventually the domain:
    parameter in inn.conf will probably be changed to allow this to be
    modified for the whole server. (Right now, domain: in inn.conf means
    something completely different.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------

    Subject: 6.8. Generate MRTG graphs for INN

    INN's CNFS storage system has direct support for producing information
    suitable for MRTG graphs on the usage of the CNFS cycbuffs. Running
    cnfsstat -m <cycbuf> will generate output suitable for MRTG, and running cnfsstat -p will generate sample MRTG configuration fragments for each
    cycbuff.

    To generate MRTG graphs of the usage of the buffindexed overview system,
    try the following configuration fragment:

    Target[overview-BUFF]: `/usr/local/etc/mrtg/overview.sh`
    MaxBytes[overview-BUFF]: 100
    Title[overview-BUFF]: BUFF1 Usage
    Options[overview-BUFF]: growright gauge
    YLegend[overview-BUFF]: Overview Buffers
    ShortLegend[overview-BUFF]: %
    PageTop[overview-BUFF]: <H1>Usage of Overview Buffers</H1>
    <BR><TT>overview</TT>

    where the overview.sh script is:

    #!/bin/sh
    echo "100"
    <pathbin in inn.conf>/inndf -o | awk '{print $1}'
    echo "0"
    echo "overview"

    This sample configuration is from Basil Kruglov. Note that you can
    instead use -n (for total count of articles); in that case, you'll want to remove the MaxBytes setting above or change it to be some sensible limit
    on the total number of articles you receive. You'll also want to change a
    few of the other labels in the MRTG configuration.

    I'm not aware of any packaged solutions for generating MRTG data from
    other things, such as incoming or outcoming news flows. If anyone has any pointers, let me know.

    ------------------------------

    Subject: 6.9. Hide the junk and control groups from users

    The junk, control, and control.cancel groups must exist in the active file
    for the proper operation of INN, so you can't remove the groups entirely.
    You can, however, hide them completely from users.

    To do this, edit readers.conf, and for each user access group where you
    want to hide the junk and control groups, add "!junk,!control,!control.*"
    to the newsgroups pattern. In other words, if you have a line like:

    newsgroups: *

    just change that to:

    newsgroups: *,!junk,!control,!control.*

    If you use read and post patterns instead, do the same for each of them

    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Wed Jul 21 07:01:02 2021
    [continued from previous message]

    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports and to mail messages to moderators, and it assumes that you
    have a program installed as /usr/sbin/sendmail or /usr/lib/sendmail that
    it can use to do this. It does not speak SMTP, nor is it likely to ever
    speak SMTP; it's hard enough maintaining a package to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, the default cc requires -O be given when -Kalloca is
    given (which is added by default by configure since the parsers generated
    by bison need it). However, there appears to be a bug in the compiler
    that causes it to miscompile nnrpd/commands.c under -O, generating the
    error:

    Assembler: commands.c
    aline 1505 : Syntax error

    I had to get around this by cd'ing into nnrpd, running:

    make COPT=-g commands.o

    and then cd'ing back to the top level and running make again. On
    OpenServer, to build with cc, you have to use:

    env CC=cc CFLAGS=-O ./configure --with-sendmail=/usr/lib/sendmail

    Building under gcc is cleaner, but of course if you want to use
    --with-perl you want to build with the same compiler that you built Perl
    with.

    It's also worth noting that with a shared Perl library, Perl on this
    platform doesn't apparently generate the right link magic to include the
    path to the dynamic Perl libraries. You need to either set LD_RUN_PATH
    before building or LD_LIBRARY_PATH before running any binaries so that
    they can find the Perl libraries. (The former is preferred, since then
    the path is encoded into the binaries and you don't have to remember to
    set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    Note that rejected articles are remembered during the number of days
    specified by the /remember/ line in expire.ctl; so, in case you forgot
    to change the above parameters, you'll have to wait that number of
    days before being able to inject them again. You can of course set
    /remember/ to 0, run the expire process (for instance via news.daily),
    undo the change in expire.ctl and then start the feed again.

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles.

    In case you wish to only feed articles arrived on the old server
    between two dates, you can adapt the previous commands. For instance,
    the following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref headers. Current versions of INN
    will always add an Xref header, but very old versions (earlier 1.x
    versions) will only add an Xref header to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles
    to add an Xref header.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups: header
    is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using the tradspool storage method. The newsgroup of an article is stored in the Newsgroups header and Xref header of the article as stored on disk (and possibly in Followup-To), as well as determining where the overview
    information is stored, and in the case of tradspool is also encoded in the article's storage token. To rename a newsgroup in tradspool, stop the
    server, move the directory containing all of the articles to its
    appropriate new location in the news spool, edit every article to change
    the old name to the new name in Newsgroups, Followup-To, and Xref, create
    the new newsgroup with ctlinnd newgroup, and then rebuild history and
    overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups headers didn't break those links, you may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To headers of the articles in that newsgroup and then rebuild history and
    overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To headers
    to rename the newsgroup, and then create a file containing paths to all of
    the articles, one per line. You can then use that file as input to
    innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message IDs generated by INN will use the domain of the
    local system for the right-hand-side of the message ID. In some cases,
    this isn't desirable for various reasons (the server may have an internal
    name that doesn't make sense on Usenet at large, or one may not want to
    expose the name of the server).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will use
    that domain to generate message IDs. So if you need to change the domain
    used to generate message IDs for every local post from your server, just
    add virtualhost: and domain: keys to every access stanza in readers.conf.

    This is really overkill for this option, and eventually the domain:
    parameter in inn.conf will probably be changed to allow this to be
    modified for the whole server. (Right now, domain: in inn.conf means
    something completely different.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------

    Subject: 6.8. Generate MRTG graphs for INN

    INN's CNFS storage system has direct support for producing information
    suitable for MRTG graphs on the usage of the CNFS cycbuffs. Running
    cnfsstat -m <cycbuf> will generate output suitable for MRTG, and running cnfsstat -p will generate sample MRTG configuration fragments for each
    cycbuff.

    To generate MRTG graphs of the usage of the buffindexed overview system,
    try the following configuration fragment:

    Target[overview-BUFF]: `/usr/local/etc/mrtg/overview.sh`
    MaxBytes[overview-BUFF]: 100
    Title[overview-BUFF]: BUFF1 Usage
    Options[overview-BUFF]: growright gauge
    YLegend[overview-BUFF]: Overview Buffers
    ShortLegend[overview-BUFF]: %
    PageTop[overview-BUFF]: <H1>Usage of Overview Buffers</H1>
    <BR><TT>overview</TT>

    where the overview.sh script is:

    #!/bin/sh
    echo "100"
    <pathbin in inn.conf>/inndf -o | awk '{print $1}'
    echo "0"
    echo "overview"

    This sample configuration is from Basil Kruglov. Note that you can
    instead use -n (for total count of articles); in that case, you'll want to remove the MaxBytes setting above or change it to be some sensible limit
    on the total number of articles you receive. You'll also want to change a
    few of the other labels in the MRTG configuration.

    I'm not aware of any packaged solutions for generating MRTG data from
    other things, such as incoming or outcoming news flows. If anyone has any pointers, let me know.

    ------------------------------

    Subject: 6.9. Hide the junk and control groups from users

    The junk, control, and control.cancel groups must exist in the active file
    for the proper operation of INN, so you can't remove the groups entirely.
    You can, however, hide them completely from users.

    To do this, edit readers.conf, and for each user access group where you
    want to hide the junk and control groups, add "!junk,!control,!control.*"
    to the newsgroups pattern. In other words, if you have a line like:

    newsgroups: *

    just change that to:

    newsgroups: *,!junk,!control,!control.*

    If you use read and post patterns instead, do the same for each of them

    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Sat Aug 21 07:01:03 2021
    [continued from previous message]

    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports and to mail messages to moderators, and it assumes that you
    have a program installed as /usr/sbin/sendmail or /usr/lib/sendmail that
    it can use to do this. It does not speak SMTP, nor is it likely to ever
    speak SMTP; it's hard enough maintaining a package to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, the default cc requires -O be given when -Kalloca is
    given (which is added by default by configure since the parsers generated
    by bison need it). However, there appears to be a bug in the compiler
    that causes it to miscompile nnrpd/commands.c under -O, generating the
    error:

    Assembler: commands.c
    aline 1505 : Syntax error

    I had to get around this by cd'ing into nnrpd, running:

    make COPT=-g commands.o

    and then cd'ing back to the top level and running make again. On
    OpenServer, to build with cc, you have to use:

    env CC=cc CFLAGS=-O ./configure --with-sendmail=/usr/lib/sendmail

    Building under gcc is cleaner, but of course if you want to use
    --with-perl you want to build with the same compiler that you built Perl
    with.

    It's also worth noting that with a shared Perl library, Perl on this
    platform doesn't apparently generate the right link magic to include the
    path to the dynamic Perl libraries. You need to either set LD_RUN_PATH
    before building or LD_LIBRARY_PATH before running any binaries so that
    they can find the Perl libraries. (The former is preferred, since then
    the path is encoded into the binaries and you don't have to remember to
    set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    Note that rejected articles are remembered during the number of days
    specified by the /remember/ line in expire.ctl; so, in case you forgot
    to change the above parameters, you'll have to wait that number of
    days before being able to inject them again. You can of course set
    /remember/ to 0, run the expire process (for instance via news.daily),
    undo the change in expire.ctl and then start the feed again.

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles.

    In case you wish to only feed articles arrived on the old server
    between two dates, you can adapt the previous commands. For instance,
    the following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref headers. Current versions of INN
    will always add an Xref header, but very old versions (earlier 1.x
    versions) will only add an Xref header to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles
    to add an Xref header.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups: header
    is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using the tradspool storage method. The newsgroup of an article is stored in the Newsgroups header and Xref header of the article as stored on disk (and possibly in Followup-To), as well as determining where the overview
    information is stored, and in the case of tradspool is also encoded in the article's storage token. To rename a newsgroup in tradspool, stop the
    server, move the directory containing all of the articles to its
    appropriate new location in the news spool, edit every article to change
    the old name to the new name in Newsgroups, Followup-To, and Xref, create
    the new newsgroup with ctlinnd newgroup, and then rebuild history and
    overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups headers didn't break those links, you may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To headers of the articles in that newsgroup and then rebuild history and
    overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To headers
    to rename the newsgroup, and then create a file containing paths to all of
    the articles, one per line. You can then use that file as input to
    innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message IDs generated by INN will use the domain of the
    local system for the right-hand-side of the message ID. In some cases,
    this isn't desirable for various reasons (the server may have an internal
    name that doesn't make sense on Usenet at large, or one may not want to
    expose the name of the server).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will use
    that domain to generate message IDs. So if you need to change the domain
    used to generate message IDs for every local post from your server, just
    add virtualhost: and domain: keys to every access stanza in readers.conf.

    This is really overkill for this option, and eventually the domain:
    parameter in inn.conf will probably be changed to allow this to be
    modified for the whole server. (Right now, domain: in inn.conf means
    something completely different.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------

    Subject: 6.8. Generate MRTG graphs for INN

    INN's CNFS storage system has direct support for producing information
    suitable for MRTG graphs on the usage of the CNFS cycbuffs. Running
    cnfsstat -m <cycbuf> will generate output suitable for MRTG, and running cnfsstat -p will generate sample MRTG configuration fragments for each
    cycbuff.

    To generate MRTG graphs of the usage of the buffindexed overview system,
    try the following configuration fragment:

    Target[overview-BUFF]: `/usr/local/etc/mrtg/overview.sh`
    MaxBytes[overview-BUFF]: 100
    Title[overview-BUFF]: BUFF1 Usage
    Options[overview-BUFF]: growright gauge
    YLegend[overview-BUFF]: Overview Buffers
    ShortLegend[overview-BUFF]: %
    PageTop[overview-BUFF]: <H1>Usage of Overview Buffers</H1>
    <BR><TT>overview</TT>

    where the overview.sh script is:

    #!/bin/sh
    echo "100"
    <pathbin in inn.conf>/inndf -o | awk '{print $1}'
    echo "0"
    echo "overview"

    This sample configuration is from Basil Kruglov. Note that you can
    instead use -n (for total count of articles); in that case, you'll want to remove the MaxBytes setting above or change it to be some sensible limit
    on the total number of articles you receive. You'll also want to change a
    few of the other labels in the MRTG configuration.

    I'm not aware of any packaged solutions for generating MRTG data from
    other things, such as incoming or outcoming news flows. If anyone has any pointers, let me know.

    ------------------------------

    Subject: 6.9. Hide the junk and control groups from users

    The junk, control, and control.cancel groups must exist in the active file
    for the proper operation of INN, so you can't remove the groups entirely.
    You can, however, hide them completely from users.

    To do this, edit readers.conf, and for each user access group where you
    want to hide the junk and control groups, add "!junk,!control,!control.*"
    to the newsgroups pattern. In other words, if you have a line like:

    newsgroups: *

    just change that to:

    newsgroups: *,!junk,!control,!control.*

    If you use read and post patterns instead, do the same for each of them

    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Tue Sep 21 07:01:03 2021
    [continued from previous message]

    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports and to mail messages to moderators, and it assumes that you
    have a program installed as /usr/sbin/sendmail or /usr/lib/sendmail that
    it can use to do this. It does not speak SMTP, nor is it likely to ever
    speak SMTP; it's hard enough maintaining a package to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, the default cc requires -O be given when -Kalloca is
    given (which is added by default by configure since the parsers generated
    by bison need it). However, there appears to be a bug in the compiler
    that causes it to miscompile nnrpd/commands.c under -O, generating the
    error:

    Assembler: commands.c
    aline 1505 : Syntax error

    I had to get around this by cd'ing into nnrpd, running:

    make COPT=-g commands.o

    and then cd'ing back to the top level and running make again. On
    OpenServer, to build with cc, you have to use:

    env CC=cc CFLAGS=-O ./configure --with-sendmail=/usr/lib/sendmail

    Building under gcc is cleaner, but of course if you want to use
    --with-perl you want to build with the same compiler that you built Perl
    with.

    It's also worth noting that with a shared Perl library, Perl on this
    platform doesn't apparently generate the right link magic to include the
    path to the dynamic Perl libraries. You need to either set LD_RUN_PATH
    before building or LD_LIBRARY_PATH before running any binaries so that
    they can find the Perl libraries. (The former is preferred, since then
    the path is encoded into the binaries and you don't have to remember to
    set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    Note that rejected articles are remembered during the number of days
    specified by the /remember/ line in expire.ctl; so, in case you forgot
    to change the above parameters, you'll have to wait that number of
    days before being able to inject them again. You can of course set
    /remember/ to 0, run the expire process (for instance via news.daily),
    undo the change in expire.ctl and then start the feed again.

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles.

    In case you wish to only feed articles arrived on the old server
    between two dates, you can adapt the previous commands. For instance,
    the following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref headers. Current versions of INN
    will always add an Xref header, but very old versions (earlier 1.x
    versions) will only add an Xref header to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles
    to add an Xref header.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups: header
    is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using the tradspool storage method. The newsgroup of an article is stored in the Newsgroups header and Xref header of the article as stored on disk (and possibly in Followup-To), as well as determining where the overview
    information is stored, and in the case of tradspool is also encoded in the article's storage token. To rename a newsgroup in tradspool, stop the
    server, move the directory containing all of the articles to its
    appropriate new location in the news spool, edit every article to change
    the old name to the new name in Newsgroups, Followup-To, and Xref, create
    the new newsgroup with ctlinnd newgroup, and then rebuild history and
    overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups headers didn't break those links, you may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To headers of the articles in that newsgroup and then rebuild history and
    overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To headers
    to rename the newsgroup, and then create a file containing paths to all of
    the articles, one per line. You can then use that file as input to
    innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message IDs generated by INN will use the domain of the
    local system for the right-hand-side of the message ID. In some cases,
    this isn't desirable for various reasons (the server may have an internal
    name that doesn't make sense on Usenet at large, or one may not want to
    expose the name of the server).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will use
    that domain to generate message IDs. So if you need to change the domain
    used to generate message IDs for every local post from your server, just
    add virtualhost: and domain: keys to every access stanza in readers.conf.

    This is really overkill for this option, and eventually the domain:
    parameter in inn.conf will probably be changed to allow this to be
    modified for the whole server. (Right now, domain: in inn.conf means
    something completely different.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------

    Subject: 6.8. Generate MRTG graphs for INN

    INN's CNFS storage system has direct support for producing information
    suitable for MRTG graphs on the usage of the CNFS cycbuffs. Running
    cnfsstat -m <cycbuf> will generate output suitable for MRTG, and running cnfsstat -p will generate sample MRTG configuration fragments for each
    cycbuff.

    To generate MRTG graphs of the usage of the buffindexed overview system,
    try the following configuration fragment:

    Target[overview-BUFF]: `/usr/local/etc/mrtg/overview.sh`
    MaxBytes[overview-BUFF]: 100
    Title[overview-BUFF]: BUFF1 Usage
    Options[overview-BUFF]: growright gauge
    YLegend[overview-BUFF]: Overview Buffers
    ShortLegend[overview-BUFF]: %
    PageTop[overview-BUFF]: <H1>Usage of Overview Buffers</H1>
    <BR><TT>overview</TT>

    where the overview.sh script is:

    #!/bin/sh
    echo "100"
    <pathbin in inn.conf>/inndf -o | awk '{print $1}'
    echo "0"
    echo "overview"

    This sample configuration is from Basil Kruglov. Note that you can
    instead use -n (for total count of articles); in that case, you'll want to remove the MaxBytes setting above or change it to be some sensible limit
    on the total number of articles you receive. You'll also want to change a
    few of the other labels in the MRTG configuration.

    I'm not aware of any packaged solutions for generating MRTG data from
    other things, such as incoming or outcoming news flows. If anyone has any pointers, let me know.

    ------------------------------

    Subject: 6.9. Hide the junk and control groups from users

    The junk, control, and control.cancel groups must exist in the active file
    for the proper operation of INN, so you can't remove the groups entirely.
    You can, however, hide them completely from users.

    To do this, edit readers.conf, and for each user access group where you
    want to hide the junk and control groups, add "!junk,!control,!control.*"
    to the newsgroups pattern. In other words, if you have a line like:

    newsgroups: *

    just change that to:

    newsgroups: *,!junk,!control,!control.*

    If you use read and post patterns instead, do the same for each of them

    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Thu Oct 21 07:01:04 2021
    [continued from previous message]

    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports and to mail messages to moderators, and it assumes that you
    have a program installed as /usr/sbin/sendmail or /usr/lib/sendmail that
    it can use to do this. It does not speak SMTP, nor is it likely to ever
    speak SMTP; it's hard enough maintaining a package to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, the default cc requires -O be given when -Kalloca is
    given (which is added by default by configure since the parsers generated
    by bison need it). However, there appears to be a bug in the compiler
    that causes it to miscompile nnrpd/commands.c under -O, generating the
    error:

    Assembler: commands.c
    aline 1505 : Syntax error

    I had to get around this by cd'ing into nnrpd, running:

    make COPT=-g commands.o

    and then cd'ing back to the top level and running make again. On
    OpenServer, to build with cc, you have to use:

    env CC=cc CFLAGS=-O ./configure --with-sendmail=/usr/lib/sendmail

    Building under gcc is cleaner, but of course if you want to use
    --with-perl you want to build with the same compiler that you built Perl
    with.

    It's also worth noting that with a shared Perl library, Perl on this
    platform doesn't apparently generate the right link magic to include the
    path to the dynamic Perl libraries. You need to either set LD_RUN_PATH
    before building or LD_LIBRARY_PATH before running any binaries so that
    they can find the Perl libraries. (The former is preferred, since then
    the path is encoded into the binaries and you don't have to remember to
    set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    Note that rejected articles are remembered during the number of days
    specified by the /remember/ line in expire.ctl; so, in case you forgot
    to change the above parameters, you'll have to wait that number of
    days before being able to inject them again. You can of course set
    /remember/ to 0, run the expire process (for instance via news.daily),
    undo the change in expire.ctl and then start the feed again.

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles.

    In case you wish to only feed articles arrived on the old server
    between two dates, you can adapt the previous commands. For instance,
    the following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref headers. Current versions of INN
    will always add an Xref header, but very old versions (earlier 1.x
    versions) will only add an Xref header to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles
    to add an Xref header.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups: header
    is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using the tradspool storage method. The newsgroup of an article is stored in the Newsgroups header and Xref header of the article as stored on disk (and possibly in Followup-To), as well as determining where the overview
    information is stored, and in the case of tradspool is also encoded in the article's storage token. To rename a newsgroup in tradspool, stop the
    server, move the directory containing all of the articles to its
    appropriate new location in the news spool, edit every article to change
    the old name to the new name in Newsgroups, Followup-To, and Xref, create
    the new newsgroup with ctlinnd newgroup, and then rebuild history and
    overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups headers didn't break those links, you may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To headers of the articles in that newsgroup and then rebuild history and
    overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To headers
    to rename the newsgroup, and then create a file containing paths to all of
    the articles, one per line. You can then use that file as input to
    innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message IDs generated by INN will use the domain of the
    local system for the right-hand-side of the message ID. In some cases,
    this isn't desirable for various reasons (the server may have an internal
    name that doesn't make sense on Usenet at large, or one may not want to
    expose the name of the server).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will use
    that domain to generate message IDs. So if you need to change the domain
    used to generate message IDs for every local post from your server, just
    add virtualhost: and domain: keys to every access stanza in readers.conf.

    This is really overkill for this option, and eventually the domain:
    parameter in inn.conf will probably be changed to allow this to be
    modified for the whole server. (Right now, domain: in inn.conf means
    something completely different.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------

    Subject: 6.8. Generate MRTG graphs for INN

    INN's CNFS storage system has direct support for producing information
    suitable for MRTG graphs on the usage of the CNFS cycbuffs. Running
    cnfsstat -m <cycbuf> will generate output suitable for MRTG, and running cnfsstat -p will generate sample MRTG configuration fragments for each
    cycbuff.

    To generate MRTG graphs of the usage of the buffindexed overview system,
    try the following configuration fragment:

    Target[overview-BUFF]: `/usr/local/etc/mrtg/overview.sh`
    MaxBytes[overview-BUFF]: 100
    Title[overview-BUFF]: BUFF1 Usage
    Options[overview-BUFF]: growright gauge
    YLegend[overview-BUFF]: Overview Buffers
    ShortLegend[overview-BUFF]: %
    PageTop[overview-BUFF]: <H1>Usage of Overview Buffers</H1>
    <BR><TT>overview</TT>

    where the overview.sh script is:

    #!/bin/sh
    echo "100"
    <pathbin in inn.conf>/inndf -o | awk '{print $1}'
    echo "0"
    echo "overview"

    This sample configuration is from Basil Kruglov. Note that you can
    instead use -n (for total count of articles); in that case, you'll want to remove the MaxBytes setting above or change it to be some sensible limit
    on the total number of articles you receive. You'll also want to change a
    few of the other labels in the MRTG configuration.

    I'm not aware of any packaged solutions for generating MRTG data from
    other things, such as incoming or outcoming news flows. If anyone has any pointers, let me know.

    ------------------------------

    Subject: 6.9. Hide the junk and control groups from users

    The junk, control, and control.cancel groups must exist in the active file
    for the proper operation of INN, so you can't remove the groups entirely.
    You can, however, hide them completely from users.

    To do this, edit readers.conf, and for each user access group where you
    want to hide the junk and control groups, add "!junk,!control,!control.*"
    to the newsgroups pattern. In other words, if you have a line like:

    newsgroups: *

    just change that to:

    newsgroups: *,!junk,!control,!control.*

    If you use read and post patterns instead, do the same for each of them

    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Sun Nov 21 08:01:02 2021
    [continued from previous message]

    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports and to mail messages to moderators, and it assumes that you
    have a program installed as /usr/sbin/sendmail or /usr/lib/sendmail that
    it can use to do this. It does not speak SMTP, nor is it likely to ever
    speak SMTP; it's hard enough maintaining a package to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, the default cc requires -O be given when -Kalloca is
    given (which is added by default by configure since the parsers generated
    by bison need it). However, there appears to be a bug in the compiler
    that causes it to miscompile nnrpd/commands.c under -O, generating the
    error:

    Assembler: commands.c
    aline 1505 : Syntax error

    I had to get around this by cd'ing into nnrpd, running:

    make COPT=-g commands.o

    and then cd'ing back to the top level and running make again. On
    OpenServer, to build with cc, you have to use:

    env CC=cc CFLAGS=-O ./configure --with-sendmail=/usr/lib/sendmail

    Building under gcc is cleaner, but of course if you want to use
    --with-perl you want to build with the same compiler that you built Perl
    with.

    It's also worth noting that with a shared Perl library, Perl on this
    platform doesn't apparently generate the right link magic to include the
    path to the dynamic Perl libraries. You need to either set LD_RUN_PATH
    before building or LD_LIBRARY_PATH before running any binaries so that
    they can find the Perl libraries. (The former is preferred, since then
    the path is encoded into the binaries and you don't have to remember to
    set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    Note that rejected articles are remembered during the number of days
    specified by the /remember/ line in expire.ctl; so, in case you forgot
    to change the above parameters, you'll have to wait that number of
    days before being able to inject them again. You can of course set
    /remember/ to 0, run the expire process (for instance via news.daily),
    undo the change in expire.ctl and then start the feed again.

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles.

    In case you wish to only feed articles arrived on the old server
    between two dates, you can adapt the previous commands. For instance,
    the following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref headers. Current versions of INN
    will always add an Xref header, but very old versions (earlier 1.x
    versions) will only add an Xref header to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles
    to add an Xref header.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups: header
    is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using the tradspool storage method. The newsgroup of an article is stored in the Newsgroups header and Xref header of the article as stored on disk (and possibly in Followup-To), as well as determining where the overview
    information is stored, and in the case of tradspool is also encoded in the article's storage token. To rename a newsgroup in tradspool, stop the
    server, move the directory containing all of the articles to its
    appropriate new location in the news spool, edit every article to change
    the old name to the new name in Newsgroups, Followup-To, and Xref, create
    the new newsgroup with ctlinnd newgroup, and then rebuild history and
    overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups headers didn't break those links, you may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To headers of the articles in that newsgroup and then rebuild history and
    overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To headers
    to rename the newsgroup, and then create a file containing paths to all of
    the articles, one per line. You can then use that file as input to
    innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message IDs generated by INN will use the domain of the
    local system for the right-hand-side of the message ID. In some cases,
    this isn't desirable for various reasons (the server may have an internal
    name that doesn't make sense on Usenet at large, or one may not want to
    expose the name of the server).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will use
    that domain to generate message IDs. So if you need to change the domain
    used to generate message IDs for every local post from your server, just
    add virtualhost: and domain: keys to every access stanza in readers.conf.

    This is really overkill for this option, and eventually the domain:
    parameter in inn.conf will probably be changed to allow this to be
    modified for the whole server. (Right now, domain: in inn.conf means
    something completely different.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------

    Subject: 6.8. Generate MRTG graphs for INN

    INN's CNFS storage system has direct support for producing information
    suitable for MRTG graphs on the usage of the CNFS cycbuffs. Running
    cnfsstat -m <cycbuf> will generate output suitable for MRTG, and running cnfsstat -p will generate sample MRTG configuration fragments for each
    cycbuff.

    To generate MRTG graphs of the usage of the buffindexed overview system,
    try the following configuration fragment:

    Target[overview-BUFF]: `/usr/local/etc/mrtg/overview.sh`
    MaxBytes[overview-BUFF]: 100
    Title[overview-BUFF]: BUFF1 Usage
    Options[overview-BUFF]: growright gauge
    YLegend[overview-BUFF]: Overview Buffers
    ShortLegend[overview-BUFF]: %
    PageTop[overview-BUFF]: <H1>Usage of Overview Buffers</H1>
    <BR><TT>overview</TT>

    where the overview.sh script is:

    #!/bin/sh
    echo "100"
    <pathbin in inn.conf>/inndf -o | awk '{print $1}'
    echo "0"
    echo "overview"

    This sample configuration is from Basil Kruglov. Note that you can
    instead use -n (for total count of articles); in that case, you'll want to remove the MaxBytes setting above or change it to be some sensible limit
    on the total number of articles you receive. You'll also want to change a
    few of the other labels in the MRTG configuration.

    I'm not aware of any packaged solutions for generating MRTG data from
    other things, such as incoming or outcoming news flows. If anyone has any pointers, let me know.

    ------------------------------

    Subject: 6.9. Hide the junk and control groups from users

    The junk, control, and control.cancel groups must exist in the active file
    for the proper operation of INN, so you can't remove the groups entirely.
    You can, however, hide them completely from users.

    To do this, edit readers.conf, and for each user access group where you
    want to hide the junk and control groups, add "!junk,!control,!control.*"
    to the newsgroups pattern. In other words, if you have a line like:

    newsgroups: *

    just change that to:

    newsgroups: *,!junk,!control,!control.*

    If you use read and post patterns instead, do the same for each of them

    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Tue Dec 21 08:01:02 2021
    [continued from previous message]

    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports and to mail messages to moderators, and it assumes that you
    have a program installed as /usr/sbin/sendmail or /usr/lib/sendmail that
    it can use to do this. It does not speak SMTP, nor is it likely to ever
    speak SMTP; it's hard enough maintaining a package to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, the default cc requires -O be given when -Kalloca is
    given (which is added by default by configure since the parsers generated
    by bison need it). However, there appears to be a bug in the compiler
    that causes it to miscompile nnrpd/commands.c under -O, generating the
    error:

    Assembler: commands.c
    aline 1505 : Syntax error

    I had to get around this by cd'ing into nnrpd, running:

    make COPT=-g commands.o

    and then cd'ing back to the top level and running make again. On
    OpenServer, to build with cc, you have to use:

    env CC=cc CFLAGS=-O ./configure --with-sendmail=/usr/lib/sendmail

    Building under gcc is cleaner, but of course if you want to use
    --with-perl you want to build with the same compiler that you built Perl
    with.

    It's also worth noting that with a shared Perl library, Perl on this
    platform doesn't apparently generate the right link magic to include the
    path to the dynamic Perl libraries. You need to either set LD_RUN_PATH
    before building or LD_LIBRARY_PATH before running any binaries so that
    they can find the Perl libraries. (The former is preferred, since then
    the path is encoded into the binaries and you don't have to remember to
    set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    Note that rejected articles are remembered during the number of days
    specified by the /remember/ line in expire.ctl; so, in case you forgot
    to change the above parameters, you'll have to wait that number of
    days before being able to inject them again. You can of course set
    /remember/ to 0, run the expire process (for instance via news.daily),
    undo the change in expire.ctl and then start the feed again.

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles.

    In case you wish to only feed articles arrived on the old server
    between two dates, you can adapt the previous commands. For instance,
    the following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref headers. Current versions of INN
    will always add an Xref header, but very old versions (earlier 1.x
    versions) will only add an Xref header to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles
    to add an Xref header.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups: header
    is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using the tradspool storage method. The newsgroup of an article is stored in the Newsgroups header and Xref header of the article as stored on disk (and possibly in Followup-To), as well as determining where the overview
    information is stored, and in the case of tradspool is also encoded in the article's storage token. To rename a newsgroup in tradspool, stop the
    server, move the directory containing all of the articles to its
    appropriate new location in the news spool, edit every article to change
    the old name to the new name in Newsgroups, Followup-To, and Xref, create
    the new newsgroup with ctlinnd newgroup, and then rebuild history and
    overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups headers didn't break those links, you may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To headers of the articles in that newsgroup and then rebuild history and
    overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To headers
    to rename the newsgroup, and then create a file containing paths to all of
    the articles, one per line. You can then use that file as input to
    innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message IDs generated by INN will use the domain of the
    local system for the right-hand-side of the message ID. In some cases,
    this isn't desirable for various reasons (the server may have an internal
    name that doesn't make sense on Usenet at large, or one may not want to
    expose the name of the server).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will use
    that domain to generate message IDs. So if you need to change the domain
    used to generate message IDs for every local post from your server, just
    add virtualhost: and domain: keys to every access stanza in readers.conf.

    This is really overkill for this option, and eventually the domain:
    parameter in inn.conf will probably be changed to allow this to be
    modified for the whole server. (Right now, domain: in inn.conf means
    something completely different.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------

    Subject: 6.8. Generate MRTG graphs for INN

    INN's CNFS storage system has direct support for producing information
    suitable for MRTG graphs on the usage of the CNFS cycbuffs. Running
    cnfsstat -m <cycbuf> will generate output suitable for MRTG, and running cnfsstat -p will generate sample MRTG configuration fragments for each
    cycbuff.

    To generate MRTG graphs of the usage of the buffindexed overview system,
    try the following configuration fragment:

    Target[overview-BUFF]: `/usr/local/etc/mrtg/overview.sh`
    MaxBytes[overview-BUFF]: 100
    Title[overview-BUFF]: BUFF1 Usage
    Options[overview-BUFF]: growright gauge
    YLegend[overview-BUFF]: Overview Buffers
    ShortLegend[overview-BUFF]: %
    PageTop[overview-BUFF]: <H1>Usage of Overview Buffers</H1>
    <BR><TT>overview</TT>

    where the overview.sh script is:

    #!/bin/sh
    echo "100"
    <pathbin in inn.conf>/inndf -o | awk '{print $1}'
    echo "0"
    echo "overview"

    This sample configuration is from Basil Kruglov. Note that you can
    instead use -n (for total count of articles); in that case, you'll want to remove the MaxBytes setting above or change it to be some sensible limit
    on the total number of articles you receive. You'll also want to change a
    few of the other labels in the MRTG configuration.

    I'm not aware of any packaged solutions for generating MRTG data from
    other things, such as incoming or outcoming news flows. If anyone has any pointers, let me know.

    ------------------------------

    Subject: 6.9. Hide the junk and control groups from users

    The junk, control, and control.cancel groups must exist in the active file
    for the proper operation of INN, so you can't remove the groups entirely.
    You can, however, hide them completely from users.

    To do this, edit readers.conf, and for each user access group where you
    want to hide the junk and control groups, add "!junk,!control,!control.*"
    to the newsgroups pattern. In other words, if you have a line like:

    newsgroups: *

    just change that to:

    newsgroups: *,!junk,!control,!control.*

    If you use read and post patterns instead, do the same for each of them

    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Fri Jan 21 08:01:03 2022
    [continued from previous message]

    investigate further. INN does not generally log enough information
    about outgoing articles to be able to tell more from your server alone.

    It may be possible to get a slight bit of information from the remote
    server by connecting with telnet (usually to port 119) and issuing
    "IHAVE <message-id>". The peer may respond with something like "435
    Duplicate" which means that the problem is not likely to be with your
    server (it may be still a problem with the article itself). If the
    peer responds with something like "335", your server probably did not
    offer the article after all.

    If you really are at a dead end and need to get more information about
    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports and to mail messages to moderators, and it assumes that you
    have a program installed as /usr/sbin/sendmail or /usr/lib/sendmail that
    it can use to do this. It does not speak SMTP, nor is it likely to ever
    speak SMTP; it's hard enough maintaining a package to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header field (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header field.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, it's worth noting that with a shared Perl library,
    Perl on this platform doesn't apparently generate the right link magic
    to include the path to the dynamic Perl libraries. You need to either
    set LD_RUN_PATH before building or LD_LIBRARY_PATH before running any
    binaries so that they can find the Perl libraries. (The former is
    preferred, since then the path is encoded into the binaries and you
    don't have to remember to set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    Note that rejected articles are remembered during the number of days
    specified by the /remember/ line in expire.ctl; so, in case you forgot
    to change the above parameters, you'll have to wait that number of
    days before being able to inject them again. You can of course set
    /remember/ to 0, run the expire process (for instance via news.daily
    called with the same parameters as in crontab, plus "notdaily"),
    undo the change in expire.ctl and then start the feed again.

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles.

    In case you wish to only feed articles arrived on the old server
    between two dates, you can adapt the previous commands. For instance,
    the following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref header fields. Current versions of INN will always add an Xref header field, but very old versions (earlier 1.x versions) will only add an Xref header field to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles to add an Xref header field.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups header field
    body is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using
    the tradspool storage method. The newsgroup of an article is stored in
    the Newsgroups header field and in the Xref header field of the article
    as stored on disk (and possibly in Followup-To), as well as determining
    where the overview information is stored, and in the case of tradspool
    is also encoded in the article's storage token. To rename a newsgroup
    in tradspool, stop the server, move the directory containing all of
    the articles to its appropriate new location in the news spool, edit
    every article to change the old name to the new name in Newsgroups, Followup-To, and Xref, create the new newsgroup with ctlinnd newgroup,
    and then rebuild history and overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups header fields didn't break those links, you
    may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To header fields of the articles in that newsgroup and then rebuild history
    and overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To header
    fields to rename the newsgroup, and then create a file containing paths to
    all of the articles, one per line. You can then use that file as input to innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message IDs generated by INN will use the domain of the
    local system for the right-hand-side of the message ID. In some cases,
    this isn't desirable for various reasons (the server may have an internal
    name that doesn't make sense on Usenet at large, or one may not want to
    expose the name of the server).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will use
    that domain to generate message IDs. So if you need to change the domain
    used to generate message IDs for every local post from your server, just
    add virtualhost: and domain: keys to every access stanza in readers.conf.

    This is really overkill for this option, and eventually the domain:
    parameter in inn.conf will probably be changed to allow this to be
    modified for the whole server. (Right now, domain: in inn.conf means
    something completely different.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------

    Subject: 6.8. Generate MRTG graphs for INN

    INN's CNFS storage system has direct support for producing information
    suitable for MRTG graphs on the usage of the CNFS cycbuffs. Running
    cnfsstat -m <cycbuf> will generate output suitable for MRTG, and running cnfsstat -p will generate sample MRTG configuration fragments for each
    cycbuff.

    To generate MRTG graphs of the usage of the buffindexed overview system,
    try the following configuration fragment:

    Target[overview-BUFF]: `/usr/local/etc/mrtg/overview.sh`
    MaxBytes[overview-BUFF]: 100
    Title[overview-BUFF]: BUFF1 Usage
    Options[overview-BUFF]: growright gauge
    YLegend[overview-BUFF]: Overview Buffers
    ShortLegend[overview-BUFF]: %
    PageTop[overview-BUFF]: <H1>Usage of Overview Buffers</H1>
    <BR><TT>overview</TT>

    where the overview.sh script is:

    #!/bin/sh
    echo "100"
    <pathbin in inn.conf>/inndf -o | awk '{print $1}'
    echo "0"
    echo "overview"

    This sample configuration is from Basil Kruglov. Note that you can
    instead use -n (for total count of articles); in that case, you'll want to remove the MaxBytes setting above or change it to be some sensible limit
    on the total number of articles you receive. You'll also want to change a
    few of the other labels in the MRTG configuration.

    I'm not aware of any packaged solutions for generating MRTG data from
    other things, such as incoming or outcoming news flows. If anyone has any pointers, let me know.

    ------------------------------

    Subject: 6.9. Hide the junk and control groups from users

    The junk, control, and control.cancel groups must exist in the active file
    for the proper operation of INN, so you can't remove the groups entirely.
    You can, however, hide them completely from users.

    To do this, edit readers.conf, and for each user access group where you
    want to hide the junk and control groups, add "!junk,!control,!control.*"
    to the newsgroups pattern. In other words, if you have a line like:

    newsgroups: *

    just change that to:

    newsgroups: *,!junk,!control,!control.*


    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Mon Feb 21 08:01:02 2022
    [continued from previous message]

    and not spooled, you will need the assistance of the peer's admin to
    investigate further. INN does not generally log enough information
    about outgoing articles to be able to tell more from your server alone.

    It may be possible to get a slight bit of information from the remote
    server by connecting with telnet (usually to port 119) and issuing
    "IHAVE <message-id>". The peer may respond with something like "435
    Duplicate" which means that the problem is not likely to be with your
    server (it may be still a problem with the article itself). If the
    peer responds with something like "335", your server probably did not
    offer the article after all.

    If you really are at a dead end and need to get more information about
    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports and to mail messages to moderators, and it assumes that you
    have a program installed as /usr/sbin/sendmail or /usr/lib/sendmail that
    it can use to do this. It does not speak SMTP, nor is it likely to ever
    speak SMTP; it's hard enough maintaining a package to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header field (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header field.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, it's worth noting that with a shared Perl library,
    Perl on this platform doesn't apparently generate the right link magic
    to include the path to the dynamic Perl libraries. You need to either
    set LD_RUN_PATH before building or LD_LIBRARY_PATH before running any
    binaries so that they can find the Perl libraries. (The former is
    preferred, since then the path is encoded into the binaries and you
    don't have to remember to set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    Note that rejected articles are remembered during the number of days
    specified by the /remember/ line in expire.ctl; so, in case you forgot
    to change the above parameters, you'll have to wait that number of
    days before being able to inject them again. You can of course set
    /remember/ to 0, run the expire process (for instance via news.daily
    called with the same parameters as in crontab, plus "notdaily"),
    undo the change in expire.ctl and then start the feed again.

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles.

    In case you wish to only feed articles arrived on the old server
    between two dates, you can adapt the previous commands. For instance,
    the following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref header fields. Current versions of INN will always add an Xref header field, but very old versions (earlier 1.x versions) will only add an Xref header field to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles to add an Xref header field.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups header field
    body is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using
    the tradspool storage method. The newsgroup of an article is stored in
    the Newsgroups header field and in the Xref header field of the article
    as stored on disk (and possibly in Followup-To), as well as determining
    where the overview information is stored, and in the case of tradspool
    is also encoded in the article's storage token. To rename a newsgroup
    in tradspool, stop the server, move the directory containing all of
    the articles to its appropriate new location in the news spool, edit
    every article to change the old name to the new name in Newsgroups, Followup-To, and Xref, create the new newsgroup with ctlinnd newgroup,
    and then rebuild history and overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups header fields didn't break those links, you
    may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To header fields of the articles in that newsgroup and then rebuild history
    and overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To header
    fields to rename the newsgroup, and then create a file containing paths to
    all of the articles, one per line. You can then use that file as input to innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message IDs generated by INN will use the domain of the
    local system for the right-hand-side of the message ID. In some cases,
    this isn't desirable for various reasons (the server may have an internal
    name that doesn't make sense on Usenet at large, or one may not want to
    expose the name of the server).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will use
    that domain to generate message IDs. So if you need to change the domain
    used to generate message IDs for every local post from your server, just
    add virtualhost: and domain: keys to every access stanza in readers.conf.

    This is really overkill for this option, and eventually the domain:
    parameter in inn.conf will probably be changed to allow this to be
    modified for the whole server. (Right now, domain: in inn.conf means
    something completely different.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------

    Subject: 6.8. Generate MRTG graphs for INN

    INN's CNFS storage system has direct support for producing information
    suitable for MRTG graphs on the usage of the CNFS cycbuffs. Running
    cnfsstat -m <cycbuf> will generate output suitable for MRTG, and running cnfsstat -p will generate sample MRTG configuration fragments for each
    cycbuff.

    To generate MRTG graphs of the usage of the buffindexed overview system,
    try the following configuration fragment:

    Target[overview-BUFF]: `/usr/local/etc/mrtg/overview.sh`
    MaxBytes[overview-BUFF]: 100
    Title[overview-BUFF]: BUFF1 Usage
    Options[overview-BUFF]: growright gauge
    YLegend[overview-BUFF]: Overview Buffers
    ShortLegend[overview-BUFF]: %
    PageTop[overview-BUFF]: <H1>Usage of Overview Buffers</H1>
    <BR><TT>overview</TT>

    where the overview.sh script is:

    #!/bin/sh
    echo "100"
    <pathbin in inn.conf>/inndf -o | awk '{print $1}'
    echo "0"
    echo "overview"

    This sample configuration is from Basil Kruglov. Note that you can
    instead use -n (for total count of articles); in that case, you'll want to remove the MaxBytes setting above or change it to be some sensible limit
    on the total number of articles you receive. You'll also want to change a
    few of the other labels in the MRTG configuration.

    I'm not aware of any packaged solutions for generating MRTG data from
    other things, such as incoming or outcoming news flows. If anyone has any pointers, let me know.

    ------------------------------

    Subject: 6.9. Hide the junk and control groups from users

    The junk, control, and control.cancel groups must exist in the active file
    for the proper operation of INN, so you can't remove the groups entirely.
    You can, however, hide them completely from users.

    To do this, edit readers.conf, and for each user access group where you
    want to hide the junk and control groups, add "!junk,!control,!control.*"
    to the newsgroups pattern. In other words, if you have a line like:

    newsgroups: *


    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Mon Mar 21 07:01:04 2022
    [continued from previous message]

    and not spooled, you will need the assistance of the peer's admin to
    investigate further. INN does not generally log enough information
    about outgoing articles to be able to tell more from your server alone.

    It may be possible to get a slight bit of information from the remote
    server by connecting with telnet (usually to port 119) and issuing
    "IHAVE <message-id>". The peer may respond with something like "435
    Duplicate" which means that the problem is not likely to be with your
    server (it may be still a problem with the article itself). If the
    peer responds with something like "335", your server probably did not
    offer the article after all.

    If you really are at a dead end and need to get more information about
    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports and to mail messages to moderators, and it assumes that you
    have a program installed as /usr/sbin/sendmail or /usr/lib/sendmail that
    it can use to do this. It does not speak SMTP, nor is it likely to ever
    speak SMTP; it's hard enough maintaining a package to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header field (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header field.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, it's worth noting that with a shared Perl library,
    Perl on this platform doesn't apparently generate the right link magic
    to include the path to the dynamic Perl libraries. You need to either
    set LD_RUN_PATH before building or LD_LIBRARY_PATH before running any
    binaries so that they can find the Perl libraries. (The former is
    preferred, since then the path is encoded into the binaries and you
    don't have to remember to set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    Note that rejected articles are remembered during the number of days
    specified by the /remember/ line in expire.ctl; so, in case you forgot
    to change the above parameters, you'll have to wait that number of
    days before being able to inject them again. Another possibility is to
    set /remember/ to 0, run the expire process (for instance via news.daily
    called with the same parameters as in crontab, plus "notdaily"),
    undo the change in expire.ctl and then start the feed again.

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles.

    In case you wish to only feed articles arrived on the old server
    between two dates, you can adapt the previous commands. For instance,
    the following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref header fields. Current versions of INN will always add an Xref header field, but very old versions (earlier 1.x versions) will only add an Xref header field to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles to add an Xref header field.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups header field
    body is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using
    the tradspool storage method. The newsgroup of an article is stored in
    the Newsgroups header field and in the Xref header field of the article
    as stored on disk (and possibly in Followup-To), as well as determining
    where the overview information is stored, and in the case of tradspool
    is also encoded in the article's storage token. To rename a newsgroup
    in tradspool, stop the server, move the directory containing all of
    the articles to its appropriate new location in the news spool, edit
    every article to change the old name to the new name in Newsgroups, Followup-To, and Xref, create the new newsgroup with ctlinnd newgroup,
    and then rebuild history and overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups header fields didn't break those links, you
    may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To header fields of the articles in that newsgroup and then rebuild history
    and overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To header
    fields to rename the newsgroup, and then create a file containing paths to
    all of the articles, one per line. You can then use that file as input to innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message IDs generated by INN will use the domain of the
    local system for the right-hand-side of the message ID. In some cases,
    this isn't desirable for various reasons (the server may have an internal
    name that doesn't make sense on Usenet at large, or one may not want to
    expose the name of the server).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will use
    that domain to generate message IDs. So if you need to change the domain
    used to generate message IDs for every local post from your server, just
    add virtualhost: and domain: keys to every access stanza in readers.conf.

    This is really overkill for this option, and eventually the domain:
    parameter in inn.conf will probably be changed to allow this to be
    modified for the whole server. (Right now, domain: in inn.conf means
    something completely different.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------

    Subject: 6.8. Generate MRTG graphs for INN

    INN's CNFS storage system has direct support for producing information
    suitable for MRTG graphs on the usage of the CNFS cycbuffs. Running
    cnfsstat -m <cycbuf> will generate output suitable for MRTG, and running cnfsstat -p will generate sample MRTG configuration fragments for each
    cycbuff.

    To generate MRTG graphs of the usage of the buffindexed overview system,
    try the following configuration fragment:

    Target[overview-BUFF]: `/usr/local/etc/mrtg/overview.sh`
    MaxBytes[overview-BUFF]: 100
    Title[overview-BUFF]: BUFF1 Usage
    Options[overview-BUFF]: growright gauge
    YLegend[overview-BUFF]: Overview Buffers
    ShortLegend[overview-BUFF]: %
    PageTop[overview-BUFF]: <H1>Usage of Overview Buffers</H1>
    <BR><TT>overview</TT>

    where the overview.sh script is:

    #!/bin/sh
    echo "100"
    <pathbin in inn.conf>/inndf -o | awk '{print $1}'
    echo "0"
    echo "overview"

    This sample configuration is from Basil Kruglov. Note that you can
    instead use -n (for total count of articles); in that case, you'll want to remove the MaxBytes setting above or change it to be some sensible limit
    on the total number of articles you receive. You'll also want to change a
    few of the other labels in the MRTG configuration.

    I'm not aware of any packaged solutions for generating MRTG data from
    other things, such as incoming or outcoming news flows. If anyone has any pointers, let me know.

    ------------------------------

    Subject: 6.9. Hide the junk and control groups from users

    The junk, control, and control.cancel groups must exist in the active file
    for the proper operation of INN, so you can't remove the groups entirely.
    You can, however, hide them completely from users.

    To do this, edit readers.conf, and for each user access group where you
    want to hide the junk and control groups, add "!junk,!control,!control.*"
    to the newsgroups pattern. In other words, if you have a line like:


    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Thu Apr 21 07:01:03 2022
    [continued from previous message]

    investigate further. INN does not generally log enough information
    about outgoing articles to be able to tell more from your server alone.

    It may be possible to get a slight bit of information from the remote
    server by connecting with telnet (usually to port 119) and issuing
    "IHAVE <message-id>". The peer may respond with something like "435
    Duplicate" which means that the problem is not likely to be with your
    server (it may be still a problem with the article itself). If the
    peer responds with something like "335", your server probably did not
    offer the article after all.

    If you really are at a dead end and need to get more information about
    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports and to mail messages to moderators, and it assumes that you
    have a program installed as /usr/sbin/sendmail or /usr/lib/sendmail that
    it can use to do this. It does not speak SMTP, nor is it likely to ever
    speak SMTP; it's hard enough maintaining a package to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header field (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header field.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, it's worth noting that with a shared Perl library,
    Perl on this platform doesn't apparently generate the right link magic
    to include the path to the dynamic Perl libraries. You need to either
    set LD_RUN_PATH before building or LD_LIBRARY_PATH before running any
    binaries so that they can find the Perl libraries. (The former is
    preferred, since then the path is encoded into the binaries and you
    don't have to remember to set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    Note that rejected articles are remembered during the number of days
    specified by the /remember/ line in expire.ctl; so, in case you forgot
    to change the above parameters, you'll have to wait that number of
    days before being able to inject them again. Another possibility is to
    set /remember/ to 0, run the expire process (for instance via news.daily
    called with the same parameters as in crontab, plus "notdaily"),
    undo the change in expire.ctl and then start the feed again.

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles.

    In case you wish to only feed articles arrived on the old server
    between two dates, you can adapt the previous commands. For instance,
    the following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref header fields. Current versions of INN will always add an Xref header field, but very old versions (earlier 1.x versions) will only add an Xref header field to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles to add an Xref header field.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups header field
    body is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using
    the tradspool storage method. The newsgroup of an article is stored in
    the Newsgroups header field and in the Xref header field of the article
    as stored on disk (and possibly in Followup-To), as well as determining
    where the overview information is stored, and in the case of tradspool
    is also encoded in the article's storage token. To rename a newsgroup
    in tradspool, stop the server, move the directory containing all of
    the articles to its appropriate new location in the news spool, edit
    every article to change the old name to the new name in Newsgroups, Followup-To, and Xref, create the new newsgroup with ctlinnd newgroup,
    and then rebuild history and overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups header fields didn't break those links, you
    may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To header fields of the articles in that newsgroup and then rebuild history
    and overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To header
    fields to rename the newsgroup, and then create a file containing paths to
    all of the articles, one per line. You can then use that file as input to innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message IDs generated by INN will use the domain of the
    local system for the right-hand-side of the message ID. In some cases,
    this isn't desirable for various reasons (the server may have an internal
    name that doesn't make sense on Usenet at large, or one may not want to
    expose the name of the server).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will use
    that domain to generate message IDs. So if you need to change the domain
    used to generate message IDs for every local post from your server, just
    add virtualhost: and domain: keys to every access stanza in readers.conf.

    This is really overkill for this option, and eventually the domain:
    parameter in inn.conf will probably be changed to allow this to be
    modified for the whole server. (Right now, domain: in inn.conf means
    something completely different.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------

    Subject: 6.8. Generate MRTG graphs for INN

    INN's CNFS storage system has direct support for producing information
    suitable for MRTG graphs on the usage of the CNFS cycbuffs. Running
    cnfsstat -m <cycbuf> will generate output suitable for MRTG, and running cnfsstat -p will generate sample MRTG configuration fragments for each
    cycbuff.

    To generate MRTG graphs of the usage of the buffindexed overview system,
    try the following configuration fragment:

    Target[overview-BUFF]: `/usr/local/etc/mrtg/overview.sh`
    MaxBytes[overview-BUFF]: 100
    Title[overview-BUFF]: BUFF1 Usage
    Options[overview-BUFF]: growright gauge
    YLegend[overview-BUFF]: Overview Buffers
    ShortLegend[overview-BUFF]: %
    PageTop[overview-BUFF]: <H1>Usage of Overview Buffers</H1>
    <BR><TT>overview</TT>

    where the overview.sh script is:

    #!/bin/sh
    echo "100"
    <pathbin in inn.conf>/inndf -o | awk '{print $1}'
    echo "0"
    echo "overview"

    This sample configuration is from Basil Kruglov. Note that you can
    instead use -n (for total count of articles); in that case, you'll want to remove the MaxBytes setting above or change it to be some sensible limit
    on the total number of articles you receive. You'll also want to change a
    few of the other labels in the MRTG configuration.

    I'm not aware of any packaged solutions for generating MRTG data from
    other things, such as incoming or outcoming news flows. If anyone has any pointers, let me know.

    ------------------------------

    Subject: 6.9. Hide the junk and control groups from users

    The junk, control, and control.cancel groups must exist in the active file
    for the proper operation of INN, so you can't remove the groups entirely.
    You can, however, hide them completely from users.

    To do this, edit readers.conf, and for each user access group where you
    want to hide the junk and control groups, add "!junk,!control,!control.*"
    to the newsgroups pattern. In other words, if you have a line like:

    newsgroups: *

    just change that to:

    newsgroups: *,!junk,!control,!control.*


    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Sat May 21 07:01:04 2022
    [continued from previous message]

    investigate further. INN does not generally log enough information
    about outgoing articles to be able to tell more from your server alone.

    It may be possible to get a slight bit of information from the remote
    server by connecting with telnet (usually to port 119) and issuing
    "IHAVE <message-id>". The peer may respond with something like "435
    Duplicate" which means that the problem is not likely to be with your
    server (it may be still a problem with the article itself). If the
    peer responds with something like "335", your server probably did not
    offer the article after all.

    If you really are at a dead end and need to get more information about
    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports and to mail messages to moderators, and it assumes that you
    have a program installed as /usr/sbin/sendmail or /usr/lib/sendmail that
    it can use to do this. It does not speak SMTP, nor is it likely to ever
    speak SMTP; it's hard enough maintaining a package to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header field (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header field.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, it's worth noting that with a shared Perl library,
    Perl on this platform doesn't apparently generate the right link magic
    to include the path to the dynamic Perl libraries. You need to either
    set LD_RUN_PATH before building or LD_LIBRARY_PATH before running any
    binaries so that they can find the Perl libraries. (The former is
    preferred, since then the path is encoded into the binaries and you
    don't have to remember to set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    Note that rejected articles are remembered during the number of days
    specified by the /remember/ line in expire.ctl; so, in case you forgot
    to change the above parameters, you'll have to wait that number of
    days before being able to inject them again. Another possibility is to
    set /remember/ to 0, run the expire process (for instance via news.daily
    called with the same parameters as in crontab, plus "notdaily"),
    undo the change in expire.ctl and then start the feed again.

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles.

    In case you wish to only feed articles arrived on the old server
    between two dates, you can adapt the previous commands. For instance,
    the following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref header fields. Current versions of INN will always add an Xref header field, but very old versions (earlier 1.x versions) will only add an Xref header field to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles to add an Xref header field.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups header field
    body is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using
    the tradspool storage method. The newsgroup of an article is stored in
    the Newsgroups header field and in the Xref header field of the article
    as stored on disk (and possibly in Followup-To), as well as determining
    where the overview information is stored, and in the case of tradspool
    is also encoded in the article's storage token. To rename a newsgroup
    in tradspool, stop the server, move the directory containing all of
    the articles to its appropriate new location in the news spool, edit
    every article to change the old name to the new name in Newsgroups, Followup-To, and Xref, create the new newsgroup with ctlinnd newgroup,
    and then rebuild history and overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups header fields didn't break those links, you
    may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To header fields of the articles in that newsgroup and then rebuild history
    and overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To header
    fields to rename the newsgroup, and then create a file containing paths to
    all of the articles, one per line. You can then use that file as input to innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message IDs generated by INN will use the domain of the
    local system for the right-hand-side of the message ID. In some cases,
    this isn't desirable for various reasons (the server may have an internal
    name that doesn't make sense on Usenet at large, or one may not want to
    expose the name of the server).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will use
    that domain to generate message IDs. So if you need to change the domain
    used to generate message IDs for every local post from your server, just
    add virtualhost: and domain: keys to every access stanza in readers.conf.

    This is really overkill for this option, and eventually the domain:
    parameter in inn.conf will probably be changed to allow this to be
    modified for the whole server. (Right now, domain: in inn.conf means
    something completely different.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------

    Subject: 6.8. Generate MRTG graphs for INN

    INN's CNFS storage system has direct support for producing information
    suitable for MRTG graphs on the usage of the CNFS cycbuffs. Running
    cnfsstat -m <cycbuf> will generate output suitable for MRTG, and running cnfsstat -p will generate sample MRTG configuration fragments for each
    cycbuff.

    To generate MRTG graphs of the usage of the buffindexed overview system,
    try the following configuration fragment:

    Target[overview-BUFF]: `/usr/local/etc/mrtg/overview.sh`
    MaxBytes[overview-BUFF]: 100
    Title[overview-BUFF]: BUFF1 Usage
    Options[overview-BUFF]: growright gauge
    YLegend[overview-BUFF]: Overview Buffers
    ShortLegend[overview-BUFF]: %
    PageTop[overview-BUFF]: <H1>Usage of Overview Buffers</H1>
    <BR><TT>overview</TT>

    where the overview.sh script is:

    #!/bin/sh
    echo "100"
    <pathbin in inn.conf>/inndf -o | awk '{print $1}'
    echo "0"
    echo "overview"

    This sample configuration is from Basil Kruglov. Note that you can
    instead use -n (for total count of articles); in that case, you'll want to remove the MaxBytes setting above or change it to be some sensible limit
    on the total number of articles you receive. You'll also want to change a
    few of the other labels in the MRTG configuration.

    I'm not aware of any packaged solutions for generating MRTG data from
    other things, such as incoming or outcoming news flows. If anyone has any pointers, let me know.

    ------------------------------

    Subject: 6.9. Hide the junk and control groups from users

    The junk, control, and control.cancel groups must exist in the active file
    for the proper operation of INN, so you can't remove the groups entirely.
    You can, however, hide them completely from users.

    To do this, edit readers.conf, and for each user access group where you
    want to hide the junk and control groups, add "!junk,!control,!control.*"
    to the newsgroups pattern. In other words, if you have a line like:

    newsgroups: *

    just change that to:

    newsgroups: *,!junk,!control,!control.*


    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Tue Jun 21 07:01:02 2022
    [continued from previous message]

    investigate further. INN does not generally log enough information
    about outgoing articles to be able to tell more from your server alone.

    It may be possible to get a slight bit of information from the remote
    server by connecting with telnet (usually to port 119) and issuing
    "IHAVE <message-id>". The peer may respond with something like "435
    Duplicate" which means that the problem is not likely to be with your
    server (it may be still a problem with the article itself). If the
    peer responds with something like "335", your server probably did not
    offer the article after all.

    If you really are at a dead end and need to get more information about
    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports and to mail messages to moderators, and it assumes that you
    have a program installed as /usr/sbin/sendmail or /usr/lib/sendmail that
    it can use to do this. It does not speak SMTP, nor is it likely to ever
    speak SMTP; it's hard enough maintaining a package to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header field (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header field.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, it's worth noting that with a shared Perl library,
    Perl on this platform doesn't apparently generate the right link magic
    to include the path to the dynamic Perl libraries. You need to either
    set LD_RUN_PATH before building or LD_LIBRARY_PATH before running any
    binaries so that they can find the Perl libraries. (The former is
    preferred, since then the path is encoded into the binaries and you
    don't have to remember to set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    Note that rejected articles are remembered during the number of days
    specified by the /remember/ line in expire.ctl; so, in case you forgot
    to change the above parameters, you'll have to wait that number of
    days before being able to inject them again. Another possibility is to
    set /remember/ to 0, run the expire process (for instance via news.daily
    called with the same parameters as in crontab, plus "notdaily"),
    undo the change in expire.ctl and then start the feed again.

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles.

    In case you wish to only feed articles arrived on the old server
    between two dates, you can adapt the previous commands. For instance,
    the following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref header fields. Current versions of INN will always add an Xref header field, but very old versions (earlier 1.x versions) will only add an Xref header field to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles to add an Xref header field.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups header field
    body is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using
    the tradspool storage method. The newsgroup of an article is stored in
    the Newsgroups header field and in the Xref header field of the article
    as stored on disk (and possibly in Followup-To), as well as determining
    where the overview information is stored, and in the case of tradspool
    is also encoded in the article's storage token. To rename a newsgroup
    in tradspool, stop the server, move the directory containing all of
    the articles to its appropriate new location in the news spool, edit
    every article to change the old name to the new name in Newsgroups, Followup-To, and Xref, create the new newsgroup with ctlinnd newgroup,
    and then rebuild history and overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups header fields didn't break those links, you
    may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To header fields of the articles in that newsgroup and then rebuild history
    and overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To header
    fields to rename the newsgroup, and then create a file containing paths to
    all of the articles, one per line. You can then use that file as input to innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message IDs generated by INN will use the domain of the
    local system for the right-hand-side of the message ID. In some cases,
    this isn't desirable for various reasons (the server may have an internal
    name that doesn't make sense on Usenet at large, or one may not want to
    expose the name of the server).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will use
    that domain to generate message IDs. So if you need to change the domain
    used to generate message IDs for every local post from your server, just
    add virtualhost: and domain: keys to every access stanza in readers.conf.

    This is really overkill for this option, and eventually the domain:
    parameter in inn.conf will probably be changed to allow this to be
    modified for the whole server. (Right now, domain: in inn.conf means
    something completely different.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------

    Subject: 6.8. Generate MRTG graphs for INN

    INN's CNFS storage system has direct support for producing information
    suitable for MRTG graphs on the usage of the CNFS cycbuffs. Running
    cnfsstat -m <cycbuf> will generate output suitable for MRTG, and running cnfsstat -p will generate sample MRTG configuration fragments for each
    cycbuff.

    To generate MRTG graphs of the usage of the buffindexed overview system,
    try the following configuration fragment:

    Target[overview-BUFF]: `/usr/local/etc/mrtg/overview.sh`
    MaxBytes[overview-BUFF]: 100
    Title[overview-BUFF]: BUFF1 Usage
    Options[overview-BUFF]: growright gauge
    YLegend[overview-BUFF]: Overview Buffers
    ShortLegend[overview-BUFF]: %
    PageTop[overview-BUFF]: <H1>Usage of Overview Buffers</H1>
    <BR><TT>overview</TT>

    where the overview.sh script is:

    #!/bin/sh
    echo "100"
    <pathbin in inn.conf>/inndf -o | awk '{print $1}'
    echo "0"
    echo "overview"

    This sample configuration is from Basil Kruglov. Note that you can
    instead use -n (for total count of articles); in that case, you'll want to remove the MaxBytes setting above or change it to be some sensible limit
    on the total number of articles you receive. You'll also want to change a
    few of the other labels in the MRTG configuration.

    I'm not aware of any packaged solutions for generating MRTG data from
    other things, such as incoming or outcoming news flows. If anyone has any pointers, let me know.

    ------------------------------

    Subject: 6.9. Hide the junk and control groups from users

    The junk, control, and control.cancel groups must exist in the active file
    for the proper operation of INN, so you can't remove the groups entirely.
    You can, however, hide them completely from users.

    To do this, edit readers.conf, and for each user access group where you
    want to hide the junk and control groups, add "!junk,!control,!control.*"
    to the newsgroups pattern. In other words, if you have a line like:

    newsgroups: *

    just change that to:

    newsgroups: *,!junk,!control,!control.*


    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Thu Jul 21 07:01:02 2022
    [continued from previous message]

    investigate further. INN does not generally log enough information
    about outgoing articles to be able to tell more from your server alone.

    It may be possible to get a slight bit of information from the remote
    server by connecting with telnet (usually to port 119) and issuing
    "IHAVE <message-id>". The peer may respond with something like "435
    Duplicate" which means that the problem is not likely to be with your
    server (it may be still a problem with the article itself). If the
    peer responds with something like "335", your server probably did not
    offer the article after all.

    If you really are at a dead end and need to get more information about
    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports and to mail messages to moderators, and it assumes that you
    have a program installed as /usr/sbin/sendmail or /usr/lib/sendmail that
    it can use to do this. It does not speak SMTP, nor is it likely to ever
    speak SMTP; it's hard enough maintaining a package to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header field (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header field.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, it's worth noting that with a shared Perl library,
    Perl on this platform doesn't apparently generate the right link magic
    to include the path to the dynamic Perl libraries. You need to either
    set LD_RUN_PATH before building or LD_LIBRARY_PATH before running any
    binaries so that they can find the Perl libraries. (The former is
    preferred, since then the path is encoded into the binaries and you
    don't have to remember to set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    Note that rejected articles are remembered during the number of days
    specified by the /remember/ line in expire.ctl; so, in case you forgot
    to change the above parameters, you'll have to wait that number of
    days before being able to inject them again. Another possibility is to
    set /remember/ to 0, run the expire process (for instance via news.daily
    called with the same parameters as in crontab, plus "notdaily"),
    undo the change in expire.ctl and then start the feed again.

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles.

    In case you wish to only feed articles arrived on the old server
    between two dates, you can adapt the previous commands. For instance,
    the following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref header fields. Current versions of INN will always add an Xref header field, but very old versions (earlier 1.x versions) will only add an Xref header field to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles to add an Xref header field.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups header field
    body is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using
    the tradspool storage method. The newsgroup of an article is stored in
    the Newsgroups header field and in the Xref header field of the article
    as stored on disk (and possibly in Followup-To), as well as determining
    where the overview information is stored, and in the case of tradspool
    is also encoded in the article's storage token. To rename a newsgroup
    in tradspool, stop the server, move the directory containing all of
    the articles to its appropriate new location in the news spool, edit
    every article to change the old name to the new name in Newsgroups, Followup-To, and Xref, create the new newsgroup with ctlinnd newgroup,
    and then rebuild history and overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups header fields didn't break those links, you
    may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To header fields of the articles in that newsgroup and then rebuild history
    and overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To header
    fields to rename the newsgroup, and then create a file containing paths to
    all of the articles, one per line. You can then use that file as input to innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message IDs generated by INN will use the domain of the
    local system for the right-hand-side of the message ID. In some cases,
    this isn't desirable for various reasons (the server may have an internal
    name that doesn't make sense on Usenet at large, or one may not want to
    expose the name of the server).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will use
    that domain to generate message IDs. So if you need to change the domain
    used to generate message IDs for every local post from your server, just
    add virtualhost: and domain: keys to every access stanza in readers.conf.

    This is really overkill for this option, and eventually the domain:
    parameter in inn.conf will probably be changed to allow this to be
    modified for the whole server. (Right now, domain: in inn.conf means
    something completely different.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------

    Subject: 6.8. Generate MRTG graphs for INN

    INN's CNFS storage system has direct support for producing information
    suitable for MRTG graphs on the usage of the CNFS cycbuffs. Running
    cnfsstat -m <cycbuf> will generate output suitable for MRTG, and running cnfsstat -p will generate sample MRTG configuration fragments for each
    cycbuff.

    To generate MRTG graphs of the usage of the buffindexed overview system,
    try the following configuration fragment:

    Target[overview-BUFF]: `/usr/local/etc/mrtg/overview.sh`
    MaxBytes[overview-BUFF]: 100
    Title[overview-BUFF]: BUFF1 Usage
    Options[overview-BUFF]: growright gauge
    YLegend[overview-BUFF]: Overview Buffers
    ShortLegend[overview-BUFF]: %
    PageTop[overview-BUFF]: <H1>Usage of Overview Buffers</H1>
    <BR><TT>overview</TT>

    where the overview.sh script is:

    #!/bin/sh
    echo "100"
    <pathbin in inn.conf>/inndf -o | awk '{print $1}'
    echo "0"
    echo "overview"

    This sample configuration is from Basil Kruglov. Note that you can
    instead use -n (for total count of articles); in that case, you'll want to remove the MaxBytes setting above or change it to be some sensible limit
    on the total number of articles you receive. You'll also want to change a
    few of the other labels in the MRTG configuration.

    I'm not aware of any packaged solutions for generating MRTG data from
    other things, such as incoming or outcoming news flows. If anyone has any pointers, let me know.

    ------------------------------

    Subject: 6.9. Hide the junk and control groups from users

    The junk, control, and control.cancel groups must exist in the active file
    for the proper operation of INN, so you can't remove the groups entirely.
    You can, however, hide them completely from users.

    To do this, edit readers.conf, and for each user access group where you
    want to hide the junk and control groups, add "!junk,!control,!control.*"
    to the newsgroups pattern. In other words, if you have a line like:

    newsgroups: *

    just change that to:

    newsgroups: *,!junk,!control,!control.*


    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Sun Aug 21 07:01:04 2022
    [continued from previous message]

    investigate further. INN does not generally log enough information
    about outgoing articles to be able to tell more from your server alone.

    It may be possible to get a slight bit of information from the remote
    server by connecting with telnet (usually to port 119) and issuing
    "IHAVE <message-id>". The peer may respond with something like "435
    Duplicate" which means that the problem is not likely to be with your
    server (it may be still a problem with the article itself). If the
    peer responds with something like "335", your server probably did not
    offer the article after all.

    If you really are at a dead end and need to get more information about
    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports and to mail messages to moderators, and it assumes that you
    have a program installed as /usr/sbin/sendmail or /usr/lib/sendmail that
    it can use to do this. It does not speak SMTP, nor is it likely to ever
    speak SMTP; it's hard enough maintaining a package to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header field (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header field.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, it's worth noting that with a shared Perl library,
    Perl on this platform doesn't apparently generate the right link magic
    to include the path to the dynamic Perl libraries. You need to either
    set LD_RUN_PATH before building or LD_LIBRARY_PATH before running any
    binaries so that they can find the Perl libraries. (The former is
    preferred, since then the path is encoded into the binaries and you
    don't have to remember to set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    Note that rejected articles are remembered during the number of days
    specified by the /remember/ line in expire.ctl; so, in case you forgot
    to change the above parameters, you'll have to wait that number of
    days before being able to inject them again. Another possibility is to
    set /remember/ to 0, run the expire process (for instance via news.daily
    called with the same parameters as in crontab, plus "notdaily"),
    undo the change in expire.ctl and then start the feed again.

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles.

    In case you wish to only feed articles arrived on the old server
    between two dates, you can adapt the previous commands. For instance,
    the following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref header fields. Current versions of INN will always add an Xref header field, but very old versions (earlier 1.x versions) will only add an Xref header field to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles to add an Xref header field.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups header field
    body is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using
    the tradspool storage method. The newsgroup of an article is stored in
    the Newsgroups header field and in the Xref header field of the article
    as stored on disk (and possibly in Followup-To), as well as determining
    where the overview information is stored, and in the case of tradspool
    is also encoded in the article's storage token. To rename a newsgroup
    in tradspool, stop the server, move the directory containing all of
    the articles to its appropriate new location in the news spool, edit
    every article to change the old name to the new name in Newsgroups, Followup-To, and Xref, create the new newsgroup with ctlinnd newgroup,
    and then rebuild history and overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups header fields didn't break those links, you
    may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To header fields of the articles in that newsgroup and then rebuild history
    and overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To header
    fields to rename the newsgroup, and then create a file containing paths to
    all of the articles, one per line. You can then use that file as input to innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message IDs generated by INN will use the domain of the
    local system for the right-hand-side of the message ID. In some cases,
    this isn't desirable for various reasons (the server may have an internal
    name that doesn't make sense on Usenet at large, or one may not want to
    expose the name of the server).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will use
    that domain to generate message IDs. So if you need to change the domain
    used to generate message IDs for every local post from your server, just
    add virtualhost: and domain: keys to every access stanza in readers.conf.

    This is really overkill for this option, and eventually the domain:
    parameter in inn.conf will probably be changed to allow this to be
    modified for the whole server. (Right now, domain: in inn.conf means
    something completely different.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------

    Subject: 6.8. Generate MRTG graphs for INN

    INN's CNFS storage system has direct support for producing information
    suitable for MRTG graphs on the usage of the CNFS cycbuffs. Running
    cnfsstat -m <cycbuf> will generate output suitable for MRTG, and running cnfsstat -p will generate sample MRTG configuration fragments for each
    cycbuff.

    To generate MRTG graphs of the usage of the buffindexed overview system,
    try the following configuration fragment:

    Target[overview-BUFF]: `/usr/local/etc/mrtg/overview.sh`
    MaxBytes[overview-BUFF]: 100
    Title[overview-BUFF]: BUFF1 Usage
    Options[overview-BUFF]: growright gauge
    YLegend[overview-BUFF]: Overview Buffers
    ShortLegend[overview-BUFF]: %
    PageTop[overview-BUFF]: <H1>Usage of Overview Buffers</H1>
    <BR><TT>overview</TT>

    where the overview.sh script is:

    #!/bin/sh
    echo "100"
    <pathbin in inn.conf>/inndf -o | awk '{print $1}'
    echo "0"
    echo "overview"

    This sample configuration is from Basil Kruglov. Note that you can
    instead use -n (for total count of articles); in that case, you'll want to remove the MaxBytes setting above or change it to be some sensible limit
    on the total number of articles you receive. You'll also want to change a
    few of the other labels in the MRTG configuration.

    I'm not aware of any packaged solutions for generating MRTG data from
    other things, such as incoming or outcoming news flows. If anyone has any pointers, let me know.

    ------------------------------

    Subject: 6.9. Hide the junk and control groups from users

    The junk, control, and control.cancel groups must exist in the active file
    for the proper operation of INN, so you can't remove the groups entirely.
    You can, however, hide them completely from users.

    To do this, edit readers.conf, and for each user access group where you
    want to hide the junk and control groups, add "!junk,!control,!control.*"
    to the newsgroups pattern. In other words, if you have a line like:

    newsgroups: *

    just change that to:

    newsgroups: *,!junk,!control,!control.*


    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Wed Sep 21 07:01:02 2022
    [continued from previous message]

    investigate further. INN does not generally log enough information
    about outgoing articles to be able to tell more from your server alone.

    It may be possible to get a slight bit of information from the remote
    server by connecting with telnet (usually to port 119) and issuing
    "IHAVE <message-id>". The peer may respond with something like "435
    Duplicate" which means that the problem is not likely to be with your
    server (it may be still a problem with the article itself). If the
    peer responds with something like "335", your server probably did not
    offer the article after all.

    If you really are at a dead end and need to get more information about
    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports and to mail messages to moderators, and it assumes that you
    have a program installed as /usr/sbin/sendmail or /usr/lib/sendmail that
    it can use to do this. It does not speak SMTP, nor is it likely to ever
    speak SMTP; it's hard enough maintaining a package to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header field (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header field.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, it's worth noting that with a shared Perl library,
    Perl on this platform doesn't apparently generate the right link magic
    to include the path to the dynamic Perl libraries. You need to either
    set LD_RUN_PATH before building or LD_LIBRARY_PATH before running any
    binaries so that they can find the Perl libraries. (The former is
    preferred, since then the path is encoded into the binaries and you
    don't have to remember to set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    Note that rejected articles are remembered during the number of days
    specified by the /remember/ line in expire.ctl; so, in case you forgot
    to change the above parameters, you'll have to wait that number of
    days before being able to inject them again. Another possibility is to
    set /remember/ to 0, run the expire process (for instance via news.daily
    called with the same parameters as in crontab, plus "notdaily"),
    undo the change in expire.ctl and then start the feed again.

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles.

    In case you wish to only feed articles arrived on the old server
    between two dates, you can adapt the previous commands. For instance,
    the following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref header fields. Current versions of INN will always add an Xref header field, but very old versions (earlier 1.x versions) will only add an Xref header field to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles to add an Xref header field.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups header field
    body is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using
    the tradspool storage method. The newsgroup of an article is stored in
    the Newsgroups header field and in the Xref header field of the article
    as stored on disk (and possibly in Followup-To), as well as determining
    where the overview information is stored, and in the case of tradspool
    is also encoded in the article's storage token. To rename a newsgroup
    in tradspool, stop the server, move the directory containing all of
    the articles to its appropriate new location in the news spool, edit
    every article to change the old name to the new name in Newsgroups, Followup-To, and Xref, create the new newsgroup with ctlinnd newgroup,
    and then rebuild history and overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups header fields didn't break those links, you
    may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To header fields of the articles in that newsgroup and then rebuild history
    and overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To header
    fields to rename the newsgroup, and then create a file containing paths to
    all of the articles, one per line. You can then use that file as input to innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message IDs generated by INN will use the domain of the
    local system for the right-hand-side of the message ID. In some cases,
    this isn't desirable for various reasons (the server may have an internal
    name that doesn't make sense on Usenet at large, or one may not want to
    expose the name of the server).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will use
    that domain to generate message IDs. So if you need to change the domain
    used to generate message IDs for every local post from your server, just
    add virtualhost: and domain: keys to every access stanza in readers.conf.

    This is really overkill for this option, and eventually the domain:
    parameter in inn.conf will probably be changed to allow this to be
    modified for the whole server. (Right now, domain: in inn.conf means
    something completely different.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------

    Subject: 6.8. Generate MRTG graphs for INN

    INN's CNFS storage system has direct support for producing information
    suitable for MRTG graphs on the usage of the CNFS cycbuffs. Running
    cnfsstat -m <cycbuf> will generate output suitable for MRTG, and running cnfsstat -p will generate sample MRTG configuration fragments for each
    cycbuff.

    To generate MRTG graphs of the usage of the buffindexed overview system,
    try the following configuration fragment:

    Target[overview-BUFF]: `/usr/local/etc/mrtg/overview.sh`
    MaxBytes[overview-BUFF]: 100
    Title[overview-BUFF]: BUFF1 Usage
    Options[overview-BUFF]: growright gauge
    YLegend[overview-BUFF]: Overview Buffers
    ShortLegend[overview-BUFF]: %
    PageTop[overview-BUFF]: <H1>Usage of Overview Buffers</H1>
    <BR><TT>overview</TT>

    where the overview.sh script is:

    #!/bin/sh
    echo "100"
    <pathbin in inn.conf>/inndf -o | awk '{print $1}'
    echo "0"
    echo "overview"

    This sample configuration is from Basil Kruglov. Note that you can
    instead use -n (for total count of articles); in that case, you'll want to remove the MaxBytes setting above or change it to be some sensible limit
    on the total number of articles you receive. You'll also want to change a
    few of the other labels in the MRTG configuration.

    I'm not aware of any packaged solutions for generating MRTG data from
    other things, such as incoming or outcoming news flows. If anyone has any pointers, let me know.

    ------------------------------

    Subject: 6.9. Hide the junk and control groups from users

    The junk, control, and control.cancel groups must exist in the active file
    for the proper operation of INN, so you can't remove the groups entirely.
    You can, however, hide them completely from users.

    To do this, edit readers.conf, and for each user access group where you
    want to hide the junk and control groups, add "!junk,!control,!control.*"
    to the newsgroups pattern. In other words, if you have a line like:

    newsgroups: *

    just change that to:

    newsgroups: *,!junk,!control,!control.*


    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Fri Oct 21 07:01:03 2022
    [continued from previous message]

    investigate further. INN does not generally log enough information
    about outgoing articles to be able to tell more from your server alone.

    It may be possible to get a slight bit of information from the remote
    server by connecting with telnet (usually to port 119) and issuing
    "IHAVE <message-id>". The peer may respond with something like "435
    Duplicate" which means that the problem is not likely to be with your
    server (it may be still a problem with the article itself). If the
    peer responds with something like "335", your server probably did not
    offer the article after all.

    If you really are at a dead end and need to get more information about
    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports and to mail messages to moderators, and it assumes that you
    have a program installed as /usr/sbin/sendmail or /usr/lib/sendmail that
    it can use to do this. It does not speak SMTP, nor is it likely to ever
    speak SMTP; it's hard enough maintaining a package to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header field (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header field.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, it's worth noting that with a shared Perl library,
    Perl on this platform doesn't apparently generate the right link magic
    to include the path to the dynamic Perl libraries. You need to either
    set LD_RUN_PATH before building or LD_LIBRARY_PATH before running any
    binaries so that they can find the Perl libraries. (The former is
    preferred, since then the path is encoded into the binaries and you
    don't have to remember to set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    Note that rejected articles are remembered during the number of days
    specified by the /remember/ line in expire.ctl; so, in case you forgot
    to change the above parameters, you'll have to wait that number of
    days before being able to inject them again. Another possibility is to
    set /remember/ to 0, run the expire process (for instance via news.daily
    called with the same parameters as in crontab, plus "notdaily"),
    undo the change in expire.ctl and then start the feed again.

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles. The result file
    contains tokens ordered by arrival time on the old server (which is
    usually roughly the same as the posting time). In case the history
    file was not populated chronologically, it is better to sort it by
    posting time so that articles are fed in the right order. This can be
    achieved with the following command:

    sort -t '~' -k3n < history > history.sorted

    And then, consider history.sorted instead of history for the next steps.

    If the new server has just been installed or is known not to already have
    the articles you will feed it, you may want to add the "-c" flag to
    innxmit so as to skip the check for the presence of every article before transferring them.

    In case you wish to only feed articles arrived on the old server between
    two dates, you can adapt the previous commands with a condition on the
    $arrived variable (or the $posted variable if you prefer to use the Date
    header field instead of the actual arrival time). For instance, the
    following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    Other conditions may be added in the print line to select a subset of
    articles to feed. For instance if you want only articles for the fr.* hierarchy, you may add the following condition which will retrieve and
    parse the contents of the Xref field of every article to find the
    information (note that it will take some time to run, depending on the
    number of articles to parse):

    and qx/sm -q -H "$_" | grep Xref/ =~ / fr\.\S+:\d+/

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it. The "-d" flag
    is useful to add if you want to see the feeding progress.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref header fields. Current versions of INN will always add an Xref header field, but very old versions (earlier 1.x versions) will only add an Xref header field to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles to add an Xref header field.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups header field
    body is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using
    the tradspool storage method. The newsgroup of an article is stored in
    the Newsgroups header field and in the Xref header field of the article
    as stored on disk (and possibly in Followup-To), as well as determining
    where the overview information is stored, and in the case of tradspool
    is also encoded in the article's storage token. To rename a newsgroup
    in tradspool, stop the server, move the directory containing all of
    the articles to its appropriate new location in the news spool, edit
    every article to change the old name to the new name in Newsgroups, Followup-To, and Xref, create the new newsgroup with ctlinnd newgroup,
    and then rebuild history and overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups header fields didn't break those links, you
    may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To header fields of the articles in that newsgroup and then rebuild history
    and overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To header
    fields to rename the newsgroup, and then create a file containing paths to
    all of the articles, one per line. You can then use that file as input to innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message IDs generated by INN will use the domain of the
    local system for the right-hand-side of the message ID. In some cases,
    this isn't desirable for various reasons (the server may have an internal
    name that doesn't make sense on Usenet at large, or one may not want to
    expose the name of the server).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will use
    that domain to generate message IDs. So if you need to change the domain
    used to generate message IDs for every local post from your server, just
    add virtualhost: and domain: keys to every access stanza in readers.conf.

    This is really overkill for this option, and eventually the domain:
    parameter in inn.conf will probably be changed to allow this to be
    modified for the whole server. (Right now, domain: in inn.conf means
    something completely different.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------

    Subject: 6.8. Generate MRTG graphs for INN

    INN's CNFS storage system has direct support for producing information
    suitable for MRTG graphs on the usage of the CNFS cycbuffs. Running
    cnfsstat -m <cycbuf> will generate output suitable for MRTG, and running cnfsstat -p will generate sample MRTG configuration fragments for each
    cycbuff.

    To generate MRTG graphs of the usage of the buffindexed overview system,
    try the following configuration fragment:

    Target[overview-BUFF]: `/usr/local/etc/mrtg/overview.sh`
    MaxBytes[overview-BUFF]: 100
    Title[overview-BUFF]: BUFF1 Usage
    Options[overview-BUFF]: growright gauge
    YLegend[overview-BUFF]: Overview Buffers
    ShortLegend[overview-BUFF]: %

    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Mon Nov 21 08:01:04 2022
    [continued from previous message]

    investigate further. INN does not generally log enough information
    about outgoing articles to be able to tell more from your server alone.

    It may be possible to get a slight bit of information from the remote
    server by connecting with telnet (usually to port 119) and issuing
    "IHAVE <message-id>". The peer may respond with something like "435
    Duplicate" which means that the problem is not likely to be with your
    server (it may be still a problem with the article itself). If the
    peer responds with something like "335", your server probably did not
    offer the article after all.

    If you really are at a dead end and need to get more information about
    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports and to mail messages to moderators, and it assumes that you
    have a program installed as /usr/sbin/sendmail or /usr/lib/sendmail that
    it can use to do this. It does not speak SMTP, nor is it likely to ever
    speak SMTP; it's hard enough maintaining a package to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header field (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header field.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, it's worth noting that with a shared Perl library,
    Perl on this platform doesn't apparently generate the right link magic
    to include the path to the dynamic Perl libraries. You need to either
    set LD_RUN_PATH before building or LD_LIBRARY_PATH before running any
    binaries so that they can find the Perl libraries. (The former is
    preferred, since then the path is encoded into the binaries and you
    don't have to remember to set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    Note that rejected articles are remembered during the number of days
    specified by the /remember/ line in expire.ctl; so, in case you forgot
    to change the above parameters, you'll have to wait that number of
    days before being able to inject them again. Another possibility is to
    set /remember/ to 0, run the expire process (for instance via news.daily
    called with the same parameters as in crontab, plus "notdaily"),
    undo the change in expire.ctl and then start the feed again.

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles. The result file
    contains tokens ordered by arrival time on the old server (which is
    usually roughly the same as the posting time). In case the history
    file was not populated chronologically, it is better to sort it by
    posting time so that articles are fed in the right order. This can be
    achieved with the following command:

    sort -t '~' -k3n < history > history.sorted

    And then, consider history.sorted instead of history for the next steps.

    If the new server has just been installed or is known not to already have
    the articles you will feed it, you may want to add the "-c" flag to
    innxmit so as to skip the check for the presence of every article before transferring them.

    In case you wish to only feed articles arrived on the old server between
    two dates, you can adapt the previous commands with a condition on the
    $arrived variable (or the $posted variable if you prefer to use the Date
    header field instead of the actual arrival time). For instance, the
    following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    Other conditions may be added in the print line to select a subset of
    articles to feed. For instance if you want only articles for the fr.* hierarchy, you may add the following condition which will retrieve and
    parse the contents of the Xref field of every article to find the
    information (note that it will take some time to run, depending on the
    number of articles to parse):

    and qx/sm -q -H "$_" | grep Xref/ =~ / fr\.\S+:\d+/

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it. The "-d" flag
    is useful to add if you want to see the feeding progress.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref header fields. Current versions of INN will always add an Xref header field, but very old versions (earlier 1.x versions) will only add an Xref header field to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles to add an Xref header field.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups header field
    body is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using
    the tradspool storage method. The newsgroup of an article is stored in
    the Newsgroups header field and in the Xref header field of the article
    as stored on disk (and possibly in Followup-To), as well as determining
    where the overview information is stored, and in the case of tradspool
    is also encoded in the article's storage token. To rename a newsgroup
    in tradspool, stop the server, move the directory containing all of
    the articles to its appropriate new location in the news spool, edit
    every article to change the old name to the new name in Newsgroups, Followup-To, and Xref, create the new newsgroup with ctlinnd newgroup,
    and then rebuild history and overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups header fields didn't break those links, you
    may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To header fields of the articles in that newsgroup and then rebuild history
    and overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To header
    fields to rename the newsgroup, and then create a file containing paths to
    all of the articles, one per line. You can then use that file as input to innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message IDs generated by INN will use the domain of the
    local system for the right-hand-side of the message ID. In some cases,
    this isn't desirable for various reasons (the server may have an internal
    name that doesn't make sense on Usenet at large, or one may not want to
    expose the name of the server).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will use
    that domain to generate message IDs. So if you need to change the domain
    used to generate message IDs for every local post from your server, just
    add virtualhost: and domain: keys to every access stanza in readers.conf.

    This is really overkill for this option, and eventually the domain:
    parameter in inn.conf will probably be changed to allow this to be
    modified for the whole server. (Right now, domain: in inn.conf means
    something completely different.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------

    Subject: 6.8. Generate MRTG graphs for INN

    INN's CNFS storage system has direct support for producing information
    suitable for MRTG graphs on the usage of the CNFS cycbuffs. Running
    cnfsstat -m <cycbuf> will generate output suitable for MRTG, and running cnfsstat -p will generate sample MRTG configuration fragments for each
    cycbuff.

    To generate MRTG graphs of the usage of the buffindexed overview system,
    try the following configuration fragment:

    Target[overview-BUFF]: `/usr/local/etc/mrtg/overview.sh`
    MaxBytes[overview-BUFF]: 100
    Title[overview-BUFF]: BUFF1 Usage
    Options[overview-BUFF]: growright gauge
    YLegend[overview-BUFF]: Overview Buffers
    ShortLegend[overview-BUFF]: %

    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Wed Dec 21 08:01:02 2022
    [continued from previous message]

    investigate further. INN does not generally log enough information
    about outgoing articles to be able to tell more from your server alone.

    It may be possible to get a slight bit of information from the remote
    server by connecting with telnet (usually to port 119) and issuing
    "IHAVE <message-id>". The peer may respond with something like "435
    Duplicate" which means that the problem is not likely to be with your
    server (it may be still a problem with the article itself). If the
    peer responds with something like "335", your server probably did not
    offer the article after all.

    If you really are at a dead end and need to get more information about
    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports and to mail messages to moderators, and it assumes that you
    have a program installed as /usr/sbin/sendmail or /usr/lib/sendmail that
    it can use to do this. It does not speak SMTP, nor is it likely to ever
    speak SMTP; it's hard enough maintaining a package to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header field (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header field.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, it's worth noting that with a shared Perl library,
    Perl on this platform doesn't apparently generate the right link magic
    to include the path to the dynamic Perl libraries. You need to either
    set LD_RUN_PATH before building or LD_LIBRARY_PATH before running any
    binaries so that they can find the Perl libraries. (The former is
    preferred, since then the path is encoded into the binaries and you
    don't have to remember to set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    Note that rejected articles are remembered during the number of days
    specified by the /remember/ line in expire.ctl; so, in case you forgot
    to change the above parameters, you'll have to wait that number of
    days before being able to inject them again. Another possibility is to
    set /remember/ to 0, run the expire process (for instance via news.daily
    called with the same parameters as in crontab, plus "notdaily"),
    undo the change in expire.ctl and then start the feed again.

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles. The result file
    contains tokens ordered by arrival time on the old server (which is
    usually roughly the same as the posting time). In case the history
    file was not populated chronologically, it is better to sort it by
    posting time so that articles are fed in the right order. This can be
    achieved with the following command:

    sort -t '~' -k3n < history > history.sorted

    And then, consider history.sorted instead of history for the next steps.

    If the new server has just been installed or is known not to already have
    the articles you will feed it, you may want to add the "-c" flag to
    innxmit so as to skip the check for the presence of every article before transferring them.

    In case you wish to only feed articles arrived on the old server between
    two dates, you can adapt the previous commands with a condition on the
    $arrived variable (or the $posted variable if you prefer to use the Date
    header field instead of the actual arrival time). For instance, the
    following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    Other conditions may be added in the print line to select a subset of
    articles to feed. For instance if you want only articles for the fr.* hierarchy, you may add the following condition which will retrieve and
    parse the contents of the Xref field of every article to find the
    information (note that it will take some time to run, depending on the
    number of articles to parse):

    and qx/sm -q -H "$_" | grep Xref/ =~ / fr\.\S+:\d+/

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it. The "-d" flag
    is useful to add if you want to see the feeding progress.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref header fields. Current versions of INN will always add an Xref header field, but very old versions (earlier 1.x versions) will only add an Xref header field to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles to add an Xref header field.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups header field
    body is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using
    the tradspool storage method. The newsgroup of an article is stored in
    the Newsgroups header field and in the Xref header field of the article
    as stored on disk (and possibly in Followup-To), as well as determining
    where the overview information is stored, and in the case of tradspool
    is also encoded in the article's storage token. To rename a newsgroup
    in tradspool, stop the server, move the directory containing all of
    the articles to its appropriate new location in the news spool, edit
    every article to change the old name to the new name in Newsgroups, Followup-To, and Xref, create the new newsgroup with ctlinnd newgroup,
    and then rebuild history and overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups header fields didn't break those links, you
    may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To header fields of the articles in that newsgroup and then rebuild history
    and overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To header
    fields to rename the newsgroup, and then create a file containing paths to
    all of the articles, one per line. You can then use that file as input to innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message IDs generated by INN will use the domain of the
    local system for the right-hand-side of the message ID. In some cases,
    this isn't desirable for various reasons (the server may have an internal
    name that doesn't make sense on Usenet at large, or one may not want to
    expose the name of the server).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will use
    that domain to generate message IDs. So if you need to change the domain
    used to generate message IDs for every local post from your server, just
    add virtualhost: and domain: keys to every access stanza in readers.conf.

    This is really overkill for this option, and eventually the domain:
    parameter in inn.conf will probably be changed to allow this to be
    modified for the whole server. (Right now, domain: in inn.conf means
    something completely different.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------

    Subject: 6.8. Generate MRTG graphs for INN

    INN's CNFS storage system has direct support for producing information
    suitable for MRTG graphs on the usage of the CNFS cycbuffs. Running
    cnfsstat -m <cycbuf> will generate output suitable for MRTG, and running cnfsstat -p will generate sample MRTG configuration fragments for each
    cycbuff.

    To generate MRTG graphs of the usage of the buffindexed overview system,
    try the following configuration fragment:

    Target[overview-BUFF]: `/usr/local/etc/mrtg/overview.sh`
    MaxBytes[overview-BUFF]: 100
    Title[overview-BUFF]: BUFF1 Usage
    Options[overview-BUFF]: growright gauge
    YLegend[overview-BUFF]: Overview Buffers
    ShortLegend[overview-BUFF]: %

    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Sat Jan 21 08:01:03 2023
    [continued from previous message]

    investigate further. INN does not generally log enough information
    about outgoing articles to be able to tell more from your server alone.

    It may be possible to get a slight bit of information from the remote
    server by connecting with telnet (usually to port 119) and issuing
    "IHAVE <message-id>". The peer may respond with something like "435
    Duplicate" which means that the problem is not likely to be with your
    server (it may be still a problem with the article itself). If the
    peer responds with something like "335", your server probably did not
    offer the article after all.

    If you really are at a dead end and need to get more information about
    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports and to mail messages to moderators, and it assumes that you
    have a program installed as /usr/sbin/sendmail or /usr/lib/sendmail that
    it can use to do this. It does not speak SMTP, nor is it likely to ever
    speak SMTP; it's hard enough maintaining a package to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header field (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header field.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, it's worth noting that with a shared Perl library,
    Perl on this platform doesn't apparently generate the right link magic
    to include the path to the dynamic Perl libraries. You need to either
    set LD_RUN_PATH before building or LD_LIBRARY_PATH before running any
    binaries so that they can find the Perl libraries. (The former is
    preferred, since then the path is encoded into the binaries and you
    don't have to remember to set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    Note that rejected articles are remembered during the number of days
    specified by the /remember/ line in expire.ctl; so, in case you forgot
    to change the above parameters, you'll have to wait that number of
    days before being able to inject them again. Another possibility is to
    set /remember/ to 0, run the expire process (for instance via news.daily
    called with the same parameters as in crontab, plus "notdaily"),
    undo the change in expire.ctl and then start the feed again.

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles. The result file
    contains tokens ordered by arrival time on the old server (which is
    usually roughly the same as the posting time). In case the history
    file was not populated chronologically, it is better to sort it by
    posting time so that articles are fed in the right order. This can be
    achieved with the following command:

    sort -t '~' -k3n < history > history.sorted

    And then, consider history.sorted instead of history for the next steps.

    If the new server has just been installed or is known not to already have
    the articles you will feed it, you may want to add the "-c" flag to
    innxmit so as to skip the check for the presence of every article before transferring them.

    In case you wish to only feed articles arrived on the old server between
    two dates, you can adapt the previous commands with a condition on the
    $arrived variable (or the $posted variable if you prefer to use the Date
    header field instead of the actual arrival time). For instance, the
    following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    Other conditions may be added in the print line to select a subset of
    articles to feed. For instance if you want only articles for the fr.* hierarchy, you may add the following condition which will retrieve and
    parse the contents of the Xref field of every article to find the
    information (note that it will take some time to run, depending on the
    number of articles to parse):

    and qx/sm -q -H "$_" | grep Xref/ =~ / fr\.\S+:\d+/

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it. The "-d" flag
    is useful to add if you want to see the feeding progress.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref header fields. Current versions of INN will always add an Xref header field, but very old versions (earlier 1.x versions) will only add an Xref header field to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles to add an Xref header field.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups header field
    body is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using
    the tradspool storage method. The newsgroup of an article is stored in
    the Newsgroups header field and in the Xref header field of the article
    as stored on disk (and possibly in Followup-To), as well as determining
    where the overview information is stored, and in the case of tradspool
    is also encoded in the article's storage token. To rename a newsgroup
    in tradspool, stop the server, move the directory containing all of
    the articles to its appropriate new location in the news spool, edit
    every article to change the old name to the new name in Newsgroups, Followup-To, and Xref, create the new newsgroup with ctlinnd newgroup,
    and then rebuild history and overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups header fields didn't break those links, you
    may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To header fields of the articles in that newsgroup and then rebuild history
    and overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To header
    fields to rename the newsgroup, and then create a file containing paths to
    all of the articles, one per line. You can then use that file as input to innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message IDs generated by INN will use the domain of the
    local system for the right-hand-side of the message ID. In some cases,
    this isn't desirable for various reasons (the server may have an internal
    name that doesn't make sense on Usenet at large, or one may not want to
    expose the name of the server).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will use
    that domain to generate message IDs. So if you need to change the domain
    used to generate message IDs for every local post from your server, just
    add virtualhost: and domain: keys to every access stanza in readers.conf.

    This is really overkill for this option, and eventually the domain:
    parameter in inn.conf will probably be changed to allow this to be
    modified for the whole server. (Right now, domain: in inn.conf means
    something completely different.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------

    Subject: 6.8. Generate MRTG graphs for INN

    INN's CNFS storage system has direct support for producing information
    suitable for MRTG graphs on the usage of the CNFS cycbuffs. Running
    cnfsstat -m <cycbuf> will generate output suitable for MRTG, and running cnfsstat -p will generate sample MRTG configuration fragments for each
    cycbuff.

    To generate MRTG graphs of the usage of the buffindexed overview system,
    try the following configuration fragment:

    Target[overview-BUFF]: `/usr/local/etc/mrtg/overview.sh`
    MaxBytes[overview-BUFF]: 100
    Title[overview-BUFF]: BUFF1 Usage
    Options[overview-BUFF]: growright gauge
    YLegend[overview-BUFF]: Overview Buffers
    ShortLegend[overview-BUFF]: %

    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Tue Feb 21 08:01:02 2023
    [continued from previous message]

    investigate further. INN does not generally log enough information
    about outgoing articles to be able to tell more from your server alone.

    It may be possible to get a slight bit of information from the remote
    server by connecting with telnet (usually to port 119) and issuing
    "IHAVE <message-id>". The peer may respond with something like "435
    Duplicate" which means that the problem is not likely to be with your
    server (it may be still a problem with the article itself). If the
    peer responds with something like "335", your server probably did not
    offer the article after all.

    If you really are at a dead end and need to get more information about
    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports, mail messages to moderators, gateway news to mail, send statistics to the TOP1000 project, mail errors to the news administrator,
    etc. and it assumes that you have a program installed as /usr/sbin/sendmail
    or /usr/lib/sendmail that it can use to do this. It does not speak SMTP,
    nor is it likely to ever speak SMTP; it's hard enough maintaining a package
    to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header field (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header field.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, it's worth noting that with a shared Perl library,
    Perl on this platform doesn't apparently generate the right link magic
    to include the path to the dynamic Perl libraries. You need to either
    set LD_RUN_PATH before building or LD_LIBRARY_PATH before running any
    binaries so that they can find the Perl libraries. (The former is
    preferred, since then the path is encoded into the binaries and you
    don't have to remember to set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    Note that rejected articles are remembered during the number of days
    specified by the /remember/ line in expire.ctl; so, in case you forgot
    to change the above parameters, you'll have to wait that number of
    days before being able to inject them again. Another possibility is to
    set /remember/ to 0, run the expire process (for instance via news.daily
    called with the same parameters as in crontab, plus "notdaily"),
    undo the change in expire.ctl and then start the feed again.

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles. The result file
    contains tokens ordered by arrival time on the old server (which is
    usually roughly the same as the posting time). In case the history
    file was not populated chronologically, it is better to sort it by
    posting time so that articles are fed in the right order. This can be
    achieved with the following command:

    sort -t '~' -k3n < history > history.sorted

    And then, consider history.sorted instead of history for the next steps.

    If the new server has just been installed or is known not to already have
    the articles you will feed it, you may want to add the "-c" flag to
    innxmit so as to skip the check for the presence of every article before transferring them.

    In case you wish to only feed articles arrived on the old server between
    two dates, you can adapt the previous commands with a condition on the
    $arrived variable (or the $posted variable if you prefer to use the Date
    header field instead of the actual arrival time). For instance, the
    following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    Other conditions may be added in the print line to select a subset of
    articles to feed. For instance if you want only articles for the fr.* hierarchy, you may add the following condition which will retrieve and
    parse the contents of the Xref field of every article to find the
    information (note that it will take some time to run, depending on the
    number of articles to parse):

    and qx/sm -q -H "$_" | grep Xref/ =~ / fr\.\S+:\d+/

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it. The "-d" flag
    is useful to add if you want to see the feeding progress.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref header fields. Current versions of INN will always add an Xref header field, but very old versions (earlier 1.x versions) will only add an Xref header field to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles to add an Xref header field.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups header field
    body is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using
    the tradspool storage method. The newsgroup of an article is stored in
    the Newsgroups header field and in the Xref header field of the article
    as stored on disk (and possibly in Followup-To), as well as determining
    where the overview information is stored, and in the case of tradspool
    is also encoded in the article's storage token. To rename a newsgroup
    in tradspool, stop the server, move the directory containing all of
    the articles to its appropriate new location in the news spool, edit
    every article to change the old name to the new name in Newsgroups, Followup-To, and Xref, create the new newsgroup with ctlinnd newgroup,
    and then rebuild history and overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups header fields didn't break those links, you
    may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To header fields of the articles in that newsgroup and then rebuild history
    and overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To header
    fields to rename the newsgroup, and then create a file containing paths to
    all of the articles, one per line. You can then use that file as input to innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message identifier generated by INN will use the fully qualified domain name of the local system for the right-hand side of
    message IDs. In some cases, this isn't desirable for various reasons
    (the server may have an internal name that doesn't make sense on Usenet
    at large, or one may not want to expose the name of the server).

    INN still does not have a dedicated parameter to change the right-hand
    side of message identifiers. While waiting for it, there's a trick
    involving a special use of the domain: parameter (normally meant to be
    use only for what is related to DNS).

    In INN 2.7.1 and later, you can use an explicit domain: key in an access
    stanza of readers.conf to use its value as the right-hand side of message identifiers. (Even though domain: is set in inn.conf, note that this
    parameter also needs being present in readers.conf so as to trigger the expected behaviour).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will
    use that domain to generate message IDs. So if you need to change the
    domain used to generate message IDs for every local post from your
    server, just add virtualhost: and domain: keys to every access stanza
    in readers.conf. (Yes, this is really overkill for this option.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------

    Subject: 6.8. Generate MRTG graphs for INN

    INN's CNFS storage system has direct support for producing information
    suitable for MRTG graphs on the usage of the CNFS cycbuffs. Running

    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Fri Apr 21 07:01:01 2023
    [continued from previous message]

    misconfiguration like the ones described above.

    In either case, if you're sure that the article was offered to the peer
    and not spooled, you will need the assistance of the peer's admin to
    investigate further. INN does not generally log enough information
    about outgoing articles to be able to tell more from your server alone.

    It may be possible to get a slight bit of information from the remote
    server by connecting with telnet (usually to port 119) and issuing
    "IHAVE <message-id>". The peer may respond with something like "435
    Duplicate" which means that the problem is not likely to be with your
    server (it may be still a problem with the article itself). If the
    peer responds with something like "335", your server probably did not
    offer the article after all.

    If you really are at a dead end and need to get more information about
    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports, mail messages to moderators, gateway news to mail, send statistics to the TOP1000 project, mail errors to the news administrator,
    etc. and it assumes that you have a program installed as /usr/sbin/sendmail
    or /usr/lib/sendmail that it can use to do this. It does not speak SMTP,
    nor is it likely to ever speak SMTP; it's hard enough maintaining a package
    to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header field (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header field.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, it's worth noting that with a shared Perl library,
    Perl on this platform doesn't apparently generate the right link magic
    to include the path to the dynamic Perl libraries. You need to either
    set LD_RUN_PATH before building or LD_LIBRARY_PATH before running any
    binaries so that they can find the Perl libraries. (The former is
    preferred, since then the path is encoded into the binaries and you
    don't have to remember to set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    Note that rejected articles are remembered during the number of days
    specified by the /remember/ line in expire.ctl; so, in case you forgot
    to change the above parameters, you'll have to wait that number of
    days before being able to inject them again. Another possibility is to
    set /remember/ to 0, run the expire process (for instance via news.daily
    called with the same parameters as in crontab, plus "notdaily"),
    undo the change in expire.ctl and then start the feed again.

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles. The result file
    contains tokens ordered by arrival time on the old server (which is
    usually roughly the same as the posting time). In case the history
    file was not populated chronologically, it is better to sort it by
    posting time so that articles are fed in the right order. This can be
    achieved with the following command:

    sort -t '~' -k3n < history > history.sorted

    And then, consider history.sorted instead of history for the next steps.

    If the new server has just been installed or is known not to already have
    the articles you will feed it, you may want to add the "-c" flag to
    innxmit so as to skip the check for the presence of every article before transferring them.

    In case you wish to only feed articles arrived on the old server between
    two dates, you can adapt the previous commands with a condition on the
    $arrived variable (or the $posted variable if you prefer to use the Date
    header field instead of the actual arrival time). For instance, the
    following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    Other conditions may be added in the print line to select a subset of
    articles to feed. For instance if you want only articles for the fr.* hierarchy, you may add the following condition which will retrieve and
    parse the contents of the Xref field of every article to find the
    information (note that it will take some time to run, depending on the
    number of articles to parse):

    and qx/sm -q -H "$_" | grep Xref/ =~ / fr\.\S+:\d+/

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it. The "-d" flag
    is useful to add if you want to see the feeding progress.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref header fields. Current versions of INN will always add an Xref header field, but very old versions (earlier 1.x versions) will only add an Xref header field to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles to add an Xref header field.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups header field
    body is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using
    the tradspool storage method. The newsgroup of an article is stored in
    the Newsgroups header field and in the Xref header field of the article
    as stored on disk (and possibly in Followup-To), as well as determining
    where the overview information is stored, and in the case of tradspool
    is also encoded in the article's storage token. To rename a newsgroup
    in tradspool, stop the server, move the directory containing all of
    the articles to its appropriate new location in the news spool, edit
    every article to change the old name to the new name in Newsgroups, Followup-To, and Xref, create the new newsgroup with ctlinnd newgroup,
    and then rebuild history and overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups header fields didn't break those links, you
    may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To header fields of the articles in that newsgroup and then rebuild history
    and overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To header
    fields to rename the newsgroup, and then create a file containing paths to
    all of the articles, one per line. You can then use that file as input to innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message identifier generated by INN will use the fully qualified domain name of the local system for the right-hand side of
    message IDs. In some cases, this isn't desirable for various reasons
    (the server may have an internal name that doesn't make sense on Usenet
    at large, or one may not want to expose the name of the server).

    INN still does not have a dedicated parameter to change the right-hand
    side of message identifiers. While waiting for it, there's a trick
    involving a special use of the domain: parameter (normally meant to be
    use only for what is related to DNS).

    In INN 2.7.1 and later, you can use an explicit domain: key in an access
    stanza of readers.conf to use its value as the right-hand side of message identifiers. (Even though domain: is set in inn.conf, note that this
    parameter also needs being present in readers.conf so as to trigger the expected behaviour).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will
    use that domain to generate message IDs. So if you need to change the
    domain used to generate message IDs for every local post from your
    server, just add virtualhost: and domain: keys to every access stanza
    in readers.conf. (Yes, this is really overkill for this option.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------


    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Sun May 21 07:01:03 2023
    [continued from previous message]

    misconfiguration like the ones described above.

    In either case, if you're sure that the article was offered to the peer
    and not spooled, you will need the assistance of the peer's admin to
    investigate further. INN does not generally log enough information
    about outgoing articles to be able to tell more from your server alone.

    It may be possible to get a slight bit of information from the remote
    server by connecting with telnet (usually to port 119) and issuing
    "IHAVE <message-id>". The peer may respond with something like "435
    Duplicate" which means that the problem is not likely to be with your
    server (it may be still a problem with the article itself). If the
    peer responds with something like "335", your server probably did not
    offer the article after all.

    If you really are at a dead end and need to get more information about
    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports, mail messages to moderators, gateway news to mail, send statistics to the TOP1000 project, mail errors to the news administrator,
    etc. and it assumes that you have a program installed as /usr/sbin/sendmail
    or /usr/lib/sendmail that it can use to do this. It does not speak SMTP,
    nor is it likely to ever speak SMTP; it's hard enough maintaining a package
    to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header field (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header field.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, it's worth noting that with a shared Perl library,
    Perl on this platform doesn't apparently generate the right link magic
    to include the path to the dynamic Perl libraries. You need to either
    set LD_RUN_PATH before building or LD_LIBRARY_PATH before running any
    binaries so that they can find the Perl libraries. (The former is
    preferred, since then the path is encoded into the binaries and you
    don't have to remember to set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    Note that rejected articles are remembered during the number of days
    specified by the /remember/ line in expire.ctl; so, in case you forgot
    to change the above parameters, you'll have to wait that number of
    days before being able to inject them again. Another possibility is to
    set /remember/ to 0, run the expire process (for instance via news.daily
    called with the same parameters as in crontab, plus "notdaily"),
    undo the change in expire.ctl and then start the feed again.

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles. The result file
    contains tokens ordered by arrival time on the old server (which is
    usually roughly the same as the posting time). In case the history
    file was not populated chronologically, it is better to sort it by
    posting time so that articles are fed in the right order. This can be
    achieved with the following command:

    sort -t '~' -k3n < history > history.sorted

    And then, consider history.sorted instead of history for the next steps.

    If the new server has just been installed or is known not to already have
    the articles you will feed it, you may want to add the "-c" flag to
    innxmit so as to skip the check for the presence of every article before transferring them.

    In case you wish to only feed articles arrived on the old server between
    two dates, you can adapt the previous commands with a condition on the
    $arrived variable (or the $posted variable if you prefer to use the Date
    header field instead of the actual arrival time). For instance, the
    following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    Other conditions may be added in the print line to select a subset of
    articles to feed. For instance if you want only articles for the fr.* hierarchy, you may add the following condition which will retrieve and
    parse the contents of the Xref field of every article to find the
    information (note that it will take some time to run, depending on the
    number of articles to parse):

    and qx/sm -q -H "$_" | grep Xref/ =~ / fr\.\S+:\d+/

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it. The "-d" flag
    is useful to add if you want to see the feeding progress.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref header fields. Current versions of INN will always add an Xref header field, but very old versions (earlier 1.x versions) will only add an Xref header field to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles to add an Xref header field.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups header field
    body is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using
    the tradspool storage method. The newsgroup of an article is stored in
    the Newsgroups header field and in the Xref header field of the article
    as stored on disk (and possibly in Followup-To), as well as determining
    where the overview information is stored, and in the case of tradspool
    is also encoded in the article's storage token. To rename a newsgroup
    in tradspool, stop the server, move the directory containing all of
    the articles to its appropriate new location in the news spool, edit
    every article to change the old name to the new name in Newsgroups, Followup-To, and Xref, create the new newsgroup with ctlinnd newgroup,
    and then rebuild history and overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups header fields didn't break those links, you
    may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To header fields of the articles in that newsgroup and then rebuild history
    and overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To header
    fields to rename the newsgroup, and then create a file containing paths to
    all of the articles, one per line. You can then use that file as input to innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message identifier generated by INN will use the fully qualified domain name of the local system for the right-hand side of
    message IDs. In some cases, this isn't desirable for various reasons
    (the server may have an internal name that doesn't make sense on Usenet
    at large, or one may not want to expose the name of the server).

    INN still does not have a dedicated parameter to change the right-hand
    side of message identifiers. While waiting for it, there's a trick
    involving a special use of the domain: parameter (normally meant to be
    use only for what is related to DNS).

    In INN 2.7.1 and later, you can use an explicit domain: key in an access
    stanza of readers.conf to use its value as the right-hand side of message identifiers. (Even though domain: is set in inn.conf, note that this
    parameter also needs being present in readers.conf so as to trigger the expected behaviour).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will
    use that domain to generate message IDs. So if you need to change the
    domain used to generate message IDs for every local post from your
    server, just add virtualhost: and domain: keys to every access stanza
    in readers.conf. (Yes, this is really overkill for this option.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------


    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Wed Jun 21 07:01:04 2023
    [continued from previous message]

    misconfiguration like the ones described above.

    In either case, if you're sure that the article was offered to the peer
    and not spooled, you will need the assistance of the peer's admin to
    investigate further. INN does not generally log enough information
    about outgoing articles to be able to tell more from your server alone.

    It may be possible to get a slight bit of information from the remote
    server by connecting with telnet (usually to port 119) and issuing
    "IHAVE <message-id>". The peer may respond with something like "435
    Duplicate" which means that the problem is not likely to be with your
    server (it may be still a problem with the article itself). If the
    peer responds with something like "335", your server probably did not
    offer the article after all.

    If you really are at a dead end and need to get more information about
    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports, mail messages to moderators, gateway news to mail, send statistics to the TOP1000 project, mail errors to the news administrator,
    etc. and it assumes that you have a program installed as /usr/sbin/sendmail
    or /usr/lib/sendmail that it can use to do this. It does not speak SMTP,
    nor is it likely to ever speak SMTP; it's hard enough maintaining a package
    to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header field (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header field.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, it's worth noting that with a shared Perl library,
    Perl on this platform doesn't apparently generate the right link magic
    to include the path to the dynamic Perl libraries. You need to either
    set LD_RUN_PATH before building or LD_LIBRARY_PATH before running any
    binaries so that they can find the Perl libraries. (The former is
    preferred, since then the path is encoded into the binaries and you
    don't have to remember to set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    Note that rejected articles are remembered during the number of days
    specified by the /remember/ line in expire.ctl; so, in case you forgot
    to change the above parameters, you'll have to wait that number of
    days before being able to inject them again. Another possibility is to
    set /remember/ to 0, run the expire process (for instance via news.daily
    called with the same parameters as in crontab, plus "notdaily"),
    undo the change in expire.ctl and then start the feed again.

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles. The result file
    contains tokens ordered by arrival time on the old server (which is
    usually roughly the same as the posting time). In case the history
    file was not populated chronologically, it is better to sort it by
    posting time so that articles are fed in the right order. This can be
    achieved with the following command:

    sort -t '~' -k3n < history > history.sorted

    And then, consider history.sorted instead of history for the next steps.

    If the new server has just been installed or is known not to already have
    the articles you will feed it, you may want to add the "-c" flag to
    innxmit so as to skip the check for the presence of every article before transferring them.

    In case you wish to only feed articles arrived on the old server between
    two dates, you can adapt the previous commands with a condition on the
    $arrived variable (or the $posted variable if you prefer to use the Date
    header field instead of the actual arrival time). For instance, the
    following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    Other conditions may be added in the print line to select a subset of
    articles to feed. For instance if you want only articles for the fr.* hierarchy, you may add the following condition which will retrieve and
    parse the contents of the Xref field of every article to find the
    information (note that it will take some time to run, depending on the
    number of articles to parse):

    and qx/sm -q -H "$_" | grep Xref/ =~ / fr\.\S+:\d+/

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it. The "-d" flag
    is useful to add if you want to see the feeding progress.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref header fields. Current versions of INN will always add an Xref header field, but very old versions (earlier 1.x versions) will only add an Xref header field to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles to add an Xref header field.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups header field
    body is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using
    the tradspool storage method. The newsgroup of an article is stored in
    the Newsgroups header field and in the Xref header field of the article
    as stored on disk (and possibly in Followup-To), as well as determining
    where the overview information is stored, and in the case of tradspool
    is also encoded in the article's storage token. To rename a newsgroup
    in tradspool, stop the server, move the directory containing all of
    the articles to its appropriate new location in the news spool, edit
    every article to change the old name to the new name in Newsgroups, Followup-To, and Xref, create the new newsgroup with ctlinnd newgroup,
    and then rebuild history and overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups header fields didn't break those links, you
    may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To header fields of the articles in that newsgroup and then rebuild history
    and overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To header
    fields to rename the newsgroup, and then create a file containing paths to
    all of the articles, one per line. You can then use that file as input to innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message identifier generated by INN will use the fully qualified domain name of the local system for the right-hand side of
    message IDs. In some cases, this isn't desirable for various reasons
    (the server may have an internal name that doesn't make sense on Usenet
    at large, or one may not want to expose the name of the server).

    INN still does not have a dedicated parameter to change the right-hand
    side of message identifiers. While waiting for it, there's a trick
    involving a special use of the domain: parameter (normally meant to be
    use only for what is related to DNS).

    In INN 2.7.1 and later, you can use an explicit domain: key in an access
    stanza of readers.conf to use its value as the right-hand side of message identifiers. (Even though domain: is set in inn.conf, note that this
    parameter also needs being present in readers.conf so as to trigger the expected behaviour).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will
    use that domain to generate message IDs. So if you need to change the
    domain used to generate message IDs for every local post from your
    server, just add virtualhost: and domain: keys to every access stanza
    in readers.conf. (Yes, this is really overkill for this option.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------


    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Fri Jul 21 07:01:04 2023
    [continued from previous message]

    misconfiguration like the ones described above.

    In either case, if you're sure that the article was offered to the peer
    and not spooled, you will need the assistance of the peer's admin to
    investigate further. INN does not generally log enough information
    about outgoing articles to be able to tell more from your server alone.

    It may be possible to get a slight bit of information from the remote
    server by connecting with telnet (usually to port 119) and issuing
    "IHAVE <message-id>". The peer may respond with something like "435
    Duplicate" which means that the problem is not likely to be with your
    server (it may be still a problem with the article itself). If the
    peer responds with something like "335", your server probably did not
    offer the article after all.

    If you really are at a dead end and need to get more information about
    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports, mail messages to moderators, gateway news to mail, send statistics to the TOP1000 project, mail errors to the news administrator,
    etc. and it assumes that you have a program installed as /usr/sbin/sendmail
    or /usr/lib/sendmail that it can use to do this. It does not speak SMTP,
    nor is it likely to ever speak SMTP; it's hard enough maintaining a package
    to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header field (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header field.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, it's worth noting that with a shared Perl library,
    Perl on this platform doesn't apparently generate the right link magic
    to include the path to the dynamic Perl libraries. You need to either
    set LD_RUN_PATH before building or LD_LIBRARY_PATH before running any
    binaries so that they can find the Perl libraries. (The former is
    preferred, since then the path is encoded into the binaries and you
    don't have to remember to set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    Note that rejected articles are remembered during the number of days
    specified by the /remember/ line in expire.ctl; so, in case you forgot
    to change the above parameters, you'll have to wait that number of
    days before being able to inject them again. Another possibility is to
    set /remember/ to 0, run the expire process (for instance via news.daily
    called with the same parameters as in crontab, plus "notdaily"),
    undo the change in expire.ctl and then start the feed again.

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles. The result file
    contains tokens ordered by arrival time on the old server (which is
    usually roughly the same as the posting time). In case the history
    file was not populated chronologically, it is better to sort it by
    posting time so that articles are fed in the right order. This can be
    achieved with the following command:

    sort -t '~' -k3n < history > history.sorted

    And then, consider history.sorted instead of history for the next steps.

    If the new server has just been installed or is known not to already have
    the articles you will feed it, you may want to add the "-c" flag to
    innxmit so as to skip the check for the presence of every article before transferring them.

    In case you wish to only feed articles arrived on the old server between
    two dates, you can adapt the previous commands with a condition on the
    $arrived variable (or the $posted variable if you prefer to use the Date
    header field instead of the actual arrival time). For instance, the
    following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    Other conditions may be added in the print line to select a subset of
    articles to feed. For instance if you want only articles for the fr.* hierarchy, you may add the following condition which will retrieve and
    parse the contents of the Xref field of every article to find the
    information (note that it will take some time to run, depending on the
    number of articles to parse):

    and qx/sm -q -H "$_" | grep Xref/ =~ / fr\.\S+:\d+/

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it. The "-d" flag
    is useful to add if you want to see the feeding progress.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref header fields. Current versions of INN will always add an Xref header field, but very old versions (earlier 1.x versions) will only add an Xref header field to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles to add an Xref header field.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups header field
    body is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using
    the tradspool storage method. The newsgroup of an article is stored in
    the Newsgroups header field and in the Xref header field of the article
    as stored on disk (and possibly in Followup-To), as well as determining
    where the overview information is stored, and in the case of tradspool
    is also encoded in the article's storage token. To rename a newsgroup
    in tradspool, stop the server, move the directory containing all of
    the articles to its appropriate new location in the news spool, edit
    every article to change the old name to the new name in Newsgroups, Followup-To, and Xref, create the new newsgroup with ctlinnd newgroup,
    and then rebuild history and overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups header fields didn't break those links, you
    may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To header fields of the articles in that newsgroup and then rebuild history
    and overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To header
    fields to rename the newsgroup, and then create a file containing paths to
    all of the articles, one per line. You can then use that file as input to innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message identifier generated by INN will use the fully qualified domain name of the local system for the right-hand side of
    message IDs. In some cases, this isn't desirable for various reasons
    (the server may have an internal name that doesn't make sense on Usenet
    at large, or one may not want to expose the name of the server).

    INN still does not have a dedicated parameter to change the right-hand
    side of message identifiers. While waiting for it, there's a trick
    involving a special use of the domain: parameter (normally meant to be
    use only for what is related to DNS).

    In INN 2.7.1 and later, you can use an explicit domain: key in an access
    stanza of readers.conf to use its value as the right-hand side of message identifiers. (Even though domain: is set in inn.conf, note that this
    parameter also needs being present in readers.conf so as to trigger the expected behaviour).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will
    use that domain to generate message IDs. So if you need to change the
    domain used to generate message IDs for every local post from your
    server, just add virtualhost: and domain: keys to every access stanza
    in readers.conf. (Yes, this is really overkill for this option.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------


    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Mon Aug 21 07:01:03 2023
    [continued from previous message]

    misconfiguration like the ones described above.

    In either case, if you're sure that the article was offered to the peer
    and not spooled, you will need the assistance of the peer's admin to
    investigate further. INN does not generally log enough information
    about outgoing articles to be able to tell more from your server alone.

    It may be possible to get a slight bit of information from the remote
    server by connecting with telnet (usually to port 119) and issuing
    "IHAVE <message-id>". The peer may respond with something like "435
    Duplicate" which means that the problem is not likely to be with your
    server (it may be still a problem with the article itself). If the
    peer responds with something like "335", your server probably did not
    offer the article after all.

    If you really are at a dead end and need to get more information about
    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports, mail messages to moderators, gateway news to mail, send statistics to the TOP1000 project, mail errors to the news administrator,
    etc. and it assumes that you have a program installed as /usr/sbin/sendmail
    or /usr/lib/sendmail that it can use to do this. It does not speak SMTP,
    nor is it likely to ever speak SMTP; it's hard enough maintaining a package
    to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header field (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header field.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, it's worth noting that with a shared Perl library,
    Perl on this platform doesn't apparently generate the right link magic
    to include the path to the dynamic Perl libraries. You need to either
    set LD_RUN_PATH before building or LD_LIBRARY_PATH before running any
    binaries so that they can find the Perl libraries. (The former is
    preferred, since then the path is encoded into the binaries and you
    don't have to remember to set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    Note that rejected articles are remembered during the number of days
    specified by the /remember/ line in expire.ctl; so, in case you forgot
    to change the above parameters, you'll have to wait that number of
    days before being able to inject them again. Another possibility is to
    set /remember/ to 0, run the expire process (for instance via news.daily
    called with the same parameters as in crontab, plus "notdaily"),
    undo the change in expire.ctl and then start the feed again.

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles. The result file
    contains tokens ordered by arrival time on the old server (which is
    usually roughly the same as the posting time). In case the history
    file was not populated chronologically, it is better to sort it by
    posting time so that articles are fed in the right order. This can be
    achieved with the following command:

    sort -t '~' -k3n < history > history.sorted

    And then, consider history.sorted instead of history for the next steps.

    If the new server has just been installed or is known not to already have
    the articles you will feed it, you may want to add the "-c" flag to
    innxmit so as to skip the check for the presence of every article before transferring them.

    In case you wish to only feed articles arrived on the old server between
    two dates, you can adapt the previous commands with a condition on the
    $arrived variable (or the $posted variable if you prefer to use the Date
    header field instead of the actual arrival time). For instance, the
    following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    Other conditions may be added in the print line to select a subset of
    articles to feed. For instance if you want only articles for the fr.* hierarchy, you may add the following condition which will retrieve and
    parse the contents of the Xref field of every article to find the
    information (note that it will take some time to run, depending on the
    number of articles to parse):

    and qx/sm -q -H "$_" | grep Xref/ =~ / fr\.\S+:\d+/

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it. The "-d" flag
    is useful to add if you want to see the feeding progress.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref header fields. Current versions of INN will always add an Xref header field, but very old versions (earlier 1.x versions) will only add an Xref header field to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles to add an Xref header field.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups header field
    body is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using
    the tradspool storage method. The newsgroup of an article is stored in
    the Newsgroups header field and in the Xref header field of the article
    as stored on disk (and possibly in Followup-To), as well as determining
    where the overview information is stored, and in the case of tradspool
    is also encoded in the article's storage token. To rename a newsgroup
    in tradspool, stop the server, move the directory containing all of
    the articles to its appropriate new location in the news spool, edit
    every article to change the old name to the new name in Newsgroups, Followup-To, and Xref, create the new newsgroup with ctlinnd newgroup,
    and then rebuild history and overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups header fields didn't break those links, you
    may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To header fields of the articles in that newsgroup and then rebuild history
    and overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To header
    fields to rename the newsgroup, and then create a file containing paths to
    all of the articles, one per line. You can then use that file as input to innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message identifier generated by INN will use the fully qualified domain name of the local system for the right-hand side of
    message IDs. In some cases, this isn't desirable for various reasons
    (the server may have an internal name that doesn't make sense on Usenet
    at large, or one may not want to expose the name of the server).

    INN still does not have a dedicated parameter to change the right-hand
    side of message identifiers. While waiting for it, there's a trick
    involving a special use of the domain: parameter (normally meant to be
    use only for what is related to DNS).

    In INN 2.7.1 and later, you can use an explicit domain: key in an access
    stanza of readers.conf to use its value as the right-hand side of message identifiers. (Even though domain: is set in inn.conf, note that this
    parameter also needs being present in readers.conf so as to trigger the expected behaviour).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will
    use that domain to generate message IDs. So if you need to change the
    domain used to generate message IDs for every local post from your
    server, just add virtualhost: and domain: keys to every access stanza
    in readers.conf. (Yes, this is really overkill for this option.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------


    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Thu Sep 21 07:01:03 2023
    [continued from previous message]

    misconfiguration like the ones described above.

    In either case, if you're sure that the article was offered to the peer
    and not spooled, you will need the assistance of the peer's admin to
    investigate further. INN does not generally log enough information
    about outgoing articles to be able to tell more from your server alone.

    It may be possible to get a slight bit of information from the remote
    server by connecting with telnet (usually to port 119) and issuing
    "IHAVE <message-id>". The peer may respond with something like "435
    Duplicate" which means that the problem is not likely to be with your
    server (it may be still a problem with the article itself). If the
    peer responds with something like "335", your server probably did not
    offer the article after all.

    If you really are at a dead end and need to get more information about
    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports, mail messages to moderators, gateway news to mail, send statistics to the TOP1000 project, mail errors to the news administrator,
    etc. and it assumes that you have a program installed as /usr/sbin/sendmail
    or /usr/lib/sendmail that it can use to do this. It does not speak SMTP,
    nor is it likely to ever speak SMTP; it's hard enough maintaining a package
    to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header field (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header field.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, it's worth noting that with a shared Perl library,
    Perl on this platform doesn't apparently generate the right link magic
    to include the path to the dynamic Perl libraries. You need to either
    set LD_RUN_PATH before building or LD_LIBRARY_PATH before running any
    binaries so that they can find the Perl libraries. (The former is
    preferred, since then the path is encoded into the binaries and you
    don't have to remember to set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    Note that rejected articles are remembered during the number of days
    specified by the /remember/ line in expire.ctl; so, in case you forgot
    to change the above parameters, you'll have to wait that number of
    days before being able to inject them again. Another possibility is to
    set /remember/ to 0, run the expire process (for instance via news.daily
    called with the same parameters as in crontab, plus "notdaily"),
    undo the change in expire.ctl and then start the feed again.

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles. The result file
    contains tokens ordered by arrival time on the old server (which is
    usually roughly the same as the posting time). In case the history
    file was not populated chronologically, it is better to sort it by
    posting time so that articles are fed in the right order. This can be
    achieved with the following command:

    sort -t '~' -k3n < history > history.sorted

    And then, consider history.sorted instead of history for the next steps.

    If the new server has just been installed or is known not to already have
    the articles you will feed it, you may want to add the "-c" flag to
    innxmit so as to skip the check for the presence of every article before transferring them.

    In case you wish to only feed articles arrived on the old server between
    two dates, you can adapt the previous commands with a condition on the
    $arrived variable (or the $posted variable if you prefer to use the Date
    header field instead of the actual arrival time). For instance, the
    following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    Other conditions may be added in the print line to select a subset of
    articles to feed. For instance if you want only articles for the fr.* hierarchy, you may add the following condition which will retrieve and
    parse the contents of the Xref field of every article to find the
    information (note that it will take some time to run, depending on the
    number of articles to parse):

    and qx/sm -q -H "$_" | grep Xref/ =~ / fr\.\S+:\d+/

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it. The "-d" flag
    is useful to add if you want to see the feeding progress.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref header fields. Current versions of INN will always add an Xref header field, but very old versions (earlier 1.x versions) will only add an Xref header field to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles to add an Xref header field.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups header field
    body is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using
    the tradspool storage method. The newsgroup of an article is stored in
    the Newsgroups header field and in the Xref header field of the article
    as stored on disk (and possibly in Followup-To), as well as determining
    where the overview information is stored, and in the case of tradspool
    is also encoded in the article's storage token. To rename a newsgroup
    in tradspool, stop the server, move the directory containing all of
    the articles to its appropriate new location in the news spool, edit
    every article to change the old name to the new name in Newsgroups, Followup-To, and Xref, create the new newsgroup with ctlinnd newgroup,
    and then rebuild history and overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups header fields didn't break those links, you
    may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To header fields of the articles in that newsgroup and then rebuild history
    and overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To header
    fields to rename the newsgroup, and then create a file containing paths to
    all of the articles, one per line. You can then use that file as input to innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message identifier generated by INN will use the fully qualified domain name of the local system for the right-hand side of
    message IDs. In some cases, this isn't desirable for various reasons
    (the server may have an internal name that doesn't make sense on Usenet
    at large, or one may not want to expose the name of the server).

    INN still does not have a dedicated parameter to change the right-hand
    side of message identifiers. While waiting for it, there's a trick
    involving a special use of the domain: parameter (normally meant to be
    use only for what is related to DNS).

    In INN 2.7.1 and later, you can use an explicit domain: key in an access
    stanza of readers.conf to use its value as the right-hand side of message identifiers. (Even though domain: is set in inn.conf, note that this
    parameter also needs being present in readers.conf so as to trigger the expected behaviour).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will
    use that domain to generate message IDs. So if you need to change the
    domain used to generate message IDs for every local post from your
    server, just add virtualhost: and domain: keys to every access stanza
    in readers.conf. (Yes, this is really overkill for this option.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------


    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Sat Oct 21 07:01:04 2023
    [continued from previous message]

    misconfiguration like the ones described above.

    In either case, if you're sure that the article was offered to the peer
    and not spooled, you will need the assistance of the peer's admin to
    investigate further. INN does not generally log enough information
    about outgoing articles to be able to tell more from your server alone.

    It may be possible to get a slight bit of information from the remote
    server by connecting with telnet (usually to port 119) and issuing
    "IHAVE <message-id>". The peer may respond with something like "435
    Duplicate" which means that the problem is not likely to be with your
    server (it may be still a problem with the article itself). If the
    peer responds with something like "335", your server probably did not
    offer the article after all.

    If you really are at a dead end and need to get more information about
    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports, mail messages to moderators, gateway news to mail, send statistics to the TOP1000 project, mail errors to the news administrator,
    etc. and it assumes that you have a program installed as /usr/sbin/sendmail
    or /usr/lib/sendmail that it can use to do this. It does not speak SMTP,
    nor is it likely to ever speak SMTP; it's hard enough maintaining a package
    to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header field (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header field.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, it's worth noting that with a shared Perl library,
    Perl on this platform doesn't apparently generate the right link magic
    to include the path to the dynamic Perl libraries. You need to either
    set LD_RUN_PATH before building or LD_LIBRARY_PATH before running any
    binaries so that they can find the Perl libraries. (The former is
    preferred, since then the path is encoded into the binaries and you
    don't have to remember to set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    Note that rejected articles are remembered during the number of days
    specified by the /remember/ line in expire.ctl; so, in case you forgot
    to change the above parameters, you'll have to wait that number of
    days before being able to inject them again. Another possibility is to
    set /remember/ to 0, run the expire process (for instance via news.daily
    called with the same parameters as in crontab, plus "notdaily"),
    undo the change in expire.ctl and then start the feed again.

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles. The result file
    contains tokens ordered by arrival time on the old server (which is
    usually roughly the same as the posting time). In case the history
    file was not populated chronologically, it is better to sort it by
    posting time so that articles are fed in the right order. This can be
    achieved with the following command:

    sort -t '~' -k3n < history > history.sorted

    And then, consider history.sorted instead of history for the next steps.

    If the new server has just been installed or is known not to already have
    the articles you will feed it, you may want to add the "-c" flag to
    innxmit so as to skip the check for the presence of every article before transferring them.

    In case you wish to only feed articles arrived on the old server between
    two dates, you can adapt the previous commands with a condition on the
    $arrived variable (or the $posted variable if you prefer to use the Date
    header field instead of the actual arrival time). For instance, the
    following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    Other conditions may be added in the print line to select a subset of
    articles to feed. For instance if you want only articles for the fr.* hierarchy, you may add the following condition which will retrieve and
    parse the contents of the Xref field of every article to find the
    information (note that it will take some time to run, depending on the
    number of articles to parse):

    and qx/sm -q -H "$_" | grep Xref/ =~ / fr\.\S+:\d+/

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it. The "-d" flag
    is useful to add if you want to see the feeding progress.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref header fields. Current versions of INN will always add an Xref header field, but very old versions (earlier 1.x versions) will only add an Xref header field to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles to add an Xref header field.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups header field
    body is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using
    the tradspool storage method. The newsgroup of an article is stored in
    the Newsgroups header field and in the Xref header field of the article
    as stored on disk (and possibly in Followup-To), as well as determining
    where the overview information is stored, and in the case of tradspool
    is also encoded in the article's storage token. To rename a newsgroup
    in tradspool, stop the server, move the directory containing all of
    the articles to its appropriate new location in the news spool, edit
    every article to change the old name to the new name in Newsgroups, Followup-To, and Xref, create the new newsgroup with ctlinnd newgroup,
    and then rebuild history and overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups header fields didn't break those links, you
    may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To header fields of the articles in that newsgroup and then rebuild history
    and overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To header
    fields to rename the newsgroup, and then create a file containing paths to
    all of the articles, one per line. You can then use that file as input to innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message identifier generated by INN will use the fully qualified domain name of the local system for the right-hand side of
    message IDs. In some cases, this isn't desirable for various reasons
    (the server may have an internal name that doesn't make sense on Usenet
    at large, or one may not want to expose the name of the server).

    INN still does not have a dedicated parameter to change the right-hand
    side of message identifiers. While waiting for it, there's a trick
    involving a special use of the domain: parameter (normally meant to be
    use only for what is related to DNS).

    In INN 2.7.1 and later, you can use an explicit domain: key in an access
    stanza of readers.conf to use its value as the right-hand side of message identifiers. (Even though domain: is set in inn.conf, note that this
    parameter also needs being present in readers.conf so as to trigger the expected behaviour).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will
    use that domain to generate message IDs. So if you need to change the
    domain used to generate message IDs for every local post from your
    server, just add virtualhost: and domain: keys to every access stanza
    in readers.conf. (Yes, this is really overkill for this option.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------


    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From KP2 KP2@21:1/5 to Russ Allbery on Mon Nov 6 18:01:38 2023
    [continued from previous message]

    3. At this point, you should be looking at a line with "+" in the second field. The article should be on your server at this point.

    If it's not, either it's been cancelled, or has already expired.

    4. You're now interested in whether the article was sent to your peers.
    At the end of the same line in $pathlog/news, innd puts all of the
    peers it thinks should receive this article.

    If you don't see a peer you expect there, it indicates that $pathetc/newsfeeds is not configured in the way you think it is.

    5. If a peer is listed at the end of the line, the article should have
    been fed to that peer.

    If a peer doesn't have that article, it's possible that the article is spooled on your system somewhere. Check $pathoutgoing, or the
    innfeed spool if the peer is configured to use innfeed. (It's probably easier to look for error messages in $pathlog/news.notice than to
    actually wade around in $pathspool/innfeed.)

    6. If you're sure the article isn't spooled, and it doesn't show up on the peer, you have to consider the possibility that the peer has rejected
    the article. Alternatively, it's possible that the peer has some misconfiguration like the ones described above.

    In either case, if you're sure that the article was offered to the peer
    and not spooled, you will need the assistance of the peer's admin to investigate further. INN does not generally log enough information
    about outgoing articles to be able to tell more from your server alone.

    It may be possible to get a slight bit of information from the remote
    server by connecting with telnet (usually to port 119) and issuing
    "IHAVE <message-id>". The peer may respond with something like "435 Duplicate" which means that the problem is not likely to be with your
    server (it may be still a problem with the article itself). If the
    peer responds with something like "335", your server probably did not
    offer the article after all.

    If you really are at a dead end and need to get more information about what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the daily reports, mail messages to moderators, gateway news to mail, send statistics to the TOP1000 project, mail errors to the news administrator, etc. and it assumes that you have a program installed as /usr/sbin/sendmail or /usr/lib/sendmail that it can use to do this. It does not speak SMTP,
    nor is it likely to ever speak SMTP; it's hard enough maintaining a package to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where applicable.

    INN logs nearly all messages to syslog, so in general these error messages will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very carefully if you're using the expires key, since it may not do what you think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without an Expires header field (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or it's missing some required entries. Even if you're running a small stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN will refile them automatically based on the Control header field.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can solve this problem by rebuilding the overview for any affected group. Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't allow binary postings to regular Usenet newsgroups that you don't know should have binaries, as they consume large amounts of bandwidth and possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, it's worth noting that with a shared Perl library,
    Perl on this platform doesn't apparently generate the right link magic
    to include the path to the dynamic Perl libraries. You need to either
    set LD_RUN_PATH before building or LD_LIBRARY_PATH before running any binaries so that they can find the Perl libraries. (The former is
    preferred, since then the path is encoded into the binaries and you
    don't have to remember to set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control
    pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control
    pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter
    pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on 2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN, but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    Note that rejected articles are remembered during the number of days specified by the /remember/ line in expire.ctl; so, in case you forgot
    to change the above parameters, you'll have to wait that number of
    days before being able to inject them again. Another possibility is to
    set /remember/ to 0, run the expire process (for instance via news.daily called with the same parameters as in crontab, plus "notdaily"),
    undo the change in expire.ctl and then start the feed again.

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of newsgroups as the old server.

    You may also want the new server not to propagate the articles it will receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles. The result file contains tokens ordered by arrival time on the old server (which is
    usually roughly the same as the posting time). In case the history
    file was not populated chronologically, it is better to sort it by
    posting time so that articles are fed in the right order. This can be achieved with the following command:

    sort -t '~' -k3n < history > history.sorted

    And then, consider history.sorted instead of history for the next steps.

    If the new server has just been installed or is known not to already have the articles you will feed it, you may want to add the "-c" flag to
    innxmit so as to skip the check for the presence of every article before transferring them.

    In case you wish to only feed articles arrived on the old server between
    two dates, you can adapt the previous commands with a condition on the $arrived variable (or the $posted variable if you prefer to use the Date header field instead of the actual arrival time). For instance, the following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    Other conditions may be added in the print line to select a subset of articles to feed. For instance if you want only articles for the fr.* hierarchy, you may add the following condition which will retrieve and
    parse the contents of the Xref field of every article to find the information (note that it will take some time to run, depending on the number of articles to parse):

    and qx/sm -q -H "$_" | grep Xref/ =~ / fr\.\S+:\d+/

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it. The "-d" flag
    is useful to add if you want to see the feeding progress.

    When done, set xrefslave to false in inn.conf again if you changed it and then either restart INN on the new server (necessary if you changed xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref header fields. Current versions of INN will always add an Xref header field, but very old versions (earlier 1.x versions) will only add an Xref header field to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles to add an Xref header field.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it, and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias. Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups header field body is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using
    the tradspool storage method. The newsgroup of an article is stored in
    the Newsgroups header field and in the Xref header field of the article
    as stored on disk (and possibly in Followup-To), as well as determining where the overview information is stored, and in the case of tradspool
    is also encoded in the article's storage token. To rename a newsgroup
    in tradspool, stop the server, move the directory containing all of
    the articles to its appropriate new location in the news spool, edit
    every article to change the old name to the new name in Newsgroups, Followup-To, and Xref, create the new newsgroup with ctlinnd newgroup,
    and then rebuild history and overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group are crossposted. If that's a problem, you may have to use a more sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups header fields didn't break those links, you may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with timehash you may be able to just change the Newsgroups, Xref, Followup-To header fields of the articles in that newsgroup and then rebuild history
    and overview as above.

    One other approach that can be used regardless of storage method is to refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away, changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To header fields to rename the newsgroup, and then create a file containing paths to all of the articles, one per line. You can then use that file as input to innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time, the message IDs will probably have all been added to history on the new server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message identifier generated by INN will use the fully qualified domain name of the local system for the right-hand side of
    message IDs. In some cases, this isn't desirable for various reasons
    (the server may have an internal name that doesn't make sense on Usenet
    at large, or one may not want to expose the name of the server).

    INN still does not have a dedicated parameter to change the right-hand
    side of message identifiers. While waiting for it, there's a trick
    involving a special use of the domain: parameter (normally meant to be
    use only for what is related to DNS).

    In INN 2.7.1 and later, you can use an explicit domain: key in an access stanza of readers.conf to use its value as the right-hand side of message identifiers. (Even though domain: is set in inn.conf, note that this parameter also needs being present in readers.conf so as to trigger the expected behaviour).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all posts coming from connections to which that access stanza applies will
    use that domain to generate message IDs. So if you need to change the
    domain used to generate message IDs for every local post from your
    server, just add virtualhost: and domain: keys to every access stanza
    in readers.conf. (Yes, this is really overkill for this option.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news

    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Tue Nov 21 08:01:03 2023
    [continued from previous message]

    misconfiguration like the ones described above.

    In either case, if you're sure that the article was offered to the peer
    and not spooled, you will need the assistance of the peer's admin to
    investigate further. INN does not generally log enough information
    about outgoing articles to be able to tell more from your server alone.

    It may be possible to get a slight bit of information from the remote
    server by connecting with telnet (usually to port 119) and issuing
    "IHAVE <message-id>". The peer may respond with something like "435
    Duplicate" which means that the problem is not likely to be with your
    server (it may be still a problem with the article itself). If the
    peer responds with something like "335", your server probably did not
    offer the article after all.

    If you really are at a dead end and need to get more information about
    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports, mail messages to moderators, gateway news to mail, send statistics to the TOP1000 project, mail errors to the news administrator,
    etc. and it assumes that you have a program installed as /usr/sbin/sendmail
    or /usr/lib/sendmail that it can use to do this. It does not speak SMTP,
    nor is it likely to ever speak SMTP; it's hard enough maintaining a package
    to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header field (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header field.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, it's worth noting that with a shared Perl library,
    Perl on this platform doesn't apparently generate the right link magic
    to include the path to the dynamic Perl libraries. You need to either
    set LD_RUN_PATH before building or LD_LIBRARY_PATH before running any
    binaries so that they can find the Perl libraries. (The former is
    preferred, since then the path is encoded into the binaries and you
    don't have to remember to set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    Note that rejected articles are remembered during the number of days
    specified by the /remember/ line in expire.ctl; so, in case you forgot
    to change the above parameters, you'll have to wait that number of
    days before being able to inject them again. Another possibility is to
    set /remember/ to 0, run the expire process (for instance via news.daily
    called with the same parameters as in crontab, plus "notdaily"),
    undo the change in expire.ctl and then start the feed again.

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles. The result file
    contains tokens ordered by arrival time on the old server (which is
    usually roughly the same as the posting time). In case the history
    file was not populated chronologically, it is better to sort it by
    posting time so that articles are fed in the right order. This can be
    achieved with the following command:

    sort -t '~' -k3n < history > history.sorted

    And then, consider history.sorted instead of history for the next steps.

    If the new server has just been installed or is known not to already have
    the articles you will feed it, you may want to add the "-c" flag to
    innxmit so as to skip the check for the presence of every article before transferring them.

    In case you wish to only feed articles arrived on the old server between
    two dates, you can adapt the previous commands with a condition on the
    $arrived variable (or the $posted variable if you prefer to use the Date
    header field instead of the actual arrival time). For instance, the
    following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    Other conditions may be added in the print line to select a subset of
    articles to feed. For instance if you want only articles for the fr.* hierarchy, you may add the following condition which will retrieve and
    parse the contents of the Xref field of every article to find the
    information (note that it will take some time to run, depending on the
    number of articles to parse):

    and qx/sm -q -H "$_" | grep Xref/ =~ / fr\.\S+:\d+/

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it. The "-d" flag
    is useful to add if you want to see the feeding progress.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref header fields. Current versions of INN will always add an Xref header field, but very old versions (earlier 1.x versions) will only add an Xref header field to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles to add an Xref header field.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups header field
    body is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using
    the tradspool storage method. The newsgroup of an article is stored in
    the Newsgroups header field and in the Xref header field of the article
    as stored on disk (and possibly in Followup-To), as well as determining
    where the overview information is stored, and in the case of tradspool
    is also encoded in the article's storage token. To rename a newsgroup
    in tradspool, stop the server, move the directory containing all of
    the articles to its appropriate new location in the news spool, edit
    every article to change the old name to the new name in Newsgroups, Followup-To, and Xref, create the new newsgroup with ctlinnd newgroup,
    and then rebuild history and overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups header fields didn't break those links, you
    may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To header fields of the articles in that newsgroup and then rebuild history
    and overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To header
    fields to rename the newsgroup, and then create a file containing paths to
    all of the articles, one per line. You can then use that file as input to innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message identifier generated by INN will use the fully qualified domain name of the local system for the right-hand side of
    message IDs. In some cases, this isn't desirable for various reasons
    (the server may have an internal name that doesn't make sense on Usenet
    at large, or one may not want to expose the name of the server).

    INN still does not have a dedicated parameter to change the right-hand
    side of message identifiers. While waiting for it, there's a trick
    involving a special use of the domain: parameter (normally meant to be
    use only for what is related to DNS).

    In INN 2.7.1 and later, you can use an explicit domain: key in an access
    stanza of readers.conf to use its value as the right-hand side of message identifiers. (Even though domain: is set in inn.conf, note that this
    parameter also needs being present in readers.conf so as to trigger the expected behaviour).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will
    use that domain to generate message IDs. So if you need to change the
    domain used to generate message IDs for every local post from your
    server, just add virtualhost: and domain: keys to every access stanza
    in readers.conf. (Yes, this is really overkill for this option.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------


    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Thu Dec 21 08:01:03 2023
    [continued from previous message]

    misconfiguration like the ones described above.

    In either case, if you're sure that the article was offered to the peer
    and not spooled, you will need the assistance of the peer's admin to
    investigate further. INN does not generally log enough information
    about outgoing articles to be able to tell more from your server alone.

    It may be possible to get a slight bit of information from the remote
    server by connecting with telnet (usually to port 119) and issuing
    "IHAVE <message-id>". The peer may respond with something like "435
    Duplicate" which means that the problem is not likely to be with your
    server (it may be still a problem with the article itself). If the
    peer responds with something like "335", your server probably did not
    offer the article after all.

    If you really are at a dead end and need to get more information about
    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports, mail messages to moderators, gateway news to mail, send statistics to the TOP1000 project, mail errors to the news administrator,
    etc. and it assumes that you have a program installed as /usr/sbin/sendmail
    or /usr/lib/sendmail that it can use to do this. It does not speak SMTP,
    nor is it likely to ever speak SMTP; it's hard enough maintaining a package
    to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header field (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header field.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, it's worth noting that with a shared Perl library,
    Perl on this platform doesn't apparently generate the right link magic
    to include the path to the dynamic Perl libraries. You need to either
    set LD_RUN_PATH before building or LD_LIBRARY_PATH before running any
    binaries so that they can find the Perl libraries. (The former is
    preferred, since then the path is encoded into the binaries and you
    don't have to remember to set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    Note that rejected articles are remembered during the number of days
    specified by the /remember/ line in expire.ctl; so, in case you forgot
    to change the above parameters, you'll have to wait that number of
    days before being able to inject them again. Another possibility is to
    set /remember/ to 0, run the expire process (for instance via news.daily
    called with the same parameters as in crontab, plus "notdaily"),
    undo the change in expire.ctl and then start the feed again.

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles. The result file
    contains tokens ordered by arrival time on the old server (which is
    usually roughly the same as the posting time). In case the history
    file was not populated chronologically, it is better to sort it by
    posting time so that articles are fed in the right order. This can be
    achieved with the following command:

    sort -t '~' -k3n < history > history.sorted

    And then, consider history.sorted instead of history for the next steps.

    If the new server has just been installed or is known not to already have
    the articles you will feed it, you may want to add the "-c" flag to
    innxmit so as to skip the check for the presence of every article before transferring them.

    In case you wish to only feed articles arrived on the old server between
    two dates, you can adapt the previous commands with a condition on the
    $arrived variable (or the $posted variable if you prefer to use the Date
    header field instead of the actual arrival time). For instance, the
    following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    Other conditions may be added in the print line to select a subset of
    articles to feed. For instance if you want only articles for the fr.* hierarchy, you may add the following condition which will retrieve and
    parse the contents of the Xref field of every article to find the
    information (note that it will take some time to run, depending on the
    number of articles to parse):

    and qx/sm -q -H "$_" | grep Xref/ =~ / fr\.\S+:\d+/

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it. The "-d" flag
    is useful to add if you want to see the feeding progress.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref header fields. Current versions of INN will always add an Xref header field, but very old versions (earlier 1.x versions) will only add an Xref header field to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles to add an Xref header field.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups header field
    body is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using
    the tradspool storage method. The newsgroup of an article is stored in
    the Newsgroups header field and in the Xref header field of the article
    as stored on disk (and possibly in Followup-To), as well as determining
    where the overview information is stored, and in the case of tradspool
    is also encoded in the article's storage token. To rename a newsgroup
    in tradspool, stop the server, move the directory containing all of
    the articles to its appropriate new location in the news spool, edit
    every article to change the old name to the new name in Newsgroups, Followup-To, and Xref, create the new newsgroup with ctlinnd newgroup,
    and then rebuild history and overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups header fields didn't break those links, you
    may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To header fields of the articles in that newsgroup and then rebuild history
    and overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To header
    fields to rename the newsgroup, and then create a file containing paths to
    all of the articles, one per line. You can then use that file as input to innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message identifier generated by INN will use the fully qualified domain name of the local system for the right-hand side of
    message IDs. In some cases, this isn't desirable for various reasons
    (the server may have an internal name that doesn't make sense on Usenet
    at large, or one may not want to expose the name of the server).

    INN still does not have a dedicated parameter to change the right-hand
    side of message identifiers. While waiting for it, there's a trick
    involving a special use of the domain: parameter (normally meant to be
    use only for what is related to DNS).

    In INN 2.7.1 and later, you can use an explicit domain: key in an access
    stanza of readers.conf to use its value as the right-hand side of message identifiers. (Even though domain: is set in inn.conf, note that this
    parameter also needs being present in readers.conf so as to trigger the expected behaviour).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will
    use that domain to generate message IDs. So if you need to change the
    domain used to generate message IDs for every local post from your
    server, just add virtualhost: and domain: keys to every access stanza
    in readers.conf. (Yes, this is really overkill for this option.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    with some additional software, it is also possible to use INN as, in
    essence, a local cache for a news server that you can use to read and post
    but which doesn't treat your server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    The software to pull down articles from another server and to feed
    articles to another server using post rather than peer-to-peer commands
    does not come with INN (INN has a few utilities to do this on a small
    scale, but not really anything designed to handle a lot of groups or a lot
    of articles). You will need an external package to do this. The two most popular are suck and newsx; however, both sites appear to be unavailable
    as of thos writing. You may be able to find a package in your local distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with
    a smaller, lighter package such as LeafNode or NNTPcache.

    ------------------------------


    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to All on Sun Jan 21 08:01:02 2024
    [continued from previous message]

    misconfiguration like the ones described above.

    In either case, if you're sure that the article was offered to the peer
    and not spooled, you will need the assistance of the peer's admin to
    investigate further. INN does not generally log enough information
    about outgoing articles to be able to tell more from your server alone.

    It may be possible to get a slight bit of information from the remote
    server by connecting with telnet (usually to port 119) and issuing
    "IHAVE <message-id>". The peer may respond with something like "435
    Duplicate" which means that the problem is not likely to be with your
    server (it may be still a problem with the article itself). If the
    peer responds with something like "335", your server probably did not
    offer the article after all.

    If you really are at a dead end and need to get more information about
    what's going on with an outgoing feed, you can switch it from innfeed
    to nntpsend (see INSTALL for instructions). You can then run it
    manually with innxmit -dv, which will show the full conversation with
    your remote peer.

    ------------------------------

    Subject: 3.8. sendmail isn't installed

    Yes, INN really does require sendmail. It uses sendmail to send out the
    daily reports, mail messages to moderators, gateway news to mail, send statistics to the TOP1000 project, mail errors to the news administrator,
    etc. and it assumes that you have a program installed as /usr/sbin/sendmail
    or /usr/lib/sendmail that it can use to do this. It does not speak SMTP,
    nor is it likely to ever speak SMTP; it's hard enough maintaining a package
    to speak NNTP.

    If you need a very simple local sendmail implementation that just sends
    mail to a smarthost, there are several available (nullmailer, for
    example).

    ------------------------------

    Subject: 4. Error Messages

    Explanations of specific error messages, including solutions where
    applicable.

    INN logs nearly all messages to syslog, so in general these error messages
    will be found in syslog. If you aren't seeing anything from INN in syslog
    at all, make sure that you have it set up correctly (see 3.3).

    ------------------------------

    Subject: 4.1. innd: SERVER cant store article

    You probably have a misconfigured storage.conf. In current versions of
    INN, "no matching entry in storage.conf" is added to the end of this
    message unless it really is a disk I/O problem, making the cause
    considerably clearer.

    storage.conf(5) has this to say:

    If an article doesn't match any entry, either by being posted to a
    newsgroup that doesn't match any of the <wildmat> patterns or by being
    outside the size and expires ranges of all entries whose newsgroups
    pattern it does match, the article is not stored and is rejected by
    innd(8). When this happens, the error message

    cant store article: no matching entry in storage.conf

    is logged to syslog. If you want to silently drop articles matching
    certain newsgroup patterns or size or expires ranges, assign them to the
    "trash" storage method rather than having them not match any storage
    method entry.

    One of the more frequent causes of this problem is misuse of the expires
    key in storage.conf entries. Read the man page for storage.conf very
    carefully if you're using the expires key, since it may not do what you
    think it does. In particular, if you have a storage class that specifies expires with a min-time greater than 0, it won't match any article without
    an Expires header field (the vast majority of Usenet articles).

    ------------------------------

    Subject: 4.2. innd: SERVER internal no control and/or junk group

    Your active file isn't complete. Either it's been mangled by something or
    it's missing some required entries. Even if you're running a small
    stand-alone server for internal use that only carries a handful of groups, there are some pseudogroups used internally by INN that you have to have.

    Since INN isn't running (it won't start when this error occurs), you can
    edit the active file by hand without worrying about stepping on INN's
    toes. Make sure the following lines are present in the active file (if
    the numbers are different, that's fine):

    control 0000000000 0000000000 n
    control.cancel 0000000000 0000000000 n
    control.checkgroups 0000000000 0000000000 n
    control.newgroup 0000000000 0000000000 n
    control.rmgroup 0000000000 0000000000 n
    junk 0000000000 0000000000 n

    and then start INN again. The control* groups are for control messages (messages with a named group will be filed into it, and all other control messages will go into the top-level catch-all group). The n flag is so
    that users won't post messages directly to the control* groups; control messages should be posted to the groups that they affect instead and INN
    will refile them automatically based on the Control header field.

    If you have mergetogroups: set in inn.conf, you will also need to create
    a newsgroup named "to". Otherwise, you will get the following error:

    innd: SERVER internal no to group

    ------------------------------

    Subject: 4.3. Modification of read-only value attempted (Cleanfeed)

    INN 2.3 and later have an internal optimization to the interface to
    embedded filters that makes filtering about 15-20% faster, but which
    disallows a trick that many versions of Cleanfeed use to count the number
    of lines in the article. (This problem is fixed in current versions of Cleanfeed.)

    To correct this problem, find the line in Cleanfeed that looks like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

    and change it to:

    $lines = $hdr{'__LINES__'};

    The __LINES__ hash value is set internally by all recent versions of INN
    and is guaranteed to be correct.

    ------------------------------

    Subject: 4.4. tradspool: could not open ... File exists

    This error generally happens after a crash or unclean shutdown of innd
    using the tradspool storage method, and is caused by overview information
    being out of sync with what articles are in the spool. When innd was restarted, it renumbered its active file (which determines the range of existing articles in each group and therefore what article number is
    assigned to new articles) based on the overview information. If there are newer articles already on disk that aren't mentioned in the overview
    (because the overview information for those articles hasn't been flushed
    to disk yet), new incoming articles will get assigned the same number as
    the existing article and then innd will fail to store the article and
    throttle with this error.

    In INN 2.4 and later when using the tradindexed overview method, you can
    solve this problem by rebuilding the overview for any affected group.
    Throttle the server (if it isn't already) and then run:

    tdx-util -R <path-to-articles> -n <newsgroup>

    where <newsgroup> is the newsgroup that INN is complaining about and <path-to-particles> is the full path to the directory where the articles
    for that group are stored (it's generally in the error message).
    Immediately afterwards, run ctlinnd renumber for that newsgroup, and then unthrottle the server.

    The general solution to this problem, which works with any version of INN
    and any overview method, is to shut down the server, delete all of your overview database, and then rebuild it from your news spool with:

    makehistory -O -x -F

    This takes a long time and is to some degree overkill. For versions of
    INN prior to 2.5, you will also need to run ctlinnd renumber '' immediately after restarting INN.

    A third and better solution in some cases is to just remove all articles
    in the spool that have higher numbers than the numbers in the active file. Here's a Perl script that will do that. Just save this to a file, make it executable, and run it, giving it the path to the active file as the first argument and the path to the top of your tradspool news spool as the
    second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
    my ($group, $hi, $lo, $flag) = split;
    my $directory = $group;
    next if ($hi == 0 and $lo <= 1);
    $directory =~ tr%.%/%;
    $directory = $ARGV[1] . '/' . $directory;
    if (-d $directory) {
    opendir (DIR, $directory) or die "Can't open $directory: $!\n";
    while (defined ($_ = readdir DIR)) {
    unlink "$directory/$_" if ($_ > $hi);
    }
    closedir DIR;
    }
    }

    Another alternative to find issues in your tradspool news spool is to run
    the scanspool utility. It will notably report the above too high article numbers.

    If you're not already running INN 2.4, upgrade if you can. Not only can
    you recover directly from this problem if you're using tradindexed
    overview, but INN 2.4 does a better job of flushing data to disk and is
    less likely to have this problem in the first place.

    ------------------------------

    Subject: 4.5. Binary posting to non-binary group

    This message does not actually come from INN. It's generated by
    Cleanfeed, and if you're seeing it, that means that you have Cleanfeed installed. At least at one point, the default Red Hat installation of INN included Cleanfeed without documenting this particularly well.

    In order to allow binaries in your local hierarchies, you should modify
    the Cleanfeed configuration file to set bin_allowed to a regular
    expression matching the groups that should allow binaries. Please don't
    allow binary postings to regular Usenet newsgroups that you don't know
    should have binaries, as they consume large amounts of bandwidth and
    possibly disk space for other sites.

    For more information on Cleanfeed configuration options, see the Cleanfeed documentation and the comments in the default configuration file.

    ------------------------------

    Subject: 5. Problems on Specific Systems

    Problems specific to particular operating systems or platforms. Look here
    if INN doens't behave as expected on your particular system, or if you're having trouble compiling INN in the first place.

    ------------------------------

    Subject: 5.1. INN won't compile on SCO OpenServer

    On SCO OpenServer, it's worth noting that with a shared Perl library,
    Perl on this platform doesn't apparently generate the right link magic
    to include the path to the dynamic Perl libraries. You need to either
    set LD_RUN_PATH before building or LD_LIBRARY_PATH before running any
    binaries so that they can find the Perl libraries. (The former is
    preferred, since then the path is encoded into the binaries and you
    don't have to remember to set LD_LIBRARY_PATH later.)

    ------------------------------

    Subject: 5.2. Using raw devices on Solaris destroys the partition table

    If you use slice 2, or some other disk slice that includes the entire
    disk, under Solaris as a raw partition for CNFS, you may run into this
    problem. The symptoms are that INN manages to initialize the cycbuffs
    just fine, but then gets invalid device errors when it tries to open them again, and the disks show up in format as needing to be repartitioned.

    The solution is to not use raw devices that include the first cylinder of
    the disk. Solaris doesn't protect the superblock from being overwritten
    by an application writing to raw devices and includes it in the first
    cylinder of the disk, so unless you use a slice that starts with cylinder
    1 instead of 0, INN will invalidate the partition table when it tries to initialize the cycbuff and all further accesses will fail until you repartition.

    Generally all that has to be done is to repartition the disk with slice 0 starting from cylinder 1 and extending to the end of the disk and then
    point INN at slice 0 instead of slice 2. You lose some small amount of
    space, but generally not enough to care about.

    ------------------------------

    Subject: 5.3. Will INN work on Windows?

    It won't out of the box. The standard INN distribution doesn't build on Windows. It has, however, been built for Cygwin (a Unix-like environment
    for Windows) in the past and some of the necessary patches (although
    perhaps not all of them) have been incorporated into current INN releases.

    Search for http://homepage.mac.com/imeowbot/inn/ at
    <http://web.archive.org/> for the previous work. Don't forget to peruse INSTALL if you download and want to try this.

    ------------------------------

    Subject: 5.4. Why aren't INN's files where the documentation says they are?

    INN's default installation locations are intended to be convenient for sysadmins adding INN to their system without disturbing other software.
    They don't match any of the standards used by various Linux distributions
    or other Unix packaging systems. Because of that, distributors who supply
    INN packages often rearrange the files and directories.

    Unfortunately, this is very confusing for system administrators, because
    the documentation is not updated to reflect the modified locations of
    files.

    You can always get the details of how your system is configured by looking
    in inn.conf at "pathnews" and similar parameters. But for convenience,
    here are comparisons of INN's default locations with some of the most
    common packages.

    (Data courtesy of John F. Morse.)

    DEFAULT DEBIAN
    pathnews: /usr/local/news /usr/lib/news
    pathbin: /usr/local/news/bin /usr/lib/news/bin
    pathcontrol: /usr/local/news/bin/control /usr/lib/news/bin/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /etc/news/filter
    pathhttp: /usr/local/news/http /var/www/inn
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/spool/news/incoming/tmp pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    DEFAULT FEDORA
    pathnews: /usr/local/news /usr/libexec/news
    pathbin: /usr/local/news/bin /usr/libexec/news
    pathcontrol: /usr/local/news/bin/control /usr/libexec/news/control pathdb: /usr/local/news/db /var/lib/news
    pathetc: /usr/local/news/etc /etc/news
    pathfilter: /usr/local/news/bin/filter /usr/libexec/news/filter pathhttp: /usr/local/news/http /var/lib/news/http
    pathlog: /usr/local/news/log /var/log/news
    pathrun: /usr/local/news/run /run/news
    pathtmp: /usr/local/news/tmp /var/lib/news/tmp
    pathspool: /usr/local/news/spool /var/spool/news
    patharchive: /usr/local/news/spool/archive /var/spool/news/archive patharticles: /usr/local/news/spool/articles /var/spool/news/articles pathincoming: /usr/local/news/spool/incoming /var/spool/news/incoming pathoutgoing: /usr/local/news/spool/outgoing /var/spool/news/outgoing pathoverview: /usr/local/news/spool/overview /var/spool/news/overview

    In addition, the FreeBSD port uses the standard INN paths except that it
    puts logs in /var/log/news and pathtmp in /usr/local/news/spool/tmp.

    Most packages install INN's man pages into a system man directory (/usr/share/man or /usr/local/man) rather than into a separate man
    directory under news's home directory.

    ------------------------------

    Subject: 5.5. Running INN on macOS

    Richard Tobin provided the following advice in news.software.nntp on
    2013-06-29 based on experience with running INN on Snow Leopard:

    Mac OS X, at least through the GUI, won't let you create a group with
    the same name as a user. So you can't use "news" for both.

    The Perl module GD isn't installed by default. GPG is not installed
    by default.

    You probably want to turn off Spotlight for the news spool directory.

    Configure didn't get the Perl compile flags right. PERL_CPPFLAGS had
    "-arch x86_64 -arch i386 -arch ppc", but on this x86_64 machine the
    files for the other architectures don't seem to be installed. I
    edited Makefile.global by hand to remove them.

    I needed to tell the application firewall to allow innd to accept
    incoming connections. (A window pops up to ask you, but this doesn't
    help when you're connected by ssh!)

    When I ran rc.news form a terminal window, it stopped working when I
    logged out. This is because of MacOS's convoluted and undocumented
    way of doing DNS lookups. Using "nohup" fixed it -- not because of
    anything to do with SIGHUP, but because nohup calls an undocumented
    function related to "vprocmgr". Running from launchd shouldn't have
    this problem, and it appears to be fixed in Mountain Lion.

    The Perl flags come from the Perl configuration; this problem is fixed
    with current builds of macOS.

    ------------------------------

    Subject: 6. How Do I...

    This section documents various common or uncommon tasks or configurations
    that people want to do with INN. It is mostly taken from frequently asked questions in news.software.nntp.

    ------------------------------

    Subject: 6.1. Set up a server with no external feeds, just local groups

    The basic steps are to set up a newsfeeds file empty except for internal
    feeds like controlchan or overchan (if you're using either), have only localhost in incoming.conf, and start INN with the default minimal active
    file. Then, create the groups you want to carry with ctlinnd newgroup.
    Set up reading permissions using readers.conf as appropriate for your organization.

    In other words, it's very much like setting up any other instance of INN,
    but you don't bother with innfeed, nntpsend, or any of their configuration files. INN may also complain that you have no feeds in newsfeeds; this is harmless and can be ignored.

    ------------------------------

    Subject: 6.2. Process a single control message

    To process a single control message, you can use controlchan from the
    command line. Just type either:

    echo /path/to/article-file | controlchan

    or:

    echo @token@ | controlchan

    if you have the storage API token of the article. (This assumes
    controlchan is in a directory in your path.) This is useful mostly for testing; if you just want to create, remove, or change a group, it's
    easier to use ctlinnd (newgroup, rmgroup, or changegroup).

    ------------------------------

    Subject: 6.4. Feed all articles on a server to another server

    To feed all articles on an existing server to another one, regardless of
    how they're stored on the server, first tell the new server to accept
    articles regardless of how old they are (otherwise, INN will reject
    articles older than artcutoff in inn.conf) and disable your filtering:

    ctlinnd param c 0
    ctlinnd perl n
    ctlinnd python n

    Note that rejected articles are remembered during the number of days
    specified by the /remember/ line in expire.ctl; so, in case you forgot
    to change the above parameters, you'll have to wait that number of
    days before being able to inject them again. Another possibility is to
    set /remember/ to 0, run the expire process (for instance via news.daily
    called with the same parameters as in crontab, plus "notdaily"),
    undo the change in expire.ctl and then start the feed again.

    You may also want to set xrefslave to true in inn.conf and then restart
    INN on the new server if you want to keep the same article numbers as you
    had on the old server. (It is notably helpful for news clients because
    they otherwise get confused by an article renumbering in newsgroups they
    are subscribed to.)

    Next, make sure that the old server is listed in incoming.conf of the new server, and reload incoming.conf with ctlinnd to pick up that change.
    Also make sure that the new server carries exactly the same set of
    newsgroups as the old server.

    You may also want the new server not to propagate the articles it will
    receive during this feeding operation, by checking that the newsfeeds
    file of the new server is not configured to propagate articles to other
    peers or controlchan (otherwise old control articles may be reprocessed).

    Then try these commands (a variation on commands posted by Katsuhiro
    Kondou to inn-workers) on the old server:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    print "$_\n" if $_' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    where <pathdb> is the path to the directory containing the history file (usually ~news/db), <pathoutgoing> is the path to the outgoing spool
    directory (usually ~news/spool/outgoing), and server is the name of the
    new news server to which you're feeding the articles. The result file
    contains tokens ordered by arrival time on the old server (which is
    usually roughly the same as the posting time). In case the history
    file was not populated chronologically, it is better to sort it by
    posting time so that articles are fed in the right order. This can be
    achieved with the following command:

    sort -t '~' -k3n < history > history.sorted

    And then, consider history.sorted instead of history for the next steps.

    If the new server has just been installed or is known not to already have
    the articles you will feed it, you may want to add the "-c" flag to
    innxmit so as to skip the check for the presence of every article before transferring them.

    In case you wish to only feed articles arrived on the old server between
    two dates, you can adapt the previous commands with a condition on the
    $arrived variable (or the $posted variable if you prefer to use the Date
    header field instead of the actual arrival time). For instance, the
    following commands will feed articles arrived between two given
    timestamps (that can be computed with the convdate utility shipped
    with INN).

    convdate -n '15 Apr 2014 20:42 +0200' '16 Apr 2014 12:37 +0200'

    returns the two corresponding timestamps 1397586540 and 1397644620 that
    can then be used to retrieve a subset of articles to feed:

    cd <pathdb in inn.conf>
    perl -ne 'chomp; our ($hash, $timestamps, $_) = split " "; \
    my ($arrived, $expires, $posted) = split("~", $timestamps); \
    print "$_\n" if $_ and $arrived >= 1397586540 \
    and $arrived <= 1397644620' history \
    | tr . / > <pathoutgoing in inn.conf>/list
    innxmit server list

    Other conditions may be added in the print line to select a subset of
    articles to feed. For instance if you want only articles for the fr.* hierarchy, you may add the following condition which will retrieve and
    parse the contents of the Xref field of every article to find the
    information (note that it will take some time to run, depending on the
    number of articles to parse):

    and qx/sm -q -H "$_" | grep Xref/ =~ / fr\.\S+:\d+/

    If innxmit stops transferring articles (with for instance an error like "rewriting batch file and exiting"), just re-execute it. The "-d" flag
    is useful to add if you want to see the feeding progress.

    When done, set xrefslave to false in inn.conf again if you changed it and
    then either restart INN on the new server (necessary if you changed
    xrefslave) or use another ctlinnd param command to set the cutoff value
    back to what's specified in inn.conf and use ctlinnd perl and ctlinnd
    python to reactivate your filters.

    Please note that when using xrefslave, this method requires that all of
    the articles in your spool have Xref header fields. Current versions of INN will always add an Xref header field, but very old versions (earlier 1.x versions) will only add an Xref header field to crossposted articles. If you're trying to import such a spool, you'll need to modify all of those articles to add an Xref header field.

    ------------------------------

    Subject: 6.5. Rename a newsgroup

    INN has no native support for renaming a newsgroup, and doing so is
    difficult, so the best advice is to not do this. If there's a way that
    you can just create the new newsgroup, encourage people to start using it,
    and then remove the old newsgroup, I recommend that. It's much easier.

    Although it is not a renaming, it is also possible to create an alias.
    Articles cannot be posted to that newsgroup, but they can be received
    from other sites and treated as if they were actually posted to the
    group named after the equal sign. However, their Newsgroups header field
    body is not modified.

    ctlinnd newgroup group.to.file.under y
    ctlinnd changegroup old.group =group.to.file.under

    Creating an alias newsgroup is useful in case you want residual articles received under the old newsgroup name to be filed into the new group.

    As for a renaming, if it really must be done, it's best if you're using
    the tradspool storage method. The newsgroup of an article is stored in
    the Newsgroups header field and in the Xref header field of the article
    as stored on disk (and possibly in Followup-To), as well as determining
    where the overview information is stored, and in the case of tradspool
    is also encoded in the article's storage token. To rename a newsgroup
    in tradspool, stop the server, move the directory containing all of
    the articles to its appropriate new location in the news spool, edit
    every article to change the old name to the new name in Newsgroups, Followup-To, and Xref, create the new newsgroup with ctlinnd newgroup,
    and then rebuild history and overview with makehistory.

    The following bit of Perl may help with the renaming (from Jeffrey
    Vinocur):

    #!/usr/bin/perl -wi
    my ($src, $dst) = (shift, shift);
    die "Usage: $0 oldgroup newgroup [file1 [file2 ...]]\n"
    unless(defined $dst);
    while(<>) {
    s/$src/$dst/g if 1 .. /^$/ and /^(Newsgroups|Followup-To|Xref):/i;
    print;
    } continue {
    close ARGV if eof;
    }

    Note that this may cause some problems if the newsgroup you're renaming is contained in the name of another newsgroup to which messages in that group
    are crossposted. If that's a problem, you may have to use a more
    sophisticated script.

    If any articles were crossposted to other newsgroups, you'll also have to
    find and recreate the links in those newsgroups to the new location of the articles (if the links were hard links and the process of changing the
    Xref, Followup-To, Newsgroups header fields didn't break those links, you
    may be lucky and be able to skip this).

    If you're using another storage method, this is harder, although with
    timehash you may be able to just change the Newsgroups, Xref, Followup-To header fields of the articles in that newsgroup and then rebuild history
    and overview as above.

    One other approach that can be used regardless of storage method is to
    refeed the articles to the server into a new newsgroup. This approach
    works best if you're also changing news servers at the same time;
    otherwise, the message IDs of the articles will already be in history, and you'll have to change the message IDs of all of the messages or remove
    them from the history database (such as by moving the articles away,
    changing /remember/ to 0 so that old history entries won't be retained,
    and then running expire to purge them out of history). To do this, get
    all of the messages into a directory (by pulling them down via NNTP or
    some other method), change the Newsgroups, Xref, and Followup-To header
    fields to rename the newsgroup, and then create a file containing paths to
    all of the articles, one per line. You can then use that file as input to innxmit, pointing it at the server to which to feed the articles, and if
    the articles aren't listed in history on that server and it carries the
    new group, they will be accepted into the new newsgroup.

    Note that if you use this method and something goes wrong the first time,
    the message IDs will probably have all been added to history on the new
    server and the articles now will never be accepted until those entries are removed from history again (or all the message IDs changed).

    ------------------------------

    Subject: 6.6. Change the domain used for message IDs

    By default, any message identifier generated by INN will use the fully qualified domain name of the local system for the right-hand side of
    message IDs. In some cases, this isn't desirable for various reasons
    (the server may have an internal name that doesn't make sense on Usenet
    at large, or one may not want to expose the name of the server).

    INN still does not have a dedicated parameter to change the right-hand
    side of message identifiers. While waiting for it, there's a trick
    involving a special use of the domain: parameter (normally meant to be
    use only for what is related to DNS).

    In INN 2.7.1 and later, you can use an explicit domain: key in an access
    stanza of readers.conf to use its value as the right-hand side of message identifiers. (Even though domain: is set in inn.conf, note that this
    parameter also needs being present in readers.conf so as to trigger the expected behaviour).

    In INN 2.3.3 and later, you can set virtualhost: to true in an access
    stanza of readers.conf and then set domain: in the same stanza, and all
    posts coming from connections to which that access stanza applies will
    use that domain to generate message IDs. So if you need to change the
    domain used to generate message IDs for every local post from your
    server, just add virtualhost: and domain: keys to every access stanza
    in readers.conf. (Yes, this is really overkill for this option.)

    ------------------------------

    Subject: 6.7. Use INN without a direct news feed

    INN is designed to be used as a regular news server, receiving direct news feeds from other news servers and sending news directly to other news
    servers using the peer-to-peer portions of the NNTP protocol. However,
    it is also possible to use INN as, in essence, a local cache for a news
    server that you can use to read and post but which doesn't treat your
    server like a peer.

    This configuration is generally called a "suck" feed, because rather than having news fed directly to your server, you pull it down or "suck" it
    from another news server, and because possibly the first and one of the
    most widely used packages for doing this is named suck.

    INN comes with a program named pullnews to pull down articles from another server and to feed articles to another server using either POST like a news reader or peer-to-peer commands. See the pullnews manual page for more information.

    Alternately, you can use an external package to do this. The two most
    popular are suck and newsx; however, they are no longer actively
    maintained. You may still be able to find a package in your local
    distribution or package repository.

    Note that current versions of INN refer to articles internally using a
    storage API token, not a path name, which is not always what suck or newsx expects. Read the documentation carefully; you'll need to use a script or configuration that retrieves articles using the sm program that comes with
    INN rather than trying to open files directly.

    It's also worth noting that INN is a fairly complex package, and while
    many people are running it successfully using this sort of configuration
    and like having a full-fledged news server available to them, other people
    have found INN rather complicated and difficult to configure for a small, simple personal news cache. If your needs and goals are simple and the
    number of groups you're interested in is small, you may be better off with

    [continued in next message]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)