• tradpoool v cnfs

    From Nigel Reed@21:1/5 to All on Sat Jan 27 22:25:52 2024
    The time has come for me to make a decision keep tradspool or move to
    cnfs. I know there's a lot of pros and cons with both.

    tradspool creates an issues when a newsgroup name contains number a
    numeric component that clashes with an article number. This came
    to light and I posted about it the other day.

    I'm using zfs so there are no inode issues.

    I also like the fact I can directly look at articles, grep for words
    and things like that.

    It's easy to see exactly how much disk space I have left and there's no
    issues with deleting articles.

    My system never expires articles except for test groups which are
    expired after 7 days.

    With cnfs there's no need to worry about those newsgroup name issues,
    but there's no easy direct method of viewing the articles.
    I also fear that I will lose articles if, for some how, the buffers get
    full and start to overwrite without me noticing it.

    Also, will space from expired or canceled articles be reused or does
    that usually get reclaimed when the buffer recycles? I could end up
    losing a lot of space.

    Also, if using cnfs, would would be the best way to create the buffers?
    I have a 1.9TB block of space that I can keep as a single partition or
    divvy up into multiple.

    After advise from those more in the knowledge of cnfs than me.

    Thanks,
    Nigel

    --
    End Of The Line BBS - Plano, TX
    telnet endofthelinebbs.com 23

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From noel@21:1/5 to Nigel Reed on Sun Jan 28 16:59:47 2024
    On Sat, 27 Jan 2024 22:25:52 -0600, Nigel Reed wrote:


    I also fear that I will lose articles if, for some how, the buffers get
    full and start to overwrite without me noticing it.

    This !

    apart from taking months to import news from DNews into INN, the other
    key concern stopping us is this one, I'm told CNFS buffers is about as
    close to DNews spool files (IIRC - has been a while since I discussed
    that with Julien) out of all the choices, and if we don't create enough,
    the over writing will be a problem, I think CNFS needs an option to
    create more buffer files automatically on-the-fly when existing buffer
    count hits a pre determined or user configurable, value of the total
    available space.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?Julien_=C3=89LIE?=@21:1/5 to All on Sun Jan 28 09:29:54 2024
    Hi Nigel,

    tradspool creates an issues when a newsgroup name contains number a
    numeric component that clashes with an article number. This came
    to light and I posted about it the other day.

    The issue appears only once per newsgroup with a numeric component and
    for which its direct parent exists as a newsgroup.
    That is to say if you have a newsgroup named a.b.3, and no newsgroup
    named a.b, there won't be any problem. It is only when you have both
    a.b and a.b.3 as newsgroups.

    It would be worthwhile checking how many newsgroups like that you have.
    It may not be a big deal.

    You'll then only lose an article per newsgroup when deleting it (in the example, article number 3 for a.b cannot be kept).


    Compared to the other pros/cons of CNFS, it's up to you to decide what
    you prefer.

    --
    Julien ÉLIE

    « Le cercle est le plus long chemin d'un point au même point. » (Tom
    Stoppard, _Every Good Boy Deserves Favour_)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ivo Gandolfo@21:1/5 to Nigel Reed on Sun Jan 28 10:42:57 2024
    On 28/01/2024 05:25, Nigel Reed wrote:

    With cnfs there's no need to worry about those newsgroup name issues,
    but there's no easy direct method of viewing the articles.

    You need to know the token, but it's easy with m-id. You can grab the
    tokenname with grephistory, and then read the article with sm. It's just
    a step + 1 to read, and not pointing directly the article into the
    filesystem.

    I also fear that I will lose articles if, for some how, the buffers get
    full and start to overwrite without me noticing it.


    Actually I'm setup 2 new newsserver, and grabbing news from various
    source with cnfs and buffindexer without big trouble. The only defect
    it's the harddisk space it's areally taken.

    Also, will space from expired or canceled articles be reused or does
    that usually get reclaimed when the buffer recycles? I could end up
    losing a lot of space.

    Also, if using cnfs, would would be the best way to create the buffers?
    I have a 1.9TB block of space that I can keep as a single partition or
    divvy up into multiple.

    After advise from those more in the knowledge of cnfs than me

    Thanks,
    Nigel


    See INSTALL file from inn2 tarball, section "Creating the Article Spool
    (CNFS only)" or mkbuf.in file in CONTRIB section. In my case I have do a
    little rewrite them for daily use, and it's the "easy way" to create a
    new one cyc buff and adding them to storage.conf file. I use a file into directory /var/spool/news/cycbuff (same filesystem ad /, but you can do
    a new partition for this with different filesystem).

    For warning when a cyc buff it's "full" I have created a script, added
    in crontab, running cnfsstat -a, when the value of cycles reach *.80
    send me an email and my system create in auto a new cyc buffer with
    predefined space, add it into storage.conf (using the pattern in Class
    readed with cnfsstat), and reload inn2. But in my case I have 24TB of diskspace, and create a new cyc it's not hard for me.
    In your case with only 2TB of space I recommend you create some small
    cycs for those groups (or hierarchies) that are not important to you and
    which can also expire quickly and suppose 50/500MB it's enought for
    control.* or *.test, and some "large" cycs for the groups (or
    hierarchies) that instead you want to maintain, in that case 1/10GB. If
    you set as sequential, not big trouble, and easy to reuse (or backup)
    and you can set a good retention for everything's.

    The stats of my newsserver (text only) it's a retention from '90/2000
    take 4/5TB of space. A lot of group have millions of post (one I have
    see have 20 millions, impossible to grep all with normal newsreader).


    Sincerely

    --
    Ivo Gandolfo

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)