Hi Jesse,
I performed manual manipulation of a spool by deleting lots of articles from tradspool directories and deleted a CNFS buffer containing binaries and junk).
I removed the history files and overview directory contents, and have been running makehistory since 10/12/2023 and it is *still* running...
It takes a bloody long time... I unfortunately do not have any advice
to make it run faster except for using the "-s" flag. I'll respond on
another message you sent about makehistory and dbz operations.
The previous overview directory was 67GB and the new one is at 107GB, which seems odd because I did not add articles
Maybe the first run of expireover (news.daily) will shrink it a bit, and
expire articles which should no longer be in the overview data
(cancelled or removed by a NoCeM notice).
While watching lsof output against the makehistory PID, I am finding that it is scanning the same tradspool folders more than once, but I am not sure if that is expected?
When dealing with tradspool, it should indeed treat directories in order (tradspool_next() method). Maybe the behaviour you see comes from
crossposted articles? makehistory opens the (hard-linked) file present
in each directory an article has been crossposted to, and looks at its
Xref header field to seek the first newsgroup mentioned (considered to
be the master, and only treats the article when dealing with that first newsgroup).
I also have a couple of "Bad article handle" messages that don't make sense:
makehistory: tradspool: can't determine class of @0500000087B5000000000001390000000000@: Bad article handle
I don't think the "Bad article handle" error is related to this article.
This error is set when SMgetsub() in storage/interface.c is called with
an article length of 0, whereas the "can't determine class of" error
occurs only with articles whose length is > 0 in
storage/tradspool/tradspool.c.
So I bet the "Bad article handle" error corresponds to a previous error,
and not the one for this article. Which implies that SMgetsub() did not
return NULL and therefore that the type of the token does not correspond
to tradspool.
I am very unsure why it would happen during a rebuild of the history
file. Looking at a comment, a modification of the storage.conf file is mentioned. Would it happen that the article stored in tradspool
according to the rules of your initial storage.conf file has now a
different rule?
[storage/tradspool/tradspool.c]
if ((sub = SMgetsub(*art)) == NULL || sub->type != TOKEN_TRADSPOOL) {
/* maybe storage.conf is modified, after receiving article */
token = MakeToken(priv.ngtp->ngname, artnum, 0);
if (art->len > 0)
warn("tradspool: can't determine class of %s: %s",
TokenToText(token), SMerrorstr);
}
FWIW, I think the code could be improved this way so that it does not
display an unappropriated error.
- if (art->len > 0)
+ if (art->len > 0 && sub == NULL)
warn("tradspool: can't determine class of %s: %s",
TokenToText(token), SMerrorstr);
It seems okay though?
$ sm -c '@0500000087B5000000000001390000000000@' @0500000087B5000000000001390000000000@ method=tradspool class=0 ngnum=34741 artnum=0 file=/usr/local/news/spool/articles/misc/health/alternative/80128
When the discussed error occurs, the token is computed on the fly with a
forced class of 0, and belonging to tradspool. Decoding it with "sm -c"
will naturally work (and give you the tradspool method, class 0).
The good news is that the token is properly computed, and integrated
into your history file.
The error of "can't determine class of" should not be displayed in your
case (if I am right in my understanding of what happened). It should be displayed only when there's a real error, that is to say when SMgetsub() returns NULL. Otherwise, I don't think it matters much.
--
Julien ÉLIE
« Dans toute statistique, l'inexactitude du nombre est compensée par la
précision des décimales. » (Alfred Sauvy)
--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)