Here is a summary of MAXARTNUM implemented in Spitfire News Server.


- If MAXARTNUM disabled for "anonymous" accessgroup, server will reply
with "480 authentification required".

- If MAXARTNUM disabled for another accessgroup, server reply with "502 MAXARTNUM command disabled by administrator".

- MAXARTNUM accept a number as argument or ^31 -> ^63. If MAXARTNUM
< ^31 or > ^63, server reply with "501 syntax error".

- When MAXARTNUM is not used and a newsgroup contains articles > ^31,
pointers are adjusted to fit ^31 :

- High = ^31
- Count = Estimated articles (High-Low)+1


- If a reader use NEXT to access numbers > ^31, server reply with "401
maximum article number reached".

- Articles > ^31 can be retrieved with their <Message-ID>. In this case, article number is set to "0" and ALL newsgroups in Xref header with a
value > 2^31 are "on-the-fly" set to 0. Size of article is also adjusted "on-the-fly".

- NEWNEWS display ALL <Message-ID> (< or > ^31).

- Once used, MAXARTNUM cannot be set again.



Please let me know if you find something wrong or if you have any
suggestions.

Franck

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Franck <franck@email.invalid> writes:

Here is a summary of MAXARTNUM implemented in Spitfire News Server.

- If MAXARTNUM disabled for "anonymous" accessgroup, server will reply
with "480 authentification required".

- If MAXARTNUM disabled for another accessgroup, server reply with
"502 MAXARTNUM command disabled by administrator".

- MAXARTNUM accept a number as argument or ^31 -> ^63. If MAXARTNUM <

^^^^^^^^^^^^^

It’s not clear what this means.

^31 or > ^63, server reply with "501 syntax error".

- When MAXARTNUM is not used and a newsgroup contains articles > ^31,
pointers are adjusted to fit ^31 :

- High = ^31
- Count = Estimated articles (High-Low)+1

If your notation ^31 really means 2^31 then your description is off by
one from RFC3977, where the maximum is 2^31-1, and article numbers >=
2^31-1 are forbidden.

- If a reader use NEXT to access numbers > ^31, server reply with "401
maximum article number reached".

- Articles > ^31 can be retrieved with their <Message-ID>. In this
case, article number is set to "0" and ALL newsgroups in Xref header
with a value > 2^31 are "on-the-fly" set to 0. Size of article is
also adjusted "on-the-fly".

- NEWNEWS display ALL <Message-ID> (< or > ^31).

- Once used, MAXARTNUM cannot be set again.

Please let me know if you find something wrong or if you have any suggestions.

--
http://www.greenend.org.uk/rjk/

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hello,

- MAXARTNUM accept a number as argument or ^31 -> ^63. If MAXARTNUM <

It’s not clear what this means.

C: HELP
...
S: MAXARTNUM number|^31-63
...

I hope this ABNF is correct, but perhalps it's not.

MAXARTNUM accept an article number or values from ^31 to ^63.

If your notation ^31 really means 2^31 then your description is off by
one from RFC3977, where the maximum is 2^31-1, and article numbers >=
2^31-1 are forbidden.

You can try by yourself, bu yes, I was a bit lasy to wrote 2^31-1 :-)

C: MAXARTNUM ^31
S: 202 2147483647 is our mutual maximum article number.

...

C: MAXARTNUM ^40
S: 202 1099511627775 is our mutual maximum article number.

...

MAXARTNUM ^63
202 9223372036854775807 is our mutual maximum article number.


Have nice day,
Franck

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hello,

After careful consideration, it is more reasonable to write:

- When command "MAXARTNUM" is not used and a newsgroup contains articles

MAXARTNUM,

pointers are adjusted to fit MAXARTNUM :

  - High  = MAXARTNUM
  - Count = Estimated articles (MAXARTNUM-Low)+1

- If a reader use NEXT to access numbers > MAXARTNUM, server reply with
"401
maximum article number reached" except if current article is the "real"
last one.

- Articles > MAXARTNUM can be retrieved with their <Message-ID>. In this
case,
article number is set to "0" and ALL newsgroups in Xref header with a
value > MAXARTNUM are "on-the-fly" set to 0.

Basically, if MAXARTNUM is set to ^40, then server will limit everything
to ^40.

If MAXARTNUM is set to ^32+45, then server will limit everything to ^32+45.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Franck <franck@email.invalid> writes:

- MAXARTNUM accept a number as argument or ^31 -> ^63. If MAXARTNUM <

It’s not clear what this means.

C: HELP
...
S: MAXARTNUM number|^31-63
...

I hope this ABNF is correct, but perhalps it's not.

MAXARTNUM accept an article number or values from ^31 to ^63.

So does your notation ^N mean 2^N-1 then? That’s really confusing.

I think your command should just use numbers, rather than your own
private notation.

--
http://www.greenend.org.uk/rjk/

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hello,

So does your notation ^N mean 2^N-1 then?

Yes.

C: MAXARTNUM ^31
S: 202 2147483647 => 2^31-1?

C: MAXARTNUM ^40
S: 202 1099511627775 => 2^40-1?

MAXARTNUM ^63
202 9223372036854775807 => 2^63-1?

I think your command should just use numbers, rather than your own
private notation.

As far as I know, for two decades there is nothing "official" about
large numbers, so how this notation can be "private"?

If readers wants to use MAXARTNUM with a number, it's accepted.
If readers wants to use a shorten way to set a number, it's accepted.

Btw, when I use a telnet session to test everything on my server, I'm
very happy to write ^63 instead of 9223372036854775807. The risk that I
am wrong by one number is much lower! :)


Have nice day,
Franck

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

"I could also live with a notation like "^31" being allowed, in both directions, as a synonym for 2 to that power minus 1; that is:

I preferred to code this notation in "one direction" only. Thus, the
reader does not need to interpret the result and gets the MAXARTNUM
directly.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hello,

I think your command should just use numbers, rather than your own
private notation.

To be more precise, it's stated as a "proposal" by Clive for a BIGNUM
extension :

https://lists.eyrie.org/pipermail/ietf-nntp/2005-July/005802.html


Clive said :

"I could also live with a notation like "^31" being allowed, in both directions, as a synonym for 2 to that power minus 1; that is:

BIGNUM ^40

is equivalent (if my calculator isn't lying) to:

BIGNUM 1099511627775

though, again, it isn't a major issue."

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Franck <franck@email.invalid> wrote:

BIGNUM ^40

is equivalent (if my calculator isn't lying) to:

BIGNUM 1099511627775

UNIX bc says:

schaefer@reliand:~$ echo '2^40 - 1' | bc
1099511627775

So it seems ok.

I doubt the ^ syntax is useful, everyone should have a bash around :)

--
Attention: limitez le nombre de lignes de citation à l'essentiel, sinon
je ne verrai pas votre réponse.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hello Marc,

I doubt the ^ syntax is useful, everyone should have a bash around

I am quite surprised that a syntax should be discarded just because it
seems useless to some people when it is a simple alternative.

I would understand if it were the only one offered, but it is not.

If a reader wants to use this syntax, it can and if it don't want to, it
can too. In any case, the reader will receive the same reply for both
syntaxes, namely a number.

So I don't see any problem with this.

I think it would be more interesting to question the usefulness of a
server accepting a goofy number like MAXARTNUM 2147483650 or 72341342594!

Have nice day,
Franck

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Franck <Em@il.invalid> wrote:

So I don't see any problem with this.

Nor do I see a problem, it's just not really needed as long as you know
what you put in the field.

It usually helps, when you define standards and protocols, to not have
too many ways to do things, especially on APIs.

Or at least this is my opinion after implementing a filtering proxy on
top of INN and having to handle all the different ways you can select an article :)

--
Attention: limitez le nombre de lignes de citation à l'essentiel, sinon
je ne verrai pas votre réponse.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hello Marc,

Nor do I see a problem, it's just not really needed as long as you know
what you put in the field.

It usually helps, when you define standards and protocols, to not have
too many ways to do things, especially on APIs.

Or at least this is my opinion after implementing a filtering proxy on
top of INN and having to handle all the different ways you can select an article :)

Hi Marc,

You are totally right but MAXARTNUM is largely inspired by "BIGNUM" and
its author had proposed this syntax, which is why I added it (with an obligation for the server to reply with a number).

Something in the spirit of "The client MAY use any form of syntax but
the server MUST reply with a number".

Personally, the "^N" syntax is not the one I would have chosen since in
my opinion the maximum number is only the maximum value of an integer type.

As an *** example ***, I have just added the method I would have used
instead of ^N (Again, words are example!).


C: MAXARTNUM INT32
S: 202 2147483647 is our mutual maximum article number.

C: MAXARTNUM UINT32
S: 202 4294967295 is our mutual maximum article number.

C: MAXARTNUM INT64
S: 202 9223372036854775807 is our mutual maximum article number.

Have nice day,
Franck

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Salut Franck,

Allez les Bleus !! :-)

- If MAXARTNUM disabled for "anonymous" accessgroup, server will reply
with "480 authentification required".

- If MAXARTNUM disabled for another accessgroup, server reply with "502 MAXARTNUM command disabled by administrator".

That seems good.

- MAXARTNUM accept a number as argument or ^31 -> ^63. If MAXARTNUM <
^31 or > ^63, server reply with "501 syntax error".

Incidentally, is there a reason for Spitfire not to handle numbers up to 2^64-1, using unsigned long long?

I would not return 501 if MAXARTNUM > your maximum.

MAXARTNUM ^100
202 (2^63-1) <-- the mutual maximum negotiated

I would only return 501 if there is more than 2 arguments or, when an
argument is given, this argument is below 2^31-1 or is not a number or
is not "^" followed with a number.

- If a reader use NEXT to access numbers > ^31, server reply with "401 maximum article number reached".

- Once used, MAXARTNUM cannot be set again.

So, before using MAXARTNUM, it should be:

401 MAXARTNUM

And after using it, I propose:

502 Next article has an article number larger than our mutual maximum

- Articles > ^31 can be retrieved with their <Message-ID>. In this case, article number is set to "0" and ALL newsgroups in Xref header with a
value > 2^31 are "on-the-fly" set to 0. Size of article is also adjusted "on-the-fly".

Is it really necessary to change the numbers in Xref header fields?

I note in RFC 5536 that article locators are implementation-specific.
They could even be something different than a number.
So I would just keep Xref untouched when sending it in overview data or
in header fields.
Just my opinion. Maybe there is a good reason to either rewrite article locators for large article numbers? (or even remove Xref? - a serving
agent can do that)


xref = "Xref:" SP *WSP server-name
1*( FWS location ) *WSP CRLF

server-name = path-identity

location = newsgroup-name ":" article-locator

article-locator = 1*( %x21-27 / %x29-3A / %x3C-7E )
; US-ASCII printable characters
; except '(' and ';'

The <server-name> is included so that software can determine which
news server generated the header field. The locations specify where
the article is filed -- i.e., under which newsgroups (which may
differ from those in the Newsgroups header field), and where under
those newsgroups. The exact form of an <article-locator> is
implementation-specific.

NOTE: The traditional form of an <article-locator> (as required by
[RFC3977]) is a decimal number, with articles in each newsgroup
numbered consecutively starting from 1.

- NEWNEWS display ALL <Message-ID> (< or > ^31).

Yes!

Please let me know if you find something wrong or if you have any suggestions.

It could be worth mentioning the case of an empty newsgroup with large
numbers. The preferred notation is:

Low = MAXARTNUM
High = MAXARTNUM-1
Count = 0


Also, a second call to MAXARTNUM causes 502 (already used).

--
Julien ÉLIE

« La libertad, Sancho, es uno de los más preciosos dones que a los
hombres dieron los cielos; con ella no pueden igualarse los tesoros
que encierran la tierra y el mar: por la libertad, así como por la
honra, se puede y debe aventurar la vida. » (Miguel de Cervantes
Saavedra)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi all,

We've planned to use the 202 response code for MAXARTNUM.

I see in Diablo's code that it knows:

LIST EXTENSIONS
202 Extensions supported:
HDR
OVER
.


Well, LIST EXTENSIONS is not standardized and now that CAPABILITIES
exists, it is no longer useful. So I think we can safely re-use that
202 code unless someone knows other news servers already using 202 for
an extension that could be worthwhile also standardizing?



Also, RFC 977 defined 202 for the SLAVE command:

202 slave status noted

but it was deprecated and normally no longer used today.

--
Julien ÉLIE

« Ce vieux forban d'Asthmatix, il ne manquait pas d'air ! » (Astérix)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi all,

A question about what would be better to advertise in CAPABILITIES for MAXARTNUM.

Do you think it should reflect the current maximum article number that
the news server will provide, or the maximum article number it can
handle when negotiating it?

I would tend to prefer the first one, though it is not what we have
discussed until now.


I'll use the ^n notation below, meaning 2^n-1.
We'll have to discuss whether we should keep it (client<->server or only client->server) or only use numbers.
I have mixed feelings about it. It is true that it is easier to use it
when debugging but... we're not many to do such NNTP debugging by hand,
and if need be, we can have the commands in a text file and copy/paste
them when needed.
Debugging some extensions like SASL isn't straight-forward anyway, and
needs special commands.
So maybe ^n should not be standardized, but of course could remain
"private" for use by the developers of the implementation.



First scenario
--------------

The news server handles up to 2^64-1 articles but, at startup, is
configured to show only up to 2^31-1 to clients for interoperability
reasons.
It then advertises in CAPABILITIES:
MAXARTNUM ^31

If a news client sends MAXARTNUM ^63, it will then answer 202 with that
new mutual maximum and advertise in CAPABILITIES:
MAXARTNUM ^63

Had the client sent MAXARTNUM ^64 or MAXARTNUM ^128 or MAXARTNUM without
any argument (saying it can handle arbitrary numbers), the new mutual
maximum advertised would have been:
MAXARTNUM ^64



Second scenario
---------------

Same hypothesis but the news server advertises in CAPABILITIES its
maximum article number handled:
MAXARTNUM ^64

If a news client sends MAXARTNUM ^63, it will then answer 202 with that
new mutual maximum and then no longer advertise MAXARTNUM in
CAPABILITIES (or it could still advertise "MAXARTNUM ^64" but there's no
point in doing that).




Any thoughts?

As far as I am concerned, I prefer the first scenario because it makes
it clear what the maximum article number in effect is.
If the news client is OK with that, it does not have to send MAXARTNUM. Otherwise, in the second scenario, it does not know what is the current
mutual maximum so it needs sending it in order not to be blocked to 2^31-1.

An implementation could even prefer to have MAXARTNUM set to ^64 at
startup and not ^31 (the initial number should not be mandatory - just a recommendation that in 2022, ^31 is still a good choice for
interoperability).

--
Julien ÉLIE

« La perfection est atteinte non pas lorsqu'il n'y a plus rien à
ajouter, mais lorsqu'il n'y a plus rien à retirer. » (Saint-Exupéry)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hello Julien,

I'll use the ^n notation below, meaning 2^n-1.
We'll have to discuss whether we should keep it (client<->server or only client->server) or only use numbers.

Since this number is (usually) based on the integer type used by the
developer, I really don't see the point of accepting anything other than
the maximum values of the main integer types.

So I'll go for a single, simple syntax, in both directions, like :

MAXARTNUM INT32 => ^31
MAXARTNUM UINT32 => ^32
MAXARTNUM INT64 => ^63
MAXARTNUM UINT64 => ^64
MAXARTNUM => Unlimited.

Away goofy numbers, away very confusing ^N (meaning ^N-1 !).

What is the real benefit of a MAXARTNUM 1234567890?
What is the real benefit of a MAXARTNUM ^35?

So maybe ^n should not be standardized, but of course could remain
"private" for use by the developers of the implementation.

No need to develop a hidden functionality with the previous syntax.

First scenario
--------------

The news server handles up to 2^64-1 articles but, at startup, is
configured to show only up to 2^31-1 to clients for interoperability
reasons.
It then advertises in CAPABILITIES:
MAXARTNUM ^31

MAXARTNUM INT32

If a news client sends MAXARTNUM ^63, it will then answer 202 with that
new mutual maximum and advertise in CAPABILITIES:
MAXARTNUM ^63

C: MAXARTNUM INT64
S: 202 INT64

CAPABILITIES : MAXARTNUM INT64

Had the client sent MAXARTNUM ^64 or MAXARTNUM ^128 or MAXARTNUM without
any argument (saying it can handle arbitrary numbers), the new mutual
maximum advertised would have been:
MAXARTNUM ^64

MAXARTNUM UINT64

As far as I am concerned, I prefer the first scenario because it makes
it clear what the maximum article number in effect is.
If the news client is OK with that, it does not have to send MAXARTNUM. Otherwise, in the second scenario, it does not know what is the current mutual maximum so it needs sending it in order not to be blocked to 2^31-1. An implementation could even prefer to have MAXARTNUM set to ^64 at
startup and not ^31 (the initial number should not be mandatory - just a recommendation that in 2022, ^31 is still a good choice for interoperability).

As it seems servers needs to live with old readers, i prefer the initial
choice set to MAXARTNUM INT32 and, if reader support MAXARTNUM, use it
to set an integer type.

Have nice day,
Franck

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hello,

The "MAXARTNUM" extension is deactivated on my server, while I make the
changes that will reflect the progress of the discussions.

Franck

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Salut Julien,

Allez les Bleus !! :-)

Ma télé a failli ne pas survivre! :-)

Incidentally, is there a reason for Spitfire not to handle numbers up to 2^64-1, using unsigned long long?

Now, it can.

I would not return 501 if MAXARTNUM > your maximum.

MAXARTNUM ^100
202 (2^63-1) <-- the mutual maximum negotiated

I would only return 501 if there is more than 2 arguments or, when an argument is given, this argument is below 2^31-1 or is not a number or
is not "^" followed with a number.

Hum, is there a reason to do :

C: MAXARTNUM ^100
S: 202 (2^63-1) <-- the mutual maximum negotiated

and not :

C: MAXARTNUM ^1
S: 202 (2^31-1) <-- the mutual minimum negociated

?

Again, I don't see the point to set a value other than the maximum value
of an int type or "nothing" for "unlimited".

401 MAXARTNUM

And after using it, I propose:

502 Next article has an article number larger than our mutual maximum

Ok for me, sounds cool.

- Articles > ^31 can be retrieved with their <Message-ID>. In this
case, article number is set to "0" and ALL newsgroups in Xref header
with a value > 2^31 are "on-the-fly" set to 0. Size of article is also
adjusted "on-the-fly".

Is it really necessary to change the numbers in Xref header fields?

It is not "necessary" but, I mean, if artcile number is set to "0", why
not to do the same in Xref?

I know it's a bit of coding but, why not?

So I would just keep Xref untouched when sending it in overview data or
in header fields.

For overview data (OVER/HDR), Xref is not concerned because you can't go outside the [range] (first-MAXARTNUM), so Xref is untouched.

- NEWNEWS display ALL <Message-ID> (< or > ^31).

Yes!

:)

It could be worth mentioning the case of an empty newsgroup with large numbers.  The preferred notation is:

Low = MAXARTNUM
High = MAXARTNUM-1
Count = 0

Sounds ok for me.

Also, a second call to MAXARTNUM causes 502 (already used).

Sounds good toO.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hello,

MAXARTNUM INT32        => ^31

I think this one is not needed to be recognized as it's not a large
number, but it's usefull to set the "initial" value.

:-)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hello,

An implementation could even prefer to have MAXARTNUM set to ^64 at
startup and not ^31 (the initial number should not be mandatory - just a recommendation that in 2022, ^31 is still a good choice for interoperability).

Because Russ still have trn users (:-)), I think we MUST keep the
"legacy" NNTP int type as default value.

If client want to use large numbers, this extension is for him.


So I will go for :

Startup : INT32 is initial and "legacy" MAXARTNUM value (2^31-1),
otherwise some clients will LIST and receive newsgroups with large
numbers (No pointers adjustments).


C: CAPABILITIES
S: MAXARTNUM UINT64 (The highest number the server support)

Then :

C: MAXARTNUM UINT64 (Or less)
S: 202 UINT64 (Or less)

C: CAPABILITIES
S: *** MAXARTNUM is NOT in the list (Meaning "already set") ***

C: MAXARTNUM INT32
S: 502 mutual maximum article number already negotiated and set to UINT64.

Any thoughts?

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Salut Franck,

I would only return 501 if there is more than 2 arguments or, when an
argument is given, this argument is below 2^31-1 or is not a number or
is not "^" followed with a number.

Hum, is there a reason to do :

C: MAXARTNUM ^100
S: 202 (2^63-1) <-- the mutual maximum negotiated

and not :

C: MAXARTNUM ^1
S: 202 (2^31-1) <-- the mutual minimum negociated

The only point I see in not sending a syntax error (501) to MAXARTNUM
with an article number below 2^31-1 is that there was no negotiation.
The client says it can handle article numbers up to 1, and the server
answers OK let's go up to 2^31-1. That seems weird.
Look for instance at how TLS clients and servers negotiate the cipher or protocol version to use. If a client wants TLS 1.0 and the server only
offers 1.2 and above, it won't answer OK let's do 1.2.
Or if a client sends "LIST ACTIVE news.*" and the server answers OK I'll
send you the active list of news.* and also of comp.* :-)

Again, I don't see the point to set a value other than the maximum value
of an int type

Because you don't actually know what is the value of an "int type" on a
given architecture.
For instance, in the C standard, a long long is guaranteed to be *at
least* 64 bits. It means that you could end up having implementations
which accept 2^128-1. Why restrict them by sending a syntax error if
the number is greater than 2^64-1?

Also, we don't know which types will exist in 10 years. People may
wonder why we restricted numbers to 64 bits!
Remember the first sentence in Clive's mail :-)

Firstly, let us learn from the past and *not* call this - or make
it - a "64 bit" facility.

- Articles > ^31 can be retrieved with their <Message-ID>. In this
case, article number is set to "0" and ALL newsgroups in Xref header
with a value > 2^31 are "on-the-fly" set to 0. Size of article is
also adjusted "on-the-fly".

Is it really necessary to change the numbers in Xref header fields?

It is not "necessary" but, I mean, if artcile number is set to "0", why
not to do the same in Xref?

I know it's a bit of coding but, why not?

You can; the value of the article locator is implementation-specific.
You MAY set the article to "0" or remove the whole Xref header field if
you want, but are not required to.

So I would just keep Xref untouched when sending it in overview data
or in header fields.

For overview data (OVER/HDR), Xref is not concerned because you can't go outside the [range] (first-MAXARTNUM), so Xref is untouched.

OVER and HDR both accept either an article number (or a range) or a
Message-ID. If Spitfire does not implement OVER <mid> then there's
indeed nothing to do for that command. Otherwise, you MAY do something.

--
Julien ÉLIE

« Bonjour ! Je m'appelle Bill Portes et mon boulot, c'est de vendre des
fenêtres. »

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Salut Franck,

Since this number is (usually) based on the integer type used by the developer, I really don't see the point of accepting anything other than
the maximum values of the main integer types.

So I'll go for a single, simple syntax, in both directions, like :

MAXARTNUM INT32        => ^31
MAXARTNUM UINT32    => ^32
MAXARTNUM INT64        => ^63
MAXARTNUM UINT64    => ^64
MAXARTNUM        => Unlimited.

Away goofy numbers, away very confusing ^N (meaning ^N-1 !).

OK for removing the confusing ^N notation.
However, I still have a concern about what you call "goofy numbers".
What would be their definition? :-)

I explain: how could you be sure that all implementations are limited to
the proposed types?
32-bit Perl for instance can go up to signed 2^53-1 integers on systems
with double-precision floats (after that number, the integer precision
is lost).
And I bet there are other languages with different possible limits.

If a news client or server advertises a number, it is normally for a
good reason, and not a random one changing at each session.

That's why I'm still not convinced that we should introduced names types
like UINT64 as it seems to be restrictive and not easily scalable in the future. We would for instance have to handle a IANA registry of such
names and if we want to add UINT128 or any other type (based on powers
of 3 or anything else), it will be difficult to have implementations be
updated to understand these new types.
On the contrary, numbers will always be numbers :-)

What is the real benefit of a MAXARTNUM 1234567890?
What is the real benefit of a MAXARTNUM ^35?

Probably not these numbers, but 2^53-1 and 2^128-1 may are. As well as
other kinds of ints with future processors or compilers.

--
Julien ÉLIE

« Mens sana in corpore sano. » (Juvénal)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hello,

First, please note that I didn't read the whole discussion, so please
excuse me if I say something that was said already.

Le 11/12/2022 11:42, Julien ÉLIE a écrit :

[...]

I explain: how could you be sure that all implementations are limited to
the proposed types?
32-bit Perl for instance can go up to signed 2^53-1 integers on systems
with double-precision floats (after that number, the integer precision
is lost).

Same thing with JavaScript.

And I bet there are other languages with different possible limits.

[...] any other type (based on powers of 3 or anything else), it will be difficult to have implementations be updated to understand these new types. On the contrary, numbers will always be numbers :-)

I completely agree with all that.

By the way, if an implementation is ready to accept numbers of 63 or 64 bits, it must know that 2^63-1 is 9223372036854775807 (and 18446744073709551615 for 2^64-1), so why not write simply :
MAXARTNUM 9223372036854775807
MAXARTNUM 18446744073709551615

Same with 2^53-1, or even 3^53-1 :
MAXARTNUM 9007199254740991
MAXARTNUM 19383245667680019896796722

Or, why not, 10^30 ?
MAXARTNUM 1000000000000000000000000000000

Keep It Simple Stupid...

--
Olivier Miakinen

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hey Franck,

What is the real benefit of a MAXARTNUM 1234567890?
What is the real benefit of a MAXARTNUM ^35?

Just wondering... if these numbers do not suit your implementation, you
are totally in your good right to answer that your maximum negotiated
article number is 2^31-1 or 2^32-1.
Nobody said that the negotiation should match any number is the range of
your maximum possibility of 2^64-1.

--
Julien ÉLIE

« Commencez à creuser des trous pour planter les piquets. De beaux trous
normands ! » (Astérix)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Franck wrote:
Julien ÉLIE wrote:

I'll use the ^n notation below, meaning 2^n-1.
We'll have to discuss whether we should keep it (client<->server or only client->server) or only use numbers.

Since this number is (usually) based on the integer type used by the developer, I really don't see the point of accepting anything other than
the maximum values of the main integer types.

So I'll go for a single, simple syntax, in both directions, like :

MAXARTNUM INT32 => ^31
MAXARTNUM UINT32 => ^32
MAXARTNUM INT64 => ^63
MAXARTNUM UINT64 => ^64
MAXARTNUM => Unlimited.

Away goofy numbers, away very confusing ^N (meaning ^N-1 !).

Yes, the meaning should not differ from what is written.

What is the real benefit of a MAXARTNUM 1234567890?

I would prefer plain numbers.

Possibly in hexadecimal (if this helps debugging):

MAXARTNUM 7FFFFFFFFFFFFFFF

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Salut Franck,

An implementation could even prefer to have MAXARTNUM set to ^64 at
startup and not ^31 (the initial number should not be mandatory - just
a recommendation that in 2022, ^31 is still a good choice for
interoperability).

Because Russ still have trn users (:-)), I think we MUST keep the
"legacy" NNTP int type as default value.

The main point is that 2^31-1 MUST be the default value with NNTP
Version 2. We MUST conform to RFC 3977:

Note that it is likely that the article number limit of 2,147,483,647
will be increased by a future revision or extension to this
specification. While servers MUST NOT send article numbers greater
than this current limit, client and server developers are advised to
use internal structures and datatypes capable of handling larger
values in anticipation of such a change.

So I will go for :

Startup : INT32 is initial and "legacy" MAXARTNUM value (2^31-1),
otherwise some clients will LIST and receive newsgroups with large
numbers (No pointers adjustments).

C: CAPABILITIES
S: MAXARTNUM UINT64 (The highest number the server support)

Then :

C: MAXARTNUM UINT64 (Or less)
S: 202 UINT64 (Or less)

C: CAPABILITIES
S: *** MAXARTNUM is NOT in the list (Meaning "already set") ***

C: MAXARTNUM INT32
S: 502 mutual maximum article number already negotiated and set to UINT64.

Any thoughts?

We would have:
- Until the MAXARTNUM command is used, CAPABILITIES advertises the
facility with the highest supported article number;
- Once that command has been successfully used (and has returned the
negotiated maximum number in the 202 answer), it is no longer advertised
in CAPABILITIES and returns 502 if sent again;
- The default value is 2^31-1 with VERSION 2 of the NNTP protocol, as advertised in CAPABILITIES.

A future specification of Version 3 may change the default value but is
out of scope for the current initial MAXARTNUM implementation.

By the way, we could advertise in CAPABILITIES both the current value
and the maximum value => it would answer that point and facilitate the migration towards future extensions.

C: CAPABILITIES
S: MAXARTNUM 2147483647 18446744073709551615

for 2^31-1 and 2^64-1

And note "unlimited" with 0 (because the first argument could also one
day be 0...):

C: CAPABILITIES
S: MAXARTNUM 2147483647 0

--
Julien ÉLIE

« N'as-tu jamais fait ces rêves, Neo, qui ont l'air plus vrais que la
réalité ? Si tu étais incapable de sortir de l'un de ces rêves,
comment ferais-tu la différence entre le monde du rêve et le monde
réel ? » (Morpheus, _Matrix_)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Le 11/12/2022 12:08, Michael Bäuerle a écrit :

I would prefer plain numbers.

Yes.

Possibly in hexadecimal (if this helps debugging):

MAXARTNUM 7FFFFFFFFFFFFFFF

Since all numbers are always in decimal (commands ARTICLE, HEAD or BODY, responses to GROUP, LAST or NEXT, etc.), I suppose it would more help
debugging if this number is in decimal too rather than in hexadecimal.


--
Olivier Miakinen

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Salut Julien,

The main point is that 2^31-1 MUST be the default value with NNTP

...

C: CAPABILITIES > S: MAXARTNUM 2147483647 18446744073709551615>> for 2^31-1 and 2^64-1

...

C: CAPABILITIES
S: MAXARTNUM 2147483647 0

Reading this and what you wrote in another article :

"The client says it can handle article numbers up to 1, and the server
answers OK let's go up to 2^31-1. That's seems weird."


Sorry to ask but now, does it seems always weird to reply

C: I want 1
S: Syntax error or

Or

S: You can't go below 2^31-1, so it's 2^31-1

?

So what we do?
We accept 1 or with set a minimal and maximal values?

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hello Julien,

The only point I see in not sending a syntax error (501) to MAXARTNUM
with an article number below 2^31-1 is that there was no negotiation.
The client says it can handle article numbers up to 1, and the server
answers OK let's go up to 2^31-1.  That seems weird.
Look for instance at how TLS clients and servers negotiate the cipher or protocol version to use.  If a client wants TLS 1.0 and the server only offers 1.2 and above, it won't answer OK let's do 1.2.
Or if a client sends "LIST ACTIVE news.*" and the server answers OK I'll
send you the active list of news.* and also of comp.* :-)

I may have misunderstood the purpose of this extension which, if I am
not mistaken, is mainly intended to go beyond the current limitation of
2^31-1.

So it will now allow us to go below the current limitation that we are
trying to overcome? Really?

What is there to negotiate that is not already known from RFC 3977,
namely a 2^31-1 limitation?

What seems strange to me is that an extension, once called "BIGNUM" or "LARGENUMBERS" can accept a number like 1.

But I must be missing something :-)

   Firstly, let us learn from the past and *not* call this - or make
   it - a "64 bit" facility.

Ok for that part and keep the numbers and remove the ^N notation as it
seems too confusing.

OVER and HDR both accept either an article number (or a range) or a Message-ID.  If Spitfire does not implement OVER <mid> then there's
indeed nothing to do for that command.  Otherwise, you MAY do something.

Sorry, I was talking about using theses commands with range. SNS support OVER/HDR with a Message-ID and update the Xref (Like for ARTICLE/HEAD).
With a range it's untouched.


What about the number of articles shown in LIST COUNTS?

Franck

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Salut Julien,

However, I still have a concern about what you call "goofy numbers". > What would be their definition? :-)

What is the real benefit of a MAXARTNUM 1234567890? >> What is the real benefit of a MAXARTNUM ^35?> > Probably not these

numbers, but 2^53-1 and 2^128-1 may are.  As well as > other kinds of
ints with future processors or compilers.
You wrote the definition.

Numbers less than 2^31-1 seem to me to be useless with this extension
(which "extend" the standart and allows to go beyond).

Numbers other than ^N.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Olivier Miakinen wrote:

Le 11/12/2022 12:08, Michael Bäuerle a écrit :

I would prefer plain numbers.

Yes.

Possibly in hexadecimal (if this helps debugging):

MAXARTNUM 7FFFFFFFFFFFFFFF

Since all numbers are always in decimal (commands ARTICLE, HEAD or BODY, responses to GROUP, LAST or NEXT, etc.), I suppose it would more help debugging if this number is in decimal too rather than in hexadecimal.

Agreed. It would be inconsistent if the protocol does not use the same representation for numbers here.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Franck <franck@email.invalid> writes:

So does your notation ^N mean 2^N-1 then?

Yes.

C: MAXARTNUM ^31
S: 202 2147483647 => 2^31-1?

C: MAXARTNUM ^40
S: 202 1099511627775 => 2^40-1?

MAXARTNUM ^63
202 9223372036854775807 => 2^63-1?

I think your command should just use numbers, rather than your own
private notation.

As far as I know, for two decades there is nothing "official" about
large numbers, so how this notation can be "private"?

Nobody else uses it.

If readers wants to use MAXARTNUM with a number, it's accepted.
If readers wants to use a shorten way to set a number, it's accepted.

It’s extra complexity. Complexity leads to bugs and vulnerabilities.
Keep it simple.

--
http://www.greenend.org.uk/rjk/

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Salut Franck,

C: MAXARTNUM ^1
S: 202 (2^31-1) <-- the mutual minimum negociated >>

The only point I see in not sending a syntax error (501) to MAXARTNUM
with an article number below 2^31-1 is that there was no negotiation.
The client says it can handle article numbers up to 1, and the server
answers OK let's go up to 2^31-1.  That seems weird.

I may have misunderstood the purpose of this extension which, if I am
not mistaken, is mainly intended to go beyond the current limitation of 2^31-1.

So it will now allow us to go below the current limitation that we are
trying to overcome? Really?

No, I did not say that we should go below the current mandatory 2^31-1
to support. I was just answering to your proposal of "MAXARTNUM ^1"
returning 202.

I would prefer to return a syntax error (501). It seems weird to return
202 as there was no negotiation at all with that "MAXARTNUM ^1".

My suggestion is to return 501 if there is more than 2 arguments or,
when an argument is given, this argument is strictly below 2^31-1 or is
not a number.

What about the number of articles shown in LIST COUNTS?

Same thing as the estimated article count returned in GROUP commands.
It is limited to the negotiated maximum value. No low water mark, high
water mark or estimated article could should exceed it.

--
Julien ÉLIE

« Petite annonce : Artificier cherche femme canon. »

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Salut Franck,

Sorry to ask but now, does it seems always weird to reply

C: I want 1
S: Syntax error or

Or

S: You can't go below 2^31-1, so it's 2^31-1

?

So what we do?
We accept 1 or with set a minimal and maximal values?

If the client asks for 1, I believe it is a syntax error (501). The
answer to keep 2^31-1 is not good as the client has said it cannot
handle it.

I would consider that scenario like:

C: CAPABILITIES
S: [...]
S: COMPRESS DEFLATE
S: .
C: COMPRESS LZMA
S: 206 You can't negotiate LZMA, so let's go on unencrypted

It is an error (501), not a success (206):

S: 501 Syntax error in compression algorithm name

--
Julien ÉLIE

« Tant qu'il y a des marmites, il y a de l'espoir ! » (Astérix)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Salut Julien,

My suggestion is to return 501 if there is more than 2 arguments or,
when an argument is given, this argument is strictly below 2^31-1 or is
not a number.

Clear now.

What about the number of articles shown in LIST COUNTS?

Same thing as the estimated article count returned in GROUP commands. It
is limited to the negotiated maximum value.  No low water mark, high
water mark or estimated article could should exceed it.

SNS store the real count of articles because I don't like the idea of
formula : "subtract the low water mark from the high water mark and add
one to get an estimate".

But, no way to recount articles after each GROUP/LISTGROUP commands to
get a real number.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Salut Julien,

It is an error (501), not a success (206):

Ok :)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hello,

Please let me know if I missed something.

Here is a telnet session of current updated MAXARTNUM extension :

I have a newsgroup with 4 articles with interesting numbers :

2147483646 <DQoDAeVAbws@dev.spitfire-nntp.fr>
2147483647 <DpJxkOqyBtc@dev.spitfire-nntp.fr>
2147483648 <D3yPBiC8r2A@dev.spitfire-nntp.fr>
4294967296 <DwE867tDRCo@dev.spitfire-nntp.fr>

First, let's try without MAXARTNUM.

C: LIST COUNTS
S: local.test.maxartnum 2147483647 2147483646 2 y

High and estimate are adjusted.

Estimate number is always real number of articles if all articles
numbers are < MAXARTNUM.
If not, it's (MAXARTNUM-First)+1, except if the result is > real count
number.
No number can be > MAXARTNUM.

C: STAT <DwE867tDRCo@dev.spitfire-nntp.fr>
S: 223 0 <DwE867tDRCo@dev.spitfire-nntp.fr> article exist.

Article number set to 0.

C: OVER <DwE867tDRCo@dev.spitfire-nntp.fr>
S: 224 overview information follows:
S: 0 This article is above ^32 Franck <my@mail.is.invalid> Sun, 11 Dec
2022 10:20:36 +0100 <DwE867tDRCo@dev.spitfire-nntp.fr> 1101 1 Xref: dev.spitfire-nntp.fr local.test.maxartnum:0

Article number set to "0", all newsgroup > MAXARTNUM are set to 0

C: GROUP local.test.maxartnum
S: 211 2 2147483646 2147483647 local.test.maxartnum group selected.

C: LISTGROUP
S: 211 2 2147483646 2147483647 local.test.maxartnum group selected.
S: 2147483646
S: 2147483647
S: .

Range is limited to MAXARTNUM.

C: OVER 1-
S: 224 overview information follows:
S: 2147483646 Inside ^31 Franck <my@mail.is.invalid> Fri, 9 Dec 2022 06:08:46 +0100 <DQoDAeVAbws@dev.spitfire-nntp.fr> 1084 1 Xref: dev.spitfire-nntp.fr local.test.maxartnum:2147483646
S: 2147483647 Re: Inside ^31 Franck <my@mail.is.invalid> Fri, 9 Dec 2022 06:09:30 +0100 <DpJxkOqyBtc@dev.spitfire-nntp.fr> <DQoDAeVAbws@dev.spitfire-nntp.fr> 12504
S:.

Range is limited to MAXARTNUM.

C: NEXT
S: 223 2147483647 <DpJxkOqyBtc@dev.spitfire-nntp.fr> article selected.
C: NEXT
S: 401 MAXARTNUM

C: MAXARTNUM 2147483648
S: 202 2147483648 is our mutual maximum article number.
C: NEXT
S: 223 2147483648 <D3yPBiC8r2A@dev.spitfire-nntp.fr> article selected.
C: NEXT
S: 502 next article has a number higher than our mutual maximum.


Now, if I try :

C: MAXARTNUM a
S: 501 syntax error in command or unknown 'MAXARTNUM' option.

C: MAXARTNUM ^63
S: 501 syntax error in command or unknown 'MAXARTNUM' option.

C: MAXARTNUM 1
S: 501 syntax error in command or unknown 'MAXARTNUM' option.

C: MAXARTNUM 19446744073709551616
S: 202 18446744073709551616 is our mutual maximum article number.

C: MAXARTNUM
S: 202 18446744073709551616 is our mutual maximum article number.

C: MAXARTNUM 123456789012
S: 202 123456789012 is our mutual maximum article number.
C: MAXARTNUM
S: 502 maximum article number already negociated to 123456789012.

Now, if I try CAPABILITIES :

C: CAPABILITIES
S: 101 capability list follows:
S: VERSION 2
S: IMPLEMENTATION Spitfire News Server DEV (64-bit)
S: AUTHINFO USER SASL
S: HDR
S: LIST ACTIVE ACTIVE.TIMES COUNTS HEADERS MODERATORS MOTD NEWSGROUPS OVERVIEW.FMT SUBSCRIPTIONS
S: MAXARTNUM 18446744073709551616
S: NEWNEWS
S: OVER MSGID
S: POST
S: READER
S: SASL CRAM-MD5 PLAIN
S: .

C: MAXARTNUM
S: 202 18446744073709551616 is our mutual maximum article number.

C: CAPABILITIES
S: 101 capability list follows:
S: VERSION 2
S: IMPLEMENTATION Spitfire News Server DEV (64-bit)
S: AUTHINFO USER SASL
S: HDR
S: LIST ACTIVE ACTIVE.TIMES COUNTS HEADERS MODERATORS MOTD NEWSGROUPS OVERVIEW.FMT SUBSCRIPTIONS
S: NEWNEWS
S: OVER MSGID
S: POST
S: READER
S: SASL CRAM-MD5 PLAIN
S: .

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Salut Franck,

Estimate number is always real number of articles if all articles
numbers are < MAXARTNUM.
If not, it's (MAXARTNUM-First)+1, except if the result is > real count number.

It is fine not to give the real count.
Your formula is OK. If I have to suggest an "improvement", instead of
the maximum (MAXARTNUM-First+1) it could be a ratio of it as you have
the real count on the whole group:
ceil((MAXARTNUM-First+1)*(Real count)/(High-First+1))

In your example, it would give 1 article because standard deviation is
high (3 articles near 2^31 and 1 near 2^32) but I believe it would work
better in real life.

C: STAT <DwE867tDRCo@dev.spitfire-nntp.fr>
S: 223 0 <DwE867tDRCo@dev.spitfire-nntp.fr> article exist.

exists

Article number set to "0", all newsgroup > MAXARTNUM are set to 0

OK that's fine.

C: OVER 1-
S: 224 overview information follows:
S: 2147483646    Inside ^31    Franck <my@mail.is.invalid>    Fri, 9 Dec
2022 06:08:46 +0100    <DQoDAeVAbws@dev.spitfire-nntp.fr>    1084 1    Xref: dev.spitfire-nntp.fr local.test.maxartnum:2147483646
S: 2147483647    Re: Inside ^31    Franck <my@mail.is.invalid>    Fri, 9
Dec 2022 06:09:30 +0100    <DpJxkOqyBtc@dev.spitfire-nntp.fr> <DQoDAeVAbws@dev.spitfire-nntp.fr>    12504
S:.

I bet it's a copy/paste error (otherwise overview data for the second
article is not complete).

C: NEXT
S: 223 2147483647 <DpJxkOqyBtc@dev.spitfire-nntp.fr> article selected.
C: NEXT
S: 401 MAXARTNUM

Yes, 401 because there really is an article. Otherwise, it would have
been 421 (no next article).

C: NEXT
S: 502 next article has a number higher than our mutual maximum.

OK with 502 because the client needs terminating the connection and
starting another one with a different capability in order to access the
next (existing) article.

C: MAXARTNUM
S: 202 18446744073709551616 is our mutual maximum article number.

I know that I suggested MAXARTNUM without any argument to say that the
client handles arbitrary numbers.
On second thoughts, there would be a problem with the ABNF syntax and
parsing of the 202 answer.
202 is followed with a mandatory number and an optional comment. We
cannot say that 202 is followed with an optional number and an optional
comment (otherwise it would be impossible to distinguish the result if
the comment is a number!).

It would then propose "MAXARTNUM 0" to indicate that.
And naturally, the argument 0 would not be a syntax error (contrary to
other numbers < 2^31-1). And not giving an argument would then be a
syntax error (501 response code).


Apart from these minor nits, all your NNTP session looks fine.

--
Julien ÉLIE

« Et s'il n'en reste qu'un, je serai celui-là ! » (Victor Hugo)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Salut Julien,

Your formula is OK.

Call it F1

the maximum (MAXARTNUM-First+1) it could be a ratio of it as you have
the real count on the whole group:
ceil((MAXARTNUM-First+1)*(Real count)/(High-First+1))

Call it F2

In your example, it would give 1 article because standard deviation is
high (3 articles near 2^31 and 1 near 2^32) but I believe it would work better in real life.

I've tested with theses 9 articles :

2147483646 <DQoDAeVAbws@dev.spitfire-nntp.fr>
2147483647 <DpJxkOqyBtc@dev.spitfire-nntp.fr>
2147483648 <D3yPBiC8r2A@dev.spitfire-nntp.fr>
4294967296 <DwE867tDRCo@dev.spitfire-nntp.fr>
5294967297 <CK3CrU31vF8@dev.spitfire-nntp.fr>
5294967298 <ApEvQzsxg8U@dev.spitfire-nntp.fr>
5294967299 <BqU18UDdrM0@dev.spitfire-nntp.fr>
5294967300 <BKoHnx3doC8@dev.spitfire-nntp.fr>
5294967301 <A18hkgS5YbI@dev.spitfire-nntp.fr>


LIST COUNTS (Best result is 2)
F1: local.test.maxartnum 2147483647 2147483646 2 y
F2: local.test.maxartnum 2147483647 2147483646 1 y

MAXARTNUM 4294967295
LIST COUNTS (Best result is 3)
F1: local.test.maxartnum 4294967295 2147483646 9 y
F2: local.test.maxartnum 4294967295 2147483646 7 y


MAXARTNUM 4294967296
LIST COUNTS (Best result is 4)
F1: local.test.maxartnum 4294967296 2147483646 9 y
F2: local.test.maxartnum 4294967296 2147483646 7 y


MAXARTNUM 5294967296
LIST COUNTS (Best result is 4)
F1: local.test.maxartnum 5294967296 2147483646 9 y
F2: local.test.maxartnum 5294967296 2147483646 9 y

MAXARTNUM 5294967297
LIST COUNTS (Best result is 5)
F1: local.test.maxartnum 5294967297 2147483646 9 y
F2: local.test.maxartnum 5294967297 2147483646 9 y

So, it seems F2 is a better choice except in the case F2 < F1.
I added this condition.

C: STAT <DwE867tDRCo@dev.spitfire-nntp.fr>
S: 223 0 <DwE867tDRCo@dev.spitfire-nntp.fr> article exist.

exists

Done.

Article number set to "0", all newsgroup > MAXARTNUM are set to 0

OK that's fine.

As the exact form of an <article-locator> is implementation-specific,
does it make more sense to set this value to MAXARTNUM?

Xref: dev.spitfire-nntp.fr local.test.maxartnum:MAXARTNUM

I bet it's a copy/paste error (otherwise overview data for the second
article is not complete).

You bet it right.

It would then propose "MAXARTNUM 0" to indicate that.

Done.

And naturally, the argument 0 would not be a syntax error (contrary to
other numbers < 2^31-1).  And not giving an argument would then be a
syntax error (501 response code).

Done but to be sure :

C: MAXARTNUM 0
S: 202 18446744073709551616 is our mutual maximum article number. <= The maximum the server can handle.

?

Franck

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Bonsoir Franck,

the maximum (MAXARTNUM-First+1) it could be a ratio of it as you have
the real count on the whole group:
ceil((MAXARTNUM-First+1)*(Real count)/(High-First+1))

Call it F2
[...] > So, it seems F2 is a better choice except in the case F2 < F1.
I added this condition.

Seems good. At least the estimate will be better.

Article number set to "0", all newsgroup > MAXARTNUM are set to 0

OK that's fine.

As the exact form of an <article-locator> is implementation-specific,
does it make more sense to set this value to MAXARTNUM?

Xref: dev.spitfire-nntp.fr local.test.maxartnum:MAXARTNUM

The drawback I see with setting <article-locator> to MAXARTNUM is that
you'll end up having several articles with the same non-zero article
number, which should normally not occur inside a given newsgroup (each
article has a unique non-null article number).
That's why I would tend to prefer 0.

If anyone has another point of view about it, please tell us!

I have a bit advanced in writing detailed specifications for MAXARTNUM.
I hope to send the first draft to read this week-end.

Meanwhile, here is what I have came up with as for that header field.
I believe it is the most problematic point to discuss in the whole
MAXARTNUM specification. Should we really enforce something about Xref,
and what exactly?



Handling the Xref Header Field
------------------------------

The Xref header field that can be present in the article headers or
overview data traditionally contains article numbers (as
<article-locator> in Section 3.2.14 of RFC 5536). Implementations which
use the traditional form required by RFC 3977 SHOULD set
<article-locator> to 0 (zero) in case <article-locator> would otherwise
be greater than the maximum article number negotiated. This
recommendation is done in conformance with Section 6 of RFC 3977 which
already allows the value 0 (zero) in the NNTP protocol to replace an
article number in some special situation.

It means that when a news client uses the ARTICLE, HDR, HEAD, OVER, or
XPAT commands with a Message-ID as argument, a compliant news server
SHOULD ensure that no article numbers beyond the maximum article number negotiated are present in the Xref header fields. To achieve that, the
news server acting as a serving agent (Section 3.7 of RFC 5537) MAY
alter or delete such Xref header fields. In all cases, it MUST ensure
overview data remains coherent, especially the value of the :bytes
metadata item (Section 8.1.1 of RFC 3977).




** Do we really need to have that requirement?

In fact, we also have an issue with the same NNTP commands with an
article number as argument. (I've still not written text about it.)

Suppose an article is crossposted in group1, with article number 1, and
group2, with article number 2^40... A large number will be present in
the Xref header field. It then means that the news server always has to
check the contents of the Xref header field and do an appropriate change
if needed. It seems overkill and time-consuming!

I am unsure we want to add that bargain to all compliant implementations.

Is it sure some news clients may break on invalid Xref header fields?
(I am under the impression that client implementations are robust to the variety of invalid headers there are in the wide, but I may be wrong...)


At least, I've checked trn source code (bits.c). It parses the number
after ":" in Xref with atol(), so it will get at most INT_MAX and
normally won't crash. (It should of course be tested; that's just an assumption.)


Any thoughts about how to deal with Xref in the MAXARTNUM extension?

Should we just say "MAY set"/"MAY ensure" instead of "SHOULD
set"/"SHOULD ensure"?
Or keep SHOULD so that implementations could be conditionally compliant
if that SHOULD is not honoured?

We can imagine that implementations don't do any checks when they know
they do not carry any newsgroup with large article numbers, and begin
doing such checks and alterations otherwise.

And naturally, the argument 0 would not be a syntax error (contrary to
other numbers < 2^31-1).  And not giving an argument would then be a
syntax error (501 response code).

Done but to be sure :

C: MAXARTNUM 0
S: 202 18446744073709551616 is our mutual maximum article number. <= The maximum the server can handle.

Yes.

--
Julien ÉLIE

« Il faut mépriser l'argent, surtout la petite monnaie. »

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Bonjour Julien,

Xref: dev.spitfire-nntp.fr local.test.maxartnum:MAXARTNUM

That's why I would tend to prefer 0.

Fine for me.

If anyone has another point of view about it, please tell us!

Meanwhile, here is what I have came up with as for that header field.
I believe it is the most problematic point to discuss in the whole
MAXARTNUM specification.  Should we really enforce something about Xref,
and what exactly?

Handling the Xref Header Field
------------------------------

In any case, I understood everything without rereading :)

** Do we really need to have that requirement?

If we limit numbers for the rest, I see no good reason to not do the
same with xref header. I prefer to do it or to remove the entire header
(but we lost the corrects values in case of crossposts).

Suppose an article is crossposted in group1, with article number 1, and group2, with article number 2^40...  A large number will be present in
the Xref header field.  It then means that the news server always has to check the contents of the Xref header field and do an appropriate change
if needed.  It seems overkill and time-consuming!

I don't know how Xref is generated by INN but for SNS the check on the
Xref header is very, very, very simple.

In Xref, I take the values of each numbers after ':', if > MAXARTNUM, I
replace it to 0. Done. All others values are kept untouched.

Not overkilling or time-consumming at all.

Crossposted article looks like :
Xref: dev.spitfire-nntp.fr local.test.maxartnum:0 local.test:32

I am unsure we want to add that bargain to all compliant implementations.

Servers MAY but are not required to.

Is it sure some news clients may break on invalid Xref header fields? (I
am under the impression that client implementations are robust to the
variety of invalid headers there are in the wide, but I may be wrong...)

To remove the doubt that you may be wrong, why not to set it to "0"? :)

At least, I've checked trn source code (bits.c).  It parses the number
after ":" in Xref with atol(), so it will get at most INT_MAX and
normally won't crash.  (It should of course be tested; that's just an assumption.)

Same as previous.
"0" or remove the header.

We can imagine that implementations don't do any checks when they know
they do not carry any newsgroup with large article numbers, and begin
doing such checks and alterations otherwise.

This is how SNS already works.
There is no check when no newsgroup carry large numbers and some checks (number/last/count/xref) only on newsgroups that carry large numbers.

C: MAXARTNUM 0
S: 202 18446744073709551616 is our mutual maximum article number. <=
The maximum the server can handle.

Yes.

Ok, to date, MAXARTNUM is finished for me.

I will put it online this evening or tomorow, so we can work with a
"live" extension that can be updated by your "specification".

Have nice day,
Franck

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Salut Franck,

Xref: dev.spitfire-nntp.fr local.test.maxartnum:MAXARTNUM

That's why I would tend to prefer 0.

Fine for me.

Adopted.

Handling the Xref Header Field

In any case, I understood everything without rereading :)

Thanks!
I've just sent more text to proof-read :-)
No need to hurry, it can wait for January, after the feasts of the end
of year.

... and sorry for the additional work it will probably add in some
protocol aspects to make Spitfire even more compliant with the extension :-)

** Do we really need to have that requirement?

If we limit numbers for the rest, I see no good reason to not do the
same with xref header.

Agreed.

Is it sure some news clients may break on invalid Xref header fields?
(I am under the impression that client implementations are robust to
the variety of invalid headers there are in the wide, but I may be
wrong...)

To remove the doubt that you may be wrong, why not to set it to "0"? :)

Yes!

We can imagine that implementations don't do any checks when they know
they do not carry any newsgroup with large article numbers, and begin
doing such checks and alterations otherwise.

This is how SNS already works.
There is no check when no newsgroup carry large numbers and some checks (number/last/count/xref) only on newsgroups that carry large numbers.

Crossposts to newsgroups carrying large article numbers also need
special care.

C: MAXARTNUM 0
S: 202 18446744073709551616 is our mutual maximum article number. <=
The maximum the server can handle.

Yes.

Ok, to date, MAXARTNUM is finished for me.

It is what you thought ^^

--
Julien ÉLIE

« L'ordinateur obéit à vos ordres, pas à vos intentions. »

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Salut Julien,

Ok, to date, MAXARTNUM is finished for me.

It is what you thought ^^

"To date" was used ^^

Have nice day,
Franck

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)