• Proposal specifications for MAXARTNUM

    From =?UTF-8?Q?Julien_=c3=89LIE?=@21:1/5 to All on Mon Dec 19 22:45:08 2022
    Hi all,

    I would like to submit you this first draft about MAXARTNUM, as recently discussed.
    Comments welcome :-)

    If some more explanations or examples are needed, or some sentences are
    not clear, or etc., feel free to tell what should be improved.





    This document defines an extension to the Network News Transfer Protocol
    (NNTP) that allows implementations to effectively negotiate and make use
    of article numbers greater than the NNTP base specification [RFC3977]
    allows.



    Introduction

    The NNTP base specification defines the three types of keys used to
    store and index news articles (see Section 6 of [RFC3977]). Besides the Message-ID and arrival timestamp, one of these types of keys is the
    composition of a newsgroup name and an article number within that newsgroup.

    For interoperability reasons with existing implementations that do not
    use internal structures and data types capable of handling values
    greater than 2,147,483,647 (that is to say 2^31-1) for article numbers,
    Section 6 of [RFC3977] requires that article numbers lie between 1 and 2,147,483,647, inclusive. It implies that a given newsgroup cannot
    contain more than this number of articles. Nonetheless, some newsgroups
    appear in practice to contain more articles than this limit, which led
    some NNTP implementations not to comply with that rule, and advertise
    article numbers greater than the NNTP standard allows. It results in
    some compliant news clients experiencing malfunctions upon receiving unexpectedly large article numbers in responses to NNTP commands from
    such implementations.

    The point of MAXARTNUM as an NNTP extension is to respond to a
    long-standing need for NNTP to standardize a mechanism that will show
    large article numbers only to clients capable of handling them.

    The MAXARTNUM command is sent by a news client to indicate the highest
    article number it can cope with. This is useful when the news server
    supports this extension and carries newsgroups whose reported high water
    mark is 2,147,483,647, or very close to; otherwise, a client does not
    need using this extension with this specific news server as the client
    can already access all the articles the server carries.

    Using this extension enables both the client and the server to negotiate
    a mutual maximum article number that can be used during the rest of the
    NNTP session. Article numbers (including reported high water marks and reported low water marks) as well as estimated article counts can then
    exceed 2,147,483,647, up to the mutual maximum article number.

    Some server implementations pre-dating this extension are
    unconditionally advertising article numbers greater than 2,147,483,647
    for some newsgroups no matter the client handles them. Such
    implementations are encouraged to support this extension.

    It is RECOMMENDED that server implementations permit configuring the
    maximum article number implicitly set at the start of an NNTP session,
    whose default MUST be 2,147,483,647 when version 2 of NNTP is advertised
    in CAPABILITIES (Section 3.3.2 of [RFC3977]). The default value MAY be different in future versions of NNTP. This implicit maximum article
    number MAY be global to the news server or configurable at a finer
    level. For instance, the maximum article number could be implicitly set
    to 18,446,744,073,709,551,615 (2^64-1) for certain users either at the
    start of the NNTP session or after the use of a mechanism identifying
    them (e.g. authentication with AUTHINFO). Configuring a value greater
    than this default value SHOULD be done only for users known to run a
    news client capable of handling article numbers up to that configured value.

    This specification should be read in conjunction with the NNTP base specification [RFC3977]. In the case of a conflict between these two documents, [RFC3977] takes precedence.



    The MAXARTNUM Extension

    At the start of an NNTP session, the mutual maximum article number MUST
    be implicitly set to 2,147,483,647, as required for version 2 of NNTP
    (Section 6 of [RFC3977]). The MAXARTNUM extension is used to increase
    that value, if possible, and explicitly negotiate the highest article
    number that can be used in an NNTP session between a news client and a
    news server.

    This extension provides a new MAXARTNUM command and has the capability
    label MAXARTNUM.



    Advertising the MAXARTNUM Extension

    A server supporting the MAXARTNUM command as defined in this document
    will advertise the "MAXARTNUM" capability label in response to the
    CAPABILITIES command (Section 5.2 of [RFC3977]). However, this
    capability MUST NOT be advertised once the MAXARTNUM command has
    successfully been executed.

    This capability MAY be advertised both before and after any use of the
    MODE READER command (Section 5.3 of [RFC3977]), with the same semantics.

    The MAXARTNUM capability MUST NOT be advertised in response to the
    CAPABILITIES command if the MAXARTNUM command cannot be used in the
    current state of the connection (e.g. if authentication is required
    before using it).

    The MAXARTNUM capability label is followed by a number. This number
    indicates the highest article number that the server can handle, which corresponds to the maximum number of articles a newsgroup can contain in
    this news server.

    This number MUST NOT have leading zeroes, and MUST NOT be lower than 2,147,483,647 with the exception that a value of 0 (zero) is valid,
    which indicates that the server can handle article numbers of any size.


    Example of a server advertising it can handle article numbers up to 2^64-1:

    [C] CAPABILITIES
    [S] 101 Capability list:
    [S] VERSION 2
    [S] MAXARTNUM 18446744073709551615
    [S] READER
    [S] LIST ACTIVE NEWSGROUPS
    [S] .



    MAXARTNUM Command

    Usage

    This command MUST NOT be pipelined.

    Syntax

    MAXARTNUM number

    Responses
    202 number Mutual maximum article number negotiated
    502 Command unavailable [1]

    [1] If the MAXARTNUM command has already been successfully used,
    MAXARTNUM is no longer a valid command.

    Parameter
    number Highest article number supported by the client

    Description

    The MAXARTNUM command instructs the server that it can safely return
    article numbers and estimated article counts up to the proposed number
    during the rest of the NNTP session, instead of limiting these numbers
    to 2,147,483,647 in accordance with version 2 of NNTP.

    The client MUST NOT send any further commands until it has seen the
    result of MAXARTNUM as it may affect subsequent answers by the server.

    If the MAXARTNUM command does not have exactly one argument, or the
    requested number is syntactically incorrect, or below 2,147,483,647 with
    the exception of 0 (zero), the server MUST reject the MAXARTNUM command
    with a 501 response code ([RFC3977], Section 3.2.1). Otherwise, in case
    no other generic response code representing the situation applies, the
    server issues a 202 response code followed by the negotiated mutual
    maximum article number. Although this negotiated number MAY be
    different than the number provided in the MAXARTNUM command, it MUST
    either be 0 (zero), or lie between 2,147,483,647 and the number provided
    in the MAXARTNUM command.

    Article numbers and estimated article counts in all NNTP commands and
    responses sent after this 202 response code MAY be beyond 2,147,483,647
    but MUST lie between 1 and the negotiated mutual maximum article number.

    Additionally, the client MUST NOT issue a MODE READER command after a successful negotiation of the mutual maximum article number with the
    MAXARTNUM command, and a server MUST NOT advertise the MODE-READER
    capability.

    A server MUST NOT return the MAXARTNUM capability label in response to a CAPABILITIES command received after a successful use of the MAXARTNUM
    command, and a server MUST reply with a 502 response code if a
    syntactically valid MAXARTNUM command is received after an already
    successful use of this command.


    Example of a server no longer advertising the MAXARTNUM capability label
    and refusing to negotiate a new mutual maximum article number after a successful use of MAXARTNUM:

    [C] CAPABILITIES
    [S] 101 Capability list:
    [S] VERSION 2
    [S] READER
    [S] MAXARTNUM 18446744073709551615
    [S] LIST ACTIVE NEWSGROUPS
    [S] .
    [C] MAXARTNUM 9223372036854775807
    [S] 202 9223372036854775807 is our mutual maximum article number
    [C] CAPABILITIES
    [S] 101 Capability list:
    [S] VERSION 2
    [S] READER
    [S] LIST ACTIVE NEWSGROUPS
    [S] .
    [C] MAXARTNUM 9223372036854775807
    [S] 502 Mutual maximum article number already negotiated
    [C] MAXARTNUM 18446744073709551615
    [S] 502 Mutual maximum article number already negotiated

    Example of the negotiation of an unlimited article number size (in the
    limit of the length of an NNTP argument, which is 497 octets):

    [C] CAPABILITIES
    [S] 101 Capability list:
    [S] VERSION 2
    [S] READER
    [S] MAXARTNUM 0
    [S] LIST ACTIVE NEWSGROUPS
    [S] .
    [C] MAXARTNUM 0
    [S] 202 0 No limit in article numbers. Enjoy!
    [C] GROUP misc.test
    [S] 211 1 1 1234567890123456789012345678901234567890 misc.test
    [C] STAT 1234567890123456789012345678901234567890
    [S] 223 0 <article@example.net> retrieved

    Example of syntax errors:

    [C] MAXARTNUM 42
    [S] 501 Article number too low
    [C] MAXARTNUM
    [S] 501 Article number missing
    [C] MAXARTNUM 42 4294967295
    [S] 501 Too many arguments
    [C] MAXARTNUM 4294xx7295
    [S] 501 Not an article number
    [C] MAXARTNUM ^64
    [S] 501 Argument not understood; sorry, Clive!

    Example of a mode-switching server no longer advertising the MAXARTNUM
    and MODE-READER capability labels after a successful use of MAXARTNUM:

    [C] CAPABILITIES
    [S] 101 Capability list:
    [S] VERSION 2
    [S] IHAVE
    [S] MODE-READER
    [S] MAXARTNUM 18446744073709551615
    [S] LIST ACTIVE COUNTS
    [S] .
    [C] MAXARTNUM 9223372036854775807
    [S] 202 9223372036854775807 is our mutual maximum article number
    [C] CAPABILITIES
    [S] 101 Capability list:
    [S] VERSION 2
    [S] IHAVE
    [S] LIST ACTIVE COUNTS
    [S] .



    How This Extension Affects Implementations

    Group Information

    The overall behaviour of the reported low water mark and the reported
    high water mark as defined in Section 6.1 of [RFC3977] is not changed by
    this extension, except that the news server MUST NOT return for them a
    value greater than the current mutual maximum article number. For each
    of these values that is greater than the current mutual maximum article
    number, the news server MUST use the current mutual maximum article
    number in its treatments and responses instead of the real value for
    newsgroups which are not empty. The estimated number of articles MUST
    then be limited to the range of articles between the returned reported
    low water mark and the returned reported high water mark.

    This affects the responses to GROUP, LIST ACTIVE, LIST COUNTS, and
    LISTGROUP commands.

    The representation of empty newsgroups described in Section 6.1.1.2 of [RFC3977] still applies, within the limitation of the current mutual
    maximum article number which cannot be exceeded.


    Example of the preferred way to show an empty newsgroup, with the
    reported high water mark being one less than the reported low water mark:

    [Assumes the mutual maximum article number is 2147483647.]
    [C] LIST ACTIVE
    [S] 215 List of newsgroups follows
    [S] misc.test 2147483646 2147483647 y
    [S] .
    [C] GROUP misc.test
    [S] 211 0 2147483647 2147483646 fr.test
    [C] STAT
    [S] 401 MAXARTNUM



    Range Argument

    When a range argument is provided to the HDR, LISTGROUP, or OVER
    commands with a dash not followed by another article number, the result
    MUST be limited to all following articles in the newsgroup up to the
    current mutual maximum article number.


    Example of the list of article numbers returned by the LISTGROUP command
    before and after the use of MAXARTNUM:

    [C] CAPABILITIES
    [S] 101 Capability list:
    [S] VERSION 2
    [S] READER
    [S] MAXARTNUM 18446744073709551615
    [S] LIST ACTIVE NEWSGROUPS
    [S] .
    [C] LISTGROUP misc.test 2147483645-
    [S] 211 40 2147483600 2147483647 misc.test list follows
    [S] 2147483645
    [S] 2147483646
    [S] 2147483647
    [S] .
    [C] MAXARTNUM 0
    [S] 202 18446744073709551615 is our mutual maximum article number
    [C] LISTGROUP misc.test 2147483645-
    [S] 211 43 2147483600 2147483650 misc.test list follows
    [S] 2147483645
    [S] 2147483646
    [S] 2147483647
    [S] 2147483648
    [S] 2147483649
    [S] 2147483650
    [S] .



    Article Retrieval and Selection

    The overall behaviour of article retrieval and selection, as well as the interaction of NNTP commands with the currently selected newsgroup and
    the current article number as defined in Sections 6.1, 6.2 and 8 of
    [RFC3977], is not changed by this extension, except in the following cases:

    o when a news client sends an NNTP command which uses the current
    article number (e.g. third form of ARTICLE, BODY, HDR, HEAD, OVER, STAT,
    and XPAT), and that current article number is valid and greater than the current mutual maximum article number,

    o when a news client sends an NNTP command with a specified article
    number (e.g. second form of ARTICLE, BODY, HDR, HEAD, OVER, STAT, and
    XPAT) or without (e.g. NEXT) which would alter the current article
    number to another different one, and that new different article number
    exists and is greater than the current mutual maximum article number,
    but lower than the maximum article number the news server can handle
    with the MAXARTNUM capability,

    o when a news client sends an NNTP command with a specified range
    compounded of an article number optionally followed by a dash and
    another article number (e.g. LISTGROUP, second form of HDR, OVER, and
    XPAT), and that at least one of the specified article numbers is greater
    than the current mutual maximum article number, but none is greater than
    the maximum article number the news server can handle with the MAXARTNUM capability.

    If the NNTP command sent by the news client contains an article number, individually or in a range, greater than the maximum article number the
    news server can handle with the MAXARTNUM capability, a 501 response
    code MUST be returned.

    When one of the above cases happens and the syntax of the NNTP command
    is correct, a 401 response code followed by the MAXARTNUM capability
    label MUST be returned by the news server if this extension has still
    not be successfully used in the NNTP session, and the news server
    supports it in the current state of the connection.

    Otherwise, if this extension has already been successfully used, a 502
    response code MUST be returned. If authentication is possible and
    needed before a possible use of this extension, a 480 response code MUST
    be returned.

    In all these cases, the currently selected newsgroup and the current
    article number MUST NOT be altered.


    Example of a newsgroup containing only one article number, greater than
    the current mutual maximum article number. The current article number
    is valid, and set to the one of the article present in the newsgroup
    when using the GROUP command.

    [Assumes the mutual maximum article number is 2147483647.]
    [C] LIST ACTIVE
    [S] 215 List of newsgroups follows
    [S] misc.test 2147483647 2147483647 y
    [S] .
    [C] GROUP misc.test
    [S] 211 0 2147483647 2147483647 fr.test
    [C] STAT
    [S] 401 MAXARTNUM

    Example of a newsgroup containing 3 articles whose numbers are
    2147483645, 2147483649, and 4294967300:

    [C] CAPABILITIES
    [S] 101 Capability list:
    [S] VERSION 2
    [S] READER
    [S] MAXARTNUM 18446744073709551615
    [S] LIST ACTIVE NEWSGROUPS
    [S] .
    [C] LIST ACTIVE
    [S] 215 List of newsgroups follows
    [S] misc.test 2147483647 2147483645 y
    [S] .
    [C] GROUP misc.test
    [S] 211 1 2147483645 2147483647 misc.test
    [C] STAT 2147483645
    [S] 223 2147483645 <retrievable.without.maxartnum@example.com>
    [C] STAT 2147483648
    [S] 423 No such article number 2147483648
    [C] NEXT
    [S] 401 MAXARTNUM
    [C] MAXARTNUM 4294967295
    [S] 202 4294967295 is our mutual maximum article number
    [C] NEXT
    [S] 223 2147483649 <retrievable.after.maxartnum@example.com>
    [C] NEXT
    [S] 502 Greater than our mutual maximum article number

    Note that the STAT command for article number 2147483648 returned a 423 response code, and not a 401 or 501 response code.

    Example of other NNTP commands which return 401 with the same example of newsgroup containing 3 articles as above:

    [C] CAPABILITIES
    [S] 101 Capability list:
    [S] VERSION 2
    [S] READER
    [S] MAXARTNUM 18446744073709551615
    [S] LIST ACTIVE NEWSGROUPS
    [S] .
    [C] GROUP misc.test
    [S] 211 1 2147483645 2147483647 misc.test
    [C] STAT 2147483649
    [S] 401 MAXARTNUM
    [C] OVER 2147483646-2147483648
    [S] 401 MAXARTNUM
    [C] OVER 2147483646-2147483649
    [S] 401 MAXARTNUM
    [C] OVER 2147483648-
    [S] 401 MAXARTNUM



    Article Posting

    The overall behaviour of article posting, as defined in Section 6.3 of [RFC3977] is not changed by this extension, except when the POST command
    is used and the highest article number of at least one of the newsgroups
    the article is posted to is equal or greater than the current mutual
    maximum article number. This case applies to injecting agents which are
    also acting as serving agents, and therefore maintain a list of article
    numbers in newsgroups. It aims at preventing a news user from sending
    several times the same article in a newsgroup, thinking the posting was
    not done because he does not see the article in the newsgroup (whereas
    in fact the news reading software he is using does not cope with article numbers greater than the mutual maximum article number; without this
    exception, the article would have successfully been posted and not made available to the original poster, but only to users running news
    software capable of handling greater article numbers).

    When this case happens, the article SHOULD neither be made available on
    the news server nor transferred to other news servers. A 401 response
    code followed by the MAXARTNUM capability label SHOULD be returned by
    the news server as the subsequent response to POST, if this extension
    has still not be successfully used in the NNTP session, and the news
    server supports it in the current state of the connection. Otherwise,
    if this extension has already been successfully used, a 502 response
    code SHOULD be returned. If authentication is possible and needed
    before a possible use of this extension, a 480 response code SHOULD be returned.

    NOTE: Article transfer between news servers with IHAVE or TAKETHIS is
    not changed by this extension. Relaying agents SHOULD go on relaying successfully injected messages even though they cannot store or index
    them locally. In case the transfer of an article fails because the
    relaying agent maintains a list of article numbers and the highest
    article number in a newsgroup has reached the maximum article number it
    can handle, this failure MUST be considered permanent so that the
    article is not reproposed again and again. It notably means that the
    436 response code MUST NOT be returned by IHAVE, nor the 400 response
    code by TAKETHIS. These commands MUST either explicitly accept or
    reject the article.



    Behaviour After 401 Response Code

    When a 401 response code followed by the MAXARTNUM capability label is returned, the news client SHOULD use the MAXARTNUM command described in
    this document to negotiate a greater mutual maximum article number.
    After a successful negotiation, the news client MAY send again the
    command for which it previously received a 401 response code.



    Handling the Xref Header Field

    The Xref header field that can be present in the article headers or
    overview data traditionally contains article numbers (as
    <article-locator> in Section 3.2.14 of [RFC5536]). Implementations
    which use the traditional form required by [RFC3977] SHOULD set <article-locator> to 0 (zero) in case <article-locator> would otherwise
    be greater than the current maximum article number. This recommendation
    is done in conformance with Section 6 of [RFC3977] which already allows
    the value 0 (zero) in the NNTP protocol to replace an article number in
    some special situation.

    It means that when a news client uses the ARTICLE, HDR, HEAD, OVER, or
    XPAT commands, a compliant news server SHOULD ensure that no article
    numbers beyond the current maximum article number are present in Xref
    header fields. To achieve that, the news server acting as a serving
    agent (Section 3.7 of [RFC5537]) MAY alter or delete such Xref header
    fields. In all cases, it MUST ensure overview data remains coherent, especially the value of the :bytes metadata item (Section 8.1.1 of
    [RFC3977]).

    NOTE: The Xref header field of crossposted articles can end up
    containing both newsgroups with <article-locator> set to 0 (zero) and newsgroups with <article-locator> set to real article numbers.


    Example of the use of NEWNEWS to retrieve articles without using the
    MAXARTNUM extension, although their article numbers are greater than the
    mutual maximum article number. The misc.test newsgroup contains 100
    article numbers, from 3000000000 to 3000000100. The Xref header field
    is altered in the ARTICLE multi-line response block.

    [C] CAPABILITIES
    [S] 101 Capability list:
    [S] VERSION 2
    [S] READER
    [S] MAXARTNUM 18446744073709551615
    [S] LIST ACTIVE NEWSGROUPS
    [S] NEWNEWS
    [S] .
    [C] LIST ACTIVE
    [S] 215 List of newsgroups follows
    [S] misc.test 2147483647 2147483647 y
    [S] .
    [C] NEWNEWS misc.test 20221225 090000
    [S] 230 New news follows
    [S] <article.3000000098@example.com>
    [S] <article.3000000099@example.com>
    [S] <article.3000000100@example.com>
    [S] .
    [C] ARTICLE <article.3000000098@example.com>
    [S] 220 0 <article.3000000098@example.com>
    [S] Path: example!not-for-mail
    [S] From: "Demo User" <nobody@example.com>
    [S] Newsgroups: misc.test
    [S] Subject: A test
    [S] Date: Sun, 25 Dec 2022 09:12:00 -0000 (UTC)
    [S] Message-ID: <article.3000000098@example.com>
    [S] Xref: example misc.test:0
    [S]
    [S] This is just a test article.
    [S] .

    Since the last run of NEWNEWS by the client, 3 new articles have arrived
    in the misc.test newsgroup. The news client retrieves them by
    Message-ID, like in the above example for article number 3000000098.



    Leading Zeroes and Size of Article Numbers

    Section 6 of [RFC3977] allows the use of leading zeroes in article
    numbers as long as they do not result in numbers of more than 16 digits.

    This extension changes that requirement. When the MAXARTNUM capability
    is advertised in response to CAPABILITIES, article numbers sent by a
    compliant implementation MUST NOT use more than 6 digits besides the
    number of digits the number following this capability label has, with
    the exception of when the number is zero (0), in which case 497 digits
    are allowed (corresponding to the maximum length of an NNTP argument, as defined in Section 3.1 of [RFC3977]).

    When the MAXARTNUM command has been successfully used, article numbers
    sent by a compliant implementation MUST NOT use more than 6 digits
    besides the number of digits the negotiated mutual maximum article
    number has. (This is coherent with the fact that 2,147,483,647 has 10
    digits, and the limit of 16 digits in [RFC3977].)

    Example of the change with this extension:

    [C] CAPABILITIES
    [S] 101 Capability list:
    [S] VERSION 2
    [S] READER
    [S] MAXARTNUM 18446744073709551615
    [S] LIST ACTIVE NEWSGROUPS
    [S] .
    [C] LIST ACTIVE
    [S] 215 List of newsgroups follows
    [S] misc.test 2147483647 0000000001 y
    [S] .
    [C] MAXARTNUM 18446744073709551615
    [S] 202 18446744073709551615 is our mutual maximum article number
    [C] LIST ACTIVE
    [S] 215 List of newsgroups follows
    [S] misc.test 00446744073709551615 00000000000000000001 y
    [S] .
    [C] GROUP misc.test
    [S] 211 4846845515592 1 446744073709551615 misc.test
    [C] STAT 00000000000000000001
    [S] 223 1 <first.article@example.net> retrieved
    [C] STAT 000000000000000000000000001
    [S] 501 Too many digits in article number



    Augmented BNF Syntax for the MAXARTNUM Extension

    This section describes the formal syntax of the MAXARTNUM extension
    using ABNF [RFC7405] and [RFC5234]. It extends the syntax in Section 9
    of [RFC3977], and non-terminals not defined in this document are defined
    there. The NNTP ABNF [RFC3977] should be imported first, before
    attempting to validate these rules.

    Commands

    This syntax extends the non-terminal <command>, which represents an NNTP command.

    command =/ maxartnum-command
    maxartnum-command = "MAXARTNUM" WS article-number


    Capability Entries

    This syntax extends the non-terminal <capability-entry>, which
    represents a capability that may be advertised by the server.

    capability-entry =/ maxartnum-capability
    maxartnum-capability = "MAXARTNUM" WS article-number


    General Non-terminals

    As the definition of <article-number> in Section 9.8 of [RFC3977] allows
    only 16 digits, compliant implementations of the MAXARTNUM extension
    MUST be prepared to parse larger numbers of at least the maximum length
    of an NNTP argument (497 octets, as defined in Section 3.1 of
    [RFC3977]). Such large numbers MAY indeed be provided as the argument
    of the MAXARTNUM capability or command. These numbers MUST NOT be
    considered as a syntax error.

    Therefore, implementations of this extension accept the additional syntax:

    article-number =/ 1*497DIGIT


    Summary of Response Codes

    This section defines the following new response code. It is not
    multi-line and has one argument.

    Response code 202
    Generated by: MAXARTNUM
    1 argument: article-number
    Meaning: mutual maximum number negotiated


    Responses

    This syntax extends the non-terminal <initial-response-content>, which represents an initial response line sent by the news server.

    initial-response-content =/ response-202-content
    response-202-content = "202" SP article-number

    NOTE: The 202 response code, once used by the SLAVE command, was removed
    from the NNTP protocol by [RFC3977] (see Appendix D). The MAXARTNUM can
    safely re-use that response code as there is no possibility of confusion.



    Internationalization Considerations

    No new internationalization considerations are introduced by this
    extension, beyond those already described in the core specification
    [RFC3977].



    Security Considerations

    No new security considerations are introduced by this extension, beyond
    those already described in the core specification [RFC3977].

    It may even make compliant implementations more robust in how they deal
    with article numbers as a careful bounds checking will ensure they are
    not affected by integer overflow.



    IANA Considerations

    This section gives a formal definition of this extension as required by
    Section 3.3.3 of [RFC3977] for the IANA registry.

    o The MAXARTNUM extension allows NNTP implementations to effectively
    negotiate and make use of article numbers greater than the limit of 2,147,483,647 required by [RFC3977].

    o The capability label for this extension is "MAXARTNUM", whose argument
    is the highest article number the news server can handle. A value of 0
    (zero) indicates that the news server can handle article numbers of any
    size.

    o This extension defines one new command, MAXARTNUM, whose behaviour, arguments, and responses are defined in this document.

    o This extension does not associate any new responses with pre-existing
    NNTP commands.

    o This extension does not cause any pre-existing NNTP command to produce
    a 483 response. However, in some circumstances described in this
    document, 401 and 480 response codes may be returned by ARTICLE, BODY,
    HDR, HEAD, LISTGROUP, NEXT, OVER, STAT, and XPAT (Section 2.9 of 2980).

    o This extension does not affect the maximum length of NNTP commands or
    initial response lines.

    o This extension does not alter pipelining, but the MAXARTNUM command
    cannot be pipelined.

    o Use of this extension does alter the capabilities list; once the
    MAXARTNUM command has been used successfully, the MAXARTNUM capability
    can no longer be advertised by CAPABILITIES. Additionally, the
    MODE-READER capability MUST NOT be advertised after a successful
    execution of the MAXARTNUM command.

    o This extension is unaffected by any use of the MODE READER command;
    however, the MODE READER command MUST NOT be used in the same session
    following a successful execution of the MAXARTNUM command.

    o This extension does not affect the overall behaviour of a server or
    client other than via the new MAXARTNUM command and the adjustments to
    existing NNTP commands and responses to handle how article numbers are
    shown before and after a successful execution of the MAXARTNUM command.



    Acknowledgements

    The author gratefully acknowledges the comments and additional
    information provided by Russ Allbery, Michael Bäuerle, Urs Janßen,
    Richard Kettlewell, and Olivier Miakinen on preliminary discussions
    which led to this document.

    Special thanks are due to Clive Feather, whose ideas served as the
    initial basis for the MAXARTNUM command, and Franck Rayssiguier for his valuable feedback on this document while concretely implementing it in a
    news server handling 64-bit article numbers.



    --
    Julien ÉLIE

    « J'ai un copain, il est pilote d'essai… Enfin, il ne l'est pas encore ;
    pour l'instant, il essaie d'être pilote ! » (Raymond Devos)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Franck@21:1/5 to All on Tue Dec 20 08:15:57 2022
    Hello Julien,

    Example of the negotiation of an unlimited article number size (in the
    limit of the length of an NNTP argument, which is 497 octets):

        [C] CAPABILITIES
        [S] 101 Capability list:
        [S] VERSION 2
        [S] READER
        [S] MAXARTNUM 0
        [S] LIST ACTIVE NEWSGROUPS
        [S] .
        [C] MAXARTNUM 0
        [S] 202 0 No limit in article numbers.  Enjoy!
        [C] GROUP misc.test
        [S] 211 1 1 1234567890123456789012345678901234567890 misc.test
        [C] STAT 1234567890123456789012345678901234567890
        [S] 223 0 <article@example.net> retrieved

    In this case, the client has instructed the server that it is able to
    "enjoy" article numbers without any limitations (MAXARTNUM 0).

    I think the 223 reply should mention the article number (not 0) and that
    the term "retrieved" should rather be "selected" since STAT is a
    selection command, not a retrieval one.

    "0" must be used when STAT/HEAD/BODY/ARTICLE commands are used by
    "message-id" AND if the article number is > mutual MAXARTNUM. "Xref"
    header MAY reflect the change too or mention "real" article number even
    if it is > mutual MAXARTNUM.

        [S] 501 Argument not understood; sorry, Clive!

    My server reply :

    [S] 501 Argument not understood; Clive, you can't blame me, I really did everything I could!

    :-)

    I'll read more carefully the rest of the document during the day and
    apply changes to the extention on my server ASAP, so we can work with a
    "live" extension.

    Have nice day,
    Franck

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Franck@21:1/5 to All on Tue Dec 20 13:22:36 2022
    Salut Julien,

    Example of a newsgroup containing only one article number, greater than
    the current mutual maximum article number.  The current article number
    is valid, and set to the one of the article present in the newsgroup
    when using the GROUP command.

        [Assumes the mutual maximum article number is 2147483647.]
        [C] LIST ACTIVE
        [S] 215 List of newsgroups follows
        [S] misc.test 2147483647 2147483647 y
        [S] .
        [C] GROUP misc.test
        [S] 211 0 2147483647 2147483647 fr.test
        [C] STAT
        [S] 401 MAXARTNUM


    For this case _AND IF_ I understand correctly, wouldn't it make more
    sense to consider the group "empty" (in 215 and 211 replies) for THIS
    client (with your prefered way to show an empty group with the reported
    high water mark being one less than the reported low water mark)?

    PS: My prefered is to set high & low water marks to "0".

    Also, 211 reply seems to be incoherent. GROUP ask for 'misc.test' and
    server reply with 'fr.test'. But I bet it's a copy/paste error :-)

    Have nice day,
    Franck

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Olivier Miakinen@21:1/5 to All on Tue Dec 20 23:12:35 2022
    Hi Julien,

    Le 19/12/2022 22:45, Julien ÉLIE a écrit :

    I would like to submit you this first draft about MAXARTNUM, as recently discussed.
    Comments welcome :-)

    [...]

    MAXARTNUM Command

    [...]

    If the MAXARTNUM command does not have exactly one argument, or the
    requested number is syntactically incorrect, or below 2,147,483,647 with
    the exception of 0 (zero), the server MUST reject the MAXARTNUM command
    with a 501 response code ([RFC3977], Section 3.2.1). Otherwise, in case
    no other generic response code representing the situation applies, the
    server issues a 202 response code followed by the negotiated mutual
    maximum article number. Although this negotiated number MAY be
    different than the number provided in the MAXARTNUM command, it MUST
    either be 0 (zero), or lie between 2,147,483,647 and the number provided
    in the MAXARTNUM command.

    Maybe we should be more precise on the situation when the negotiated number
    can be 0. Something such as the following.

    §
    maximum article number. Although this negotiated number MAY be
    different than the number provided in the MAXARTNUM command, it MUST
    either be 0 (zero) when the requested number was 0 (zero), or lie
    between 2,147,483,647 and the number provided in the MAXARTNUM command.
    §

    Otherwise, a (false) interpretation could be that the server is free to
    respond 0 when it agrees with the non-zero requested number.

    --
    Olivier Miakinen

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?Julien_=c3=89LIE?=@21:1/5 to All on Tue Dec 20 22:46:39 2022
    Salut Franck,

    Example of a newsgroup containing only one article number, greater
    than the current mutual maximum article number.  The current article
    number is valid, and set to the one of the article present in the
    newsgroup when using the GROUP command.

         [Assumes the mutual maximum article number is 2147483647.]
         [C] LIST ACTIVE
         [S] 215 List of newsgroups follows
         [S] misc.test 2147483647 2147483647 y
         [S] .
         [C] GROUP misc.test
         [S] 211 0 2147483647 2147483647 fr.test
         [C] STAT
         [S] 401 MAXARTNUM

    For this case _AND IF_ I understand correctly, wouldn't it make more
    sense to consider the group "empty" (in 215 and 211 replies) for THIS
    client (with your prefered way to show an empty group with the reported
    high water mark being one less than the reported low water mark)?

    Oh, that's a pretty good point!
    The newsgroup is "empty" from the point of view of this client which
    cannot use or still has not used MAXARTNUM.


    Also, 211 reply seems to be incoherent. GROUP ask for 'misc.test' and
    server reply with 'fr.test'. But I bet it's a copy/paste error :-)

    Totally! Well spotted :)

    --
    Julien ÉLIE

    « Je connais des sourds qui s'entendent très bien ! » (Philippe Geluck)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Olivier Miakinen@21:1/5 to All on Tue Dec 20 23:31:04 2022
    Le 20/12/2022 23:18, Olivier Miakinen a écrit :
    Le 20/12/2022 23:12, I wrote :

    §
    maximum article number. Although this negotiated number MAY be
    different than the number provided in the MAXARTNUM command, it MUST
    either be 0 (zero) when the requested number was 0 (zero), or lie
    between 2,147,483,647 and the number provided in the MAXARTNUM command.
    §

    Oops! It is not correct either, because when there is 0 in the MAXARTNUM command, the server can respond a non-zero number greater than 2,147,483,647, and of course this number doesn't lie between 2,147,483,647 and 0!

    §
    maximum article number. Although this negotiated number MAY be
    different than the number provided in the MAXARTNUM command, it MUST
    respect the following condition: if the number provided in the
    MAXARTNUM command was 0 (zero) it MUST be either 0 itself or a
    number greater than or equal to 2,147,483,647; otherwise it must lie
    between 2,147,483,647 and the number provided in the MAXARTNUM command.
    §


    --
    Olivier Miakinen

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Olivier Miakinen@21:1/5 to All on Tue Dec 20 23:18:29 2022
    Le 20/12/2022 23:12, I wrote :

    §
    maximum article number. Although this negotiated number MAY be
    different than the number provided in the MAXARTNUM command, it MUST
    either be 0 (zero) when the requested number was 0 (zero), or lie
    between 2,147,483,647 and the number provided in the MAXARTNUM command.
    §

    Oops! It is not correct either, because when there is 0 in the MAXARTNUM command, the server can respond a non-zero number greater than 2,147,483,647, and of course this number doesn't lie between 2,147,483,647 and 0!


    --
    Olivier Miakinen

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Franck@21:1/5 to All on Wed Dec 21 05:37:49 2022
    Hello Julien,

         [C] STAT 1234567890123456789012345678901234567890 <=== NUMBER

    Second form of the command.

    I think the 223 reply should mention the article number (not 0)

    Not necessarily.  It is just a MAY according to RFC 3977:

       In the response, the article number MUST be replaced with zero,
       unless there is a currently selected newsgroup and the article is
       present in that group, in which case the server MAY use the article's
       number in that group.

    Sorry but the part you are refering to is reserved for the first form of
    the command, witch is "by message-id".

    First form (message-id specified)
    223 0|n message-id Article exists

    Your example show that STAT use the second form, witch is "by number".

    Second form (article number specified)
    223 n message-id Article exists

    So, IN NO WAY the server may replace the article number by "0".

    If article number exists, reply is "223 number <msgid> article exists"
    and if not, reply is "423 no article with that number".

    So, I think your example MUST show the article number in 223 or you must
    change STAT to be :

    [C] STAT <article@example.net>
    [S] 223 0 <article@example.net>

    Witch mean the article <article@example.net> exists but is not in the
    selected newsgroup (in your example it's 'misc.test').

    and that the term "retrieved" should rather be "selected" since STAT
    is a selection command, not a retrieval one.
    I can indeed change "retrieved" to "selected", though RFC 3977 uses the
    term "retrieved" for ARTICLE, BODY, HEAD, STAT, LAST and NEXT.
    I believe it means "trouvé" in this context, and not "récupéré".

    For STAT, RFC 3977 use "exists" for the 3 forms of the command, witch
    mean article exists and is selected but not retrieved.

    But it's just wording and I'm fine with all terms unless code is 223 :)

    "Xref" header MAY reflect the change too or mention "real" article
    number even if it is > mutual MAXARTNUM.

    I know we discussed it and understood it was useful to alter the header
    field (it is the whole point of not showing article numbers not
    supported by a news client).  I am unsure it's worth leaving a door open
    and saying that an implementation MAY not do that whereas they SHOULD > (Not MUST, but SHOULD.)

    Ok.

    My server reply :

    [S] 501 Argument not understood; Clive, you can't blame me, I really
    did everything I could!

    :-)

    Ah ah!
    You may want to allow the syntax when the connection comes from
    localhost or your computer if it helps in debugging :-)

    Héhé... This syntax is STILL available for ALL in SNS, there is just a
    trick to use it! I called it "the clive trick" but I'll NEVER SAY how to
    use it, even under threat!

    Have nice day,
    Franck

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Franck@21:1/5 to All on Wed Dec 21 07:55:16 2022
    Salut Julien,

    Not necessarily.  It is just a MAY according to RFC 3977:

       In the response, the article number MUST be replaced with zero,
       unless there is a currently selected newsgroup and the article is
       present in that group, in which case the server MAY use the article's
       number in that group.

    Let's see what are the replies from your server :

    200 news.trigofacile.com InterNetNews NNRP server INN 2.8.0 (20221119 prerelease) ready (no posting)

    STAT <tnnd16$m9g$1@shakotay.alphanet.ch>
    223 0 <tnnd16$m9g$1@shakotay.alphanet.ch> status

    OK, article exists and 0 because no newsgroup is selected.

    GROUP fr.test
    211 25213 4089 50948 fr.test
    STAT <tnnd16$m9g$1@shakotay.alphanet.ch>
    223 0 <tnnd16$m9g$1@shakotay.alphanet.ch> status

    Ok, article exists and 0 is not an error because RFC say "MAY use the
    article's number in that group".

    STAT 50948
    223 50948 <tnnd16$m9g$1@shakotay.alphanet.ch> status

    Ok, article is requested "by number" and the reply shows the article number.


    Let's see what are the replies from my server :

    200 news.spitfire-nntp.fr running Spitfire News Server 0.9 (Posting
    allowed).
    STAT <tnnd16$m9g$1@shakotay.alphanet.ch>
    223 0 <tnnd16$m9g$1@shakotay.alphanet.ch> article exists.
    GROUP fr.test
    211 178 10557 10738 fr.test group selected.
    STAT <tnnd16$m9g$1@shakotay.alphanet.ch>
    223 10738 <tnnd16$m9g$1@shakotay.alphanet.ch> article exists.
    STAT 10738
    223 10738 <tnnd16$m9g$1@shakotay.alphanet.ch> article selected.

    The only difference is that SNS shows article number when article is
    requested by message-id (if it is in the selected newsgroup).

    Also wording is a bit more explicit than "status" and use "exist" or
    "selected" for the differents forms of command.

    Wording is different because when the command is used with a message-id,
    it's just a check, article is not selected.

    Have nice day,
    Franck

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Richard Kettlewell@21:1/5 to iulius@nom-de-mon-site.com.invalid on Wed Dec 21 11:01:40 2022
    Julien ÉLIE <iulius@nom-de-mon-site.com.invalid> writes:
    It is RECOMMENDED that server implementations permit configuring the
    maximum article number implicitly set at the start of an NNTP session,
    whose default MUST be 2,147,483,647 when version 2 of NNTP is
    advertised in CAPABILITIES (Section 3.3.2 of [RFC3977]). The default
    value MAY be different in future versions of NNTP. This implicit
    maximum article number MAY be global to the news server or
    configurable at a finer level. For instance, the maximum article
    number could be implicitly set to 18,446,744,073,709,551,615 (2^64-1)
    for certain users either at the start of the NNTP session or after the
    use of a mechanism identifying them (e.g. authentication with
    AUTHINFO). Configuring a value greater than this default value SHOULD
    be done only for users known to run a news client capable of handling
    article numbers up to that configured value.

    This paragraph seems to add scope for incompatibility between servers
    and clients without any clear benefit.

    A server supporting the MAXARTNUM command as defined in this document
    will advertise the "MAXARTNUM" capability label in response to the CAPABILITIES command (Section 5.2 of [RFC3977]). However, this
    capability MUST NOT be advertised once the MAXARTNUM command has
    successfully been executed.

    Why not? Seems like an entirely unnecessary restriction on the server.

    This number MUST NOT have leading zeroes, and MUST NOT be lower than 2,147,483,647 with the exception that a value of 0 (zero) is valid,
    which indicates that the server can handle article numbers of any
    size.

    I don’t think the special case for is 0 is necessary since there is no
    such server: the 497-octet limit provides an upper bound even for
    servers that don’t limit their article numbers to (for example) a
    uint64_t. They can send a value of 2^256-1 or something and be very
    comfortable that the limit will never be reached in reality.

    This applies to both the capabality and the command.

    Additionally, the client MUST NOT issue a MODE READER command after a successful negotiation of the mutual maximum article number with the MAXARTNUM command, and a server MUST NOT advertise the MODE-READER capability.

    The former I can see, but why forbid advertizing MODE-READER after
    MAXARTNUM has been negotiated?

    A server MUST NOT return the MAXARTNUM capability label in response to
    a CAPABILITIES command received after a successful use of the
    MAXARTNUM command, and a server MUST reply with a 502 response code if
    a syntactically valid MAXARTNUM command is received after an already successful use of this command.

    Why?

    --
    https://www.greenend.org.uk/rjk/

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?Julien_=c3=89LIE?=@21:1/5 to All on Wed Dec 21 21:03:52 2022
    Bonsoir Olivier,

    §
    maximum article number. Although this negotiated number MAY be
    different than the number provided in the MAXARTNUM command, it MUST
    either be 0 (zero) when the requested number was 0 (zero), or lie
    between 2,147,483,647 and the number provided in the MAXARTNUM command.
    §

    Oops! It is not correct either, because when there is 0 in the MAXARTNUM
    command, the server can respond a non-zero number greater than 2,147,483,647,
    and of course this number doesn't lie between 2,147,483,647 and 0!

    §
    maximum article number. Although this negotiated number MAY be
    different than the number provided in the MAXARTNUM command, it MUST
    respect the following condition: if the number provided in the
    MAXARTNUM command was 0 (zero) it MUST be either 0 itself or a
    number greater than or equal to 2,147,483,647; otherwise it must lie
    between 2,147,483,647 and the number provided in the MAXARTNUM command.
    §

    Thanks for the proposal of wording. It is indeed clearer this way,
    without any possibility of interpretation.

    I'll adopt your proposal... unless we decide to finally not use 0
    (following Richard's remark).

    --
    Julien ÉLIE

    « PowerPoint allows speakers to pretend that they are giving a real
    talk, and audiences to pretend that they are listening. » (Edward R.
    Tufte, _The Cognitive Style of PowerPoint_)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?Julien_=c3=89LIE?=@21:1/5 to All on Wed Dec 21 23:15:12 2022
    Two additions:

    [limiting high and low water marks, and article counts]
    This affects the responses to GROUP, LIST ACTIVE, LIST COUNTS, and
    LISTGROUP commands.

    And also the NEWGROUPS command.



    I believe a note about other solutions like roll over should be added in
    the document, to explicitly state it is not a good idea.
    The question will probably arise some day!

    Suggestion:

    Server implementations pre-dating this extension, or not capable of
    handling article numbers greater than 2^31-1, have sometimes performed a massive renumbering or a roll over when the highest article number in a newsgroup has been nearing that limit. Such operations come with their
    own set of drawbacks and breakage scenarios with news clients, and do
    not provide a way to populate a newsgroup with more than 2^31-1 existing articles. For these reasons, other solutions than implementing
    MAXARTNUM are not RECOMMENDED by this document.


    --
    Julien ÉLIE

    « 21.1.1 How to convert mSQL tools for MySQL?
    1. Run the shell script msql2mysql on the source. This requires the
    replace program, which is distributed with MySQL.
    2. Compile.
    3. Fix all compiler errors. » (MySQL online manual)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Franck@21:1/5 to All on Thu Dec 22 07:43:43 2022
    Hello Julien et Richard,

    And usually, after the use of commands like AUTHINFO, COMPRESS, STARTTLS which change something in the connection, they are no longer advertised
    in CAPABILITIES.

    I agree with this part.

    I don’t think the special case for is 0 is necessary since there is no
    such server: the 497-octet limit provides an upper bound even for

    It's not because no such server exist today, that it can't exist
    tomorow. We don't know.

    But, to be more honest, I think that any implementation will always have
    a "limit" and that, in fact, it will announce it in CAPABILITIES.

    So, I agree with Richard, 0 seems to be unuseless... At least today!

    So there is a problem for instance with "OVER N-M" where N and M are compounded of 497 digits.
    And also in existing responses, as we may have a large article number followed by a Message-ID up to 250 characters itself!
    And even in GROUP responses, there may be 3 large article numbers
    followed by a newsgroup name.  (Is there an upper bound for the length
    of a newsgroup name?  I do not manage to find it in RFC 3977 or 5536.)

    For newsgroup name, when I look at my code, I see that there is a limit
    of 66 characters and that each component must not exceed 14 characters.

    I don't know where I got these limits, but one thing is certain, I
    didn't impose them on my own. I have to find where I got them ^^

    When MODE-READER is advertised, we're in the case of a mode-switching
    news server.  The same way AUTHINFO, COMPRESS, or STARTTLS forbid advertizing MODE-READER after a successful use of the command, MAXARTNUM
    also has to.  Negotiated states are not shared between transit and
    reading facilities of mode-switching news servers.

    Even if SNS is not a mode-switching server, I agree with this part.

    From previous discussions, it seemed weird that a news client used
    MAXARTNUM twice (or more) with a possible different article number
    asked.  That's the reason of that paragraph.

    I agree it is not mandatory; but without it, it would mean that news
    servers have to cope with several changes in the mutual article number,
    and possibly answer several times "401 MAXARTNUM", and then again a re-negotiation of MAXARTNUM should occur.
    That seems complicated.  Whence that paragraph to forbid that.

    I totaly agree with the fact that a client MUST NOT issue MAXARTNUM more
    than once in a session.

    What is the interest for a client to say, in the same session :

    - Hey server, I can cope with 2^64-1
    - Ahaha, I was joking, I support 2^36-1
    - I changed my mind, I prefer 2^50-1.

    Deal is quite simple :

    - Client knows the limitation of the server (CAPABILITIES),
    - Client fix it's own limit,
    - CLient enjoy large numbers,
    - Done.

    Have nice day,
    Franck

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Franck@21:1/5 to All on Thu Dec 22 11:05:15 2022
    Hello Julien,

    For newsgroup name, when I look at my code, I see that there is a limit
    of 66 characters and that each component must not exceed 14 characters.

    I don't know where I got these limits, but one thing is certain, I
    didn't impose them on my own. I have to find where I got them ^^

    I think theses limits where found in an article by David W. Wright,
    "Guidelines on Usenet Newsgroup Names" (1999).

    To date it seems that components may be up to 14, 20 or 30 chars (even
    if 14 is recommended). I do not see the 66 chars size limit anymore.

    https://www.big-8.org/wiki/Naming_newsgroups#General_Guidelines

    or

    https://www.usenet.org.uk/newsgroup.names.html

    I will update theses limits in SNS, but they are only used when creating
    a local newsgroup.

    Have nice day,
    Franck

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Urs =?UTF-8?Q?Jan=C3=9Fen?=@21:1/5 to Franck on Thu Dec 22 11:01:01 2022
    Franck wrote:
    I totaly agree with the fact that a client MUST NOT issue MAXARTNUM more
    than once in a session.

    as auth requests may pop up "unexpected" like in

    S< 200 hello
    CAPABILITIES
    S< 101 Capability list:
    S< VERSION 2
    S< READER
    S< AUTHINFO USER SASL
    S< MAXARTNUM 140737488355327
    [...]
    S< .
    MAXARTNUM 140737488355327
    S< 202 ok
    [...]
    LIST COUNTS
    S< 380 authorization required
    AUTHINFO USER foo
    S< 381 pass
    AUTHINFO PASS bar
    S< 281 fine

    and authorization may alter the MAXARTNUM limit ...

    I'd expect the client to resend CAPABILITIES and MAXARTNUM here (at least if the servers limit did change after auth).

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Franck@21:1/5 to All on Thu Dec 22 12:47:11 2022
    Hello,

    S< 380 authorization required

    480? :-)

    AUTHINFO USER foo
    S< 381 pass
    AUTHINFO PASS bar
    S< 281 fine

    and authorization may alter the MAXARTNUM limit ...

    I'd expect the client to resend CAPABILITIES and MAXARTNUM here (at least if the servers limit did change after auth).
    For SNS, the MAXARTNUM is global, so it won't change after AUTH.

    But if I'm not wrong, a 480 error indicates that a resource is
    unavailable at the moment but may become available after authentication.

    So perhalps, for other implementations, it can be usefull to reset the
    state of MAXARTNUM, thus showing it again in capabilities?

    Franck

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Richard Kettlewell@21:1/5 to iulius@nom-de-mon-site.com.invalid on Thu Dec 22 12:57:16 2022
    Julien ÉLIE <iulius@nom-de-mon-site.com.invalid> writes:

    Hi Richard,

    It is RECOMMENDED that server implementations permit configuring the
    maximum article number implicitly set at the start of an NNTP session,
    whose default MUST be 2,147,483,647 when version 2 of NNTP is
    advertised in CAPABILITIES (Section 3.3.2 of [RFC3977]). The default
    value MAY be different in future versions of NNTP. This implicit
    maximum article number MAY be global to the news server or
    configurable at a finer level. For instance, the maximum article
    number could be implicitly set to 18,446,744,073,709,551,615 (2^64-1)
    for certain users either at the start of the NNTP session or after the
    use of a mechanism identifying them (e.g. authentication with
    AUTHINFO). Configuring a value greater than this default value SHOULD
    be done only for users known to run a news client capable of handling
    article numbers up to that configured value.
    This paragraph seems to add scope for incompatibility between
    servers
    and clients without any clear benefit.

    I propose that paragraph because there's a need not to break existing implementations capable of handling 64-bit article numbers (both
    servers and clients).
    If a server implements MAXARTNUM, a client which still has not would
    suddenly be no longer able to participate in newsgroups where he
    previously could!

    So here's the need I wish to deal with. In the same way MAXARTNUM is designed not to break existing legacy 32-bit implementations, it
    shouldn't break existing 64-bit implementations which went ahead and
    are using large article numbers.
    I came up with that idea formulated in the above paragraph. Note that
    a server that only implements a global parameter for the initial
    maximum article number (set to 2^31-1) is compliant!

    Do you happen to have another better solution for that need?

    I do see the problem.

    One answer is to say that those servers are out of spec and can continue
    being out of spec, including the MAXARTNUM spec. Given the speed of
    development and deployment in Usenet that’s probably what will happen
    for a while anyway l-) but it’s not a very satisfactory answer.

    Anyway, I think we shouldn’t impose unnecessary requirements (the RECOMMENDED) on other servers.

    My suggestion therefore would be something like:

    A server implementation MAY configure a maximum article number for a
    connection based on other techniques, for example client
    fingerprinting, per-user configuration or server
    configuration. Configuring a value greater than the default of
    2147483647 SHOULD be done only for clients known to be capable of
    handling article numbers up to that configured maximum article number.

    A server supporting the MAXARTNUM command as defined in this document
    will advertise the "MAXARTNUM" capability label in response to the
    CAPABILITIES command (Section 5.2 of [RFC3977]). However, this
    capability MUST NOT be advertised once the MAXARTNUM command has
    successfully been executed.
    Why not? Seems like an entirely unnecessary restriction on the
    server.

    The problem I see in going on advertising MAXARTNUM is CAPABILITIES is
    that it may be confusing.
    The server could advertise "MAXARTNUM N" whereas a value different
    than N has been negotiated.

    IMO that isn’t very likely, but the inconsistency could be explicitly forbidden if it’s a concern.

    And usually, after the use of commands like AUTHINFO, COMPRESS,
    STARTTLS which change something in the connection, they are no longer advertised in CAPABILITIES.

    Do you see a drawback in no longer advertising it either?

    My suggestion would be to forbid the server from changing its MAXARTNUM advertisement in CAPABILITIES. For example:

    A server that does not advertize MAXARTNUM in one CAPABILITIES
    response MUST NOT then advertizing it in a subsequent CAPABILITIES
    response in the same session without an intervening successful MODE
    READER.

    A server that advertizes MAXARTNUM in one CAPABILITIES response MUST
    NOT remove it, or advertize a different maximum article number, in a
    subsequent CAPABILITIES response in the same session without an
    intervening successful MODE READER.

    Nonetheless, your remark about such large article numbers makes me
    think about something I overlooked...

    From RFC 3977:

    Command lines MUST NOT exceed
    512 octets, which includes the terminating CRLF pair. The arguments
    MUST NOT exceed 497 octets. A server MAY relax these limits for
    commands defined in an extension.

    The initial line of the response MUST NOT exceed
    512 octets, which includes the response code and the terminating CRLF
    pair; an extension MAY specify a greater maximum for commands that it
    defines, but not for any other command.

    So there is a problem for instance with "OVER N-M" where N and M are compounded of 497 digits.
    And also in existing responses, as we may have a large article number followed by a Message-ID up to 250 characters itself!
    And even in GROUP responses, there may be 3 large article numbers
    followed by a newsgroup name. (Is there an upper bound for the length
    of a newsgroup name? I do not manage to find it in RFC 3977 or 5536.)

    Something should definitively be said about that in the MAXARTNUM extension...

    Should we then restrict the maximum length of article numbers to a
    precise number, given these requirements?
    For version 2 of NNTP, we may for instance take 40 bytes, which is
    enough for 2^128 articles. And just say a future version may relax
    the current limits in the length of command lines and initial response
    lines.

    Do you see any other idea to deal with that issue?

    Article numbers of 2^63 aren’t realistic. Even at a thousand articles
    per second it would take hundreds of millions of years to reach such a
    large article number. For 128-bit article numbers we’re talking 10^27
    years.

    So I don’t think we need to worry about these very large
    numbers. They’re not happening.

    Additionally, the client MUST NOT issue a MODE READER command after a
    successful negotiation of the mutual maximum article number with the
    MAXARTNUM command, and a server MUST NOT advertise the MODE-READER
    capability.
    The former I can see, but why forbid advertizing MODE-READER after
    MAXARTNUM has been negotiated?

    When MODE-READER is advertised, we're in the case of a mode-switching
    news server. The same way AUTHINFO, COMPRESS, or STARTTLS forbid
    advertizing MODE-READER after a successful use of the command,
    MAXARTNUM also has to. Negotiated states are not shared between
    transit and reading facilities of mode-switching news servers.

    Fair enough.

    A server MUST NOT return the MAXARTNUM capability label in response to
    a CAPABILITIES command received after a successful use of the
    MAXARTNUM command, and a server MUST reply with a 502 response code if
    a syntactically valid MAXARTNUM command is received after an already
    successful use of this command.
    Why?

    From previous discussions, it seemed weird that a news client used
    MAXARTNUM twice (or more) with a possible different article number
    asked. That's the reason of that paragraph.

    It doesn’t seem likely in practice (why would a client bother?) and it wouldn’t be hard for a server to cope with it (mostly it would just be updating a field in the session state, after all). However if seems
    plausible that a server might not cope well with the maximum being
    adjusted more than once, we could permit servers to reject second and subsequent MAXARTNUM commands:

    Once a MAXARTNUM command has succeeded, a server MAY reply 502 to all
    subsequent MAXARTNUM commands.

    --
    https://www.greenend.org.uk/rjk/

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Michael =?ISO-8859-1?Q?B=E4uerle?=@21:1/5 to Franck on Thu Dec 22 13:26:17 2022
    Franck wrote:
    Urs Janßen wrote:

    [...]
    AUTHINFO USER foo
    S< 381 pass
    AUTHINFO PASS bar
    S< 281 fine

    and authorization may alter the MAXARTNUM limit ...

    I'd expect the client to resend CAPABILITIES and MAXARTNUM here (at least if
    the servers limit did change after auth).

    For SNS, the MAXARTNUM is global, so it won't change after AUTH.

    But if I'm not wrong, a 480 error indicates that a resource is
    unavailable at the moment but may become available after authentication.

    So perhalps, for other implementations, it can be usefull to reset the
    state of MAXARTNUM, thus showing it again in capabilities?

    How does such an implementation would look like?

    Some frontend for authentication, that (if successful) delegates the
    connection to a backend -- that has no access to the state of the
    frontend?

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Franck@21:1/5 to All on Thu Dec 22 14:43:01 2022
    Hello Michaël & Urs,

    How does such an implementation would look like?
    After looking at my code and because I was sure to have taken in account
    with Julien some actions about authentication, the reply is in the first article of this thread (the proposal for a specification) and also in my article <D9E85xtkDAg@news.spitfire-nntp.fr>.

    Specification say :

    "Otherwise, if this extension has already been successfully used, a 502 response code MUST be returned. If authentication is possible and
    needed BEFORE A POSSIBLE USE OF THIS EXTENSION, a 480 response code MUST
    be returned."

    So, in the session proposed by Urs, the client will receive a 480 just
    after the first use of MAXARTNUM.

    Also to be more precise of what takes place on my server that you can't
    see because it's "open" :

    - If server is set to "RequiresAuth", MAXARTNUM is not available in CAPABILITIES until client is authenticated.

    - If server is not set to "RequiresAuth" and "anonymous" accessgroup
    can't use MAXARTNUM, client receive a 480 (perhalps other accesgroups can).

    And as my words can be confusing when I wrote that MAXARTNUM is global
    in SNS, I prefer to correct myself: MAXARTNUM is accesgroup based but
    the value can't be set to differents ones for differents accesgroups.
    It's global, accesgroup can or can't use the MAXARTNUM command.

    Have nice day,
    Franck

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?Julien_=c3=89LIE?=@21:1/5 to All on Thu Dec 22 18:47:34 2022
    Salut Franck,

    For newsgroup name, when I look at my code, I see that there is a
    limit of 66 characters and that each component must not exceed 14
    characters.

    I don't know where I got these limits, but one thing is certain, I
    didn't impose them on my own. I have to find where I got them ^^

    I think theses limits where found in an article by David W. Wright, "Guidelines on Usenet Newsgroup Names" (1999).

    To date it seems that components may be up to 14, 20 or 30 chars (even
    if 14 is recommended). I do not see the 66 chars size limit anymore.

    INN currently does not enforce a maximum length on either a manual
    creation by the news admin or from newgroup control messages.

    Only the number of components is checked and is limited to 9 for
    newgroup control messages treated by INN.
    control-archive (for the ftp.isc.org active and newsgroups file)
    probably also enforces some rules.

    As for limits, there no longer seems to be any in current RFCs.

    RFC 1849 (Son of 1036) enforces one for the components, but was
    obsoleted by RFC 5536, so the following paragraphs *no* longer apply:

    A plain component MUST contain at least one letter, MUST begin
    with a letter or digit, and MUST NOT be longer than 14 characters.

    NOTE: Much existing software converts the newsgroup name into a
    directory path and stores the articles themselves using numeric
    filenames, so all-digit name components can be troublesome; the
    "Great Renaming" early in the history of Usenet included revisions
    of several newsgroup names to eliminate such components.

    NOTE: The same storage technique is the reason for the
    14-character limit. The limit is now largely historical, since
    most modern systems have much larger limits on the length of a
    directory entry's name, but many old systems are still in use.
    Systems with shorter limits also exist, but news software on such
    systems has had to deal with the problem already, since there are
    several widespread newsgroups with 14-character components in
    their names. Implementors are warned that it is intended that the
    successor to this Draft will increase the 14-character limit, and
    they are urged to fix their software to handle longer names
    gracefully (if such fixes are necessary, given the intended domain
    of application of the particular software).



    Also note the USEAGE Internet-Draft which enforces a limit of 30 chars
    for a component, and 66 for a newsgroup name:
    https://www.ietf.org/archive/id/draft-ietf-usefor-useage-01.txt

    3.1.1.5. Newsgroups

    There are restrictions on the length of components of <newsgroup-
    name>s, and on the <newsgroup-name>s themselves, as described more
    fully in 7.2. Posting and injecting agents MAY attempt to enforce
    them but, because of the possibility that hierarchy policies or
    future standards may relax them, it SHOULD be possible for posters to
    override such checks, and software MUST be so written that they can
    be disabled altogether.

    7.2. Naming of Newsgroups

    A <component> is limited to 30 <component-char>s and a
    <newsgroup-name> to 66 <component-char>s (counting also the '.'s
    separating the <component>s).

    NOTE: Whilst there is no longer any technical reason to limit
    the length of a <component> (formerly, it was limited to 14
    octets) nor of a <newsgroup-name>, it should be noted that these
    names are also used in the <newsgroups-line> where another
    overall policy limit applies (7.1) and, moreover, excessively
    long names can be exceedingly inconvenient in practical use. The
    66 limit on <newsgroup-name>s ensures that a Followup-To Header
    with such a name will still fit within 79 characters overall.

    7.1. Control Messages

    Although, in accordance with [RFC 2822], a <newsgroups-line> (as
    found in both 'newgroup' and 'checkgroups' messages) could have a
    maximum length of 998 octets, as a matter of policy a far lower
    limit, expressed in characters, SHOULD be set. The current convention
    is to limit its length so that the <newsgroup-name>, the HTAB(s)
    (interpreted as 8-character tabs that takes one at least to column
    24) and the <newsgroup-description> (excluding any <moderation-flag>)
    fit into 79 characters. This document does not seek to enforce any
    such rule, but any decision to extend it should be made as a specific
    decision for the hierarchy. Reading agents SHOULD therefore enable a
    <newsgroups-line> of any length to be displayed, e.g. by wrapping it
    as required.

    --
    Julien ÉLIE

    « La moitié des hommes politiques sont des bons à rien. Les autres sont
    prêts à tout. » (Coluche)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?Julien_=c3=89LIE?=@21:1/5 to All on Thu Dec 22 18:59:16 2022
    Hi Urs,

    I totaly agree with the fact that a client MUST NOT issue MAXARTNUM more
    than once in a session.

    as auth requests may pop up "unexpected" like in

    S< 200 hello
    CAPABILITIES
    S< 101 Capability list:
    S< VERSION 2
    S< READER
    S< AUTHINFO USER SASL
    S< MAXARTNUM 140737488355327
    [...]
    S< .
    MAXARTNUM 140737488355327
    S< 202 ok
    [...]
    LIST COUNTS
    S< 380 authorization required
    AUTHINFO USER foo
    S< 381 pass
    AUTHINFO PASS bar
    S< 281 fine

    and authorization may alter the MAXARTNUM limit ...

    I'd expect the client to resend CAPABILITIES and MAXARTNUM here (at least if the servers limit did change after auth).

    The MAXARTNUM limit is not supposed to change before and after the use
    of AUTHINFO. I should probably explicitly say it.
    The news server advertises the maximum article number it can handle.

    The news client uses MAXARTNUM to inform the news server of the maximum
    article number it could send.

    Both these maximum are not expected to change during an NNTP session. Authentication MAY be needed before using MAXARTNUM if the news server
    is configured to, but on no account should it advertise a different
    maximum article number.

    The part in the specification about authentication is for a possible *implicitly negotiated* different maximum article number, without using MAXARTNUM. For instance to say that Joe can see 2^64-1 articles because
    he is using a news client I know is capable of handling them but
    unfortunately does not implement MAXARTNUM to say it.


    Does it sound good to you with these explanations?

    --
    Julien ÉLIE

    « Ibi etsi uis te non esse sed es ibi. »

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?Julien_=c3=89LIE?=@21:1/5 to That was my initial thought before on Thu Dec 22 19:49:55 2022
    Hi Richard,

    So here's the need I wish to deal with. In the same way MAXARTNUM is
    designed not to break existing legacy 32-bit implementations, it
    shouldn't break existing 64-bit implementations which went ahead and
    are using large article numbers.
    I came up with that idea formulated in the above paragraph. Note that
    a server that only implements a global parameter for the initial
    maximum article number (set to 2^31-1) is compliant!

    Do you happen to have another better solution for that need?

    I do see the problem.

    One answer is to say that those servers are out of spec and can continue being out of spec, including the MAXARTNUM spec.

    :-)
    That was my initial thought before I made up my mind and wrote the "recommendation".


    Anyway, I think we shouldn’t impose unnecessary requirements (the RECOMMENDED) on other servers.

    Ah, OK, the point was for the RECOMMENDED.
    I agree it is too strong.


    My suggestion therefore would be something like:

    A server implementation MAY configure a maximum article number for a
    connection based on other techniques, for example client
    fingerprinting, per-user configuration or server
    configuration. Configuring a value greater than the default of
    2147483647 SHOULD be done only for clients known to be capable of
    handling article numbers up to that configured maximum article number.

    Yes, MAY is better.
    I'll take into account your proposal.


    A server supporting the MAXARTNUM command as defined in this document
    will advertise the "MAXARTNUM" capability label in response to the
    CAPABILITIES command (Section 5.2 of [RFC3977]). However, this
    capability MUST NOT be advertised once the MAXARTNUM command has
    successfully been executed.
    Why not? Seems like an entirely unnecessary restriction on the
    server.

    The problem I see in going on advertising MAXARTNUM is CAPABILITIES is
    that it may be confusing.
    The server could advertise "MAXARTNUM N" whereas a value different
    than N has been negotiated.

    IMO that isn’t very likely, but the inconsistency could be explicitly forbidden if it’s a concern.

    I'll mention the reason for the restriction. This way, it will be
    clearer for a future reader.


    And usually, after the use of commands like AUTHINFO, COMPRESS,
    STARTTLS which change something in the connection, they are no longer
    advertised in CAPABILITIES.

    Do you see a drawback in no longer advertising it either?

    My suggestion would be to forbid the server from changing its MAXARTNUM advertisement in CAPABILITIES. For example:

    A server that does not advertize MAXARTNUM in one CAPABILITIES
    response MUST NOT then advertizing it in a subsequent CAPABILITIES
    response in the same session without an intervening successful MODE
    READER.

    A server that advertizes MAXARTNUM in one CAPABILITIES response MUST
    NOT remove it, or advertize a different maximum article number, in a
    subsequent CAPABILITIES response in the same session without an
    intervening successful MODE READER.

    It could be done this way, indeed, with the MAXARTNUM capability still advertised after the use of the command.
    The drawback I see is that it may be incoherent with the possibility for
    a client to use the command. Normally, CAPABILITIES shows what the
    client can use at a given moment; that's why the initial proposition was
    to hide it when the MAXARTNUM command can no longer be sent.


    Once a MAXARTNUM command has succeeded, a server MAY reply 502 to
    all subsequent MAXARTNUM commands.
    The problem with MAY is that it leaves a choice. Both the developers of
    news clients and news servers will have to decide what to do. Whereas
    the MUST NOT use twice the command is pretty clear, and everyone knows
    what to do. Doesn't it seem simpler?
    I do not see why we should allow a change of mutual maximum article,
    once negotiated. (In the same way you don't change the negotiated
    compression method or TLS cipher in the middle of an NNTP session.)


    Command lines MUST NOT exceed
    512 octets, which includes the terminating CRLF pair. The arguments
    MUST NOT exceed 497 octets. A server MAY relax these limits for
    commands defined in an extension.

    The initial line of the response MUST NOT exceed
    512 octets, which includes the response code and the terminating CRLF
    pair; an extension MAY specify a greater maximum for commands that it
    defines, but not for any other command.

    So there is a problem for instance with "OVER N-M" where N and M are
    compounded of 497 digits.
    And also in existing responses, as we may have a large article number
    followed by a Message-ID up to 250 characters itself!
    And even in GROUP responses, there may be 3 large article numbers
    followed by a newsgroup name. (Is there an upper bound for the length
    of a newsgroup name? I do not manage to find it in RFC 3977 or 5536.)

    Something should definitively be said about that in the MAXARTNUM
    extension...

    Should we then restrict the maximum length of article numbers to a
    precise number, given these requirements?
    For version 2 of NNTP, we may for instance take 40 bytes, which is
    enough for 2^128 articles. And just say a future version may relax
    the current limits in the length of command lines and initial response
    lines.

    Do you see any other idea to deal with that issue?

    Article numbers of 2^63 aren’t realistic. Even at a thousand articles
    per second it would take hundreds of millions of years to reach such a
    large article number. For 128-bit article numbers we’re talking 10^27 years.

    So I don’t think we need to worry about these very large
    numbers. They’re not happening.

    So we may just say that the maximum length of an article number is
    increased by this extension from 16 to 45 (including possible 6 leading
    zeroes) in version 2 of NNTP because of other incompressible constraints
    on the length of command lines and initial response lines.

    I believe something needs being said about the maximum allowed, so as
    not to be vague. And better put a limit corresponding to the "next
    step" (2^128) instead of the one we consider in 2022 (2^64).

    This way, the problem is closed for version 2 of the protocol. As you
    say, it should normally be enough for several years or maybe forever for
    the needs of NNTP. (As a side note, future 128-bit processors will
    probably treat "long long" as 128-bit integers, and existing
    implementations built on such architectures may advertise that length...
    I don't know how current 256-bit GPU treat "long long".)

    --
    Julien ÉLIE

    « – Mais lâchez-moi ! Vous voyez bien que je suis déjà repoussé !
    – Il y en a un qui est entré !
    – Je ressors ! Je ressors ! Je ressors ! » (Astérix)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Franck@21:1/5 to All on Thu Dec 22 22:00:52 2022
    Salut Julien,


    For the moment :

    The MAXARTNUM limit is not supposed to change before and after the use
    of AUTHINFO.  I should probably explicitly say it.

    Implemented like that.

    The news server advertises the maximum article number it can handle.

    MAXARTNUM is flagged "Need authentication" (In a way similar to CMDtable
    in nnrpd.c in INN), so if server require authentication to use the
    command, MAXARTNUM is not advertised in CAPABILITIES and a 480 is sent.

    Once authenticated, MAXARTNUM is advertised in CAPABILITIES.
    Once authenticated, if accessgroup doesn't grant access to MAXARTNUM
    command, client receive a "502 command MAXARTNUM disabled by administrator".

    When mutual maximum is fixed by the client, MAXARTNUM is no more
    advertided as MAXARTNUM MUST NOT be used more than once in a session.

    The news client uses MAXARTNUM to inform the news server of the maximum article number it could send.

    Both these maximum are not expected to change during an NNTP session. Authentication MAY be needed before using MAXARTNUM if the news server
    is configured to, but on no account should it advertise a different
    maximum article number.

    Implemented like that.

    The part in the specification about authentication is for a possible *implicitly negotiated* different maximum article number, without using MAXARTNUM.  For instance to say that Joe can see 2^64-1 articles because
    he is using a news client I know is capable of handling them but unfortunately does not implement MAXARTNUM to say it.

    Not implemented.
    This part is why I erroneous said MAXARTNUM is global in SNS.

    MAXARTNUM is accessgroup based but accessgroups doesn't works in the
    same way as in INN (They are far less flexible).

    So, for the moment, differents MAXARTNUM values can't be implicitly
    fixed, MAXARTNUM is just a global option : Users in accessgroup can or
    can't use MAXARTNUM.

    Does it sound good to you with these explanations?

    I think but let me know if something seems weird in what is already
    implemented and need to be changed.

    Franck

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Richard Kettlewell@21:1/5 to iulius@nom-de-mon-site.com.invalid on Thu Dec 22 21:14:15 2022
    Julien ÉLIE <iulius@nom-de-mon-site.com.invalid> writes:
    I believe something needs being said about the maximum allowed, so as
    not to be vague.

    Agreed...

    And better put a limit corresponding to the "next
    step" (2^128) instead of the one we consider in 2022 (2^64).

    ...but I’m not so sure that a limit around 2^128 is right.

    This way, the problem is closed for version 2 of the protocol. As you
    say, it should normally be enough for several years or maybe forever
    for the needs of NNTP. (As a side note, future 128-bit processors
    will probably treat "long long" as 128-bit integers, and existing implementations built on such architectures may advertise that
    length... I don't know how current 256-bit GPU treat "long long".)

    I’d note that there is a practical advantage to a 63-bit (or 64-bit)
    limit: parsing integers up to 64 bits is straightforward in current C implementations[1], while parsing larger integers usually involves
    stepping outside the built-in types and using some kind of large integer representation.

    If article numbers above 64 bits were realistic then this consideration
    would be irrelevant. But they aren’t; as such I don’t think we should
    make life harder for implementors to handle an implausible contingency.

    [1] strtoull has existed since C99; even Microsoft support it now.

    --
    https://www.greenend.org.uk/rjk/

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?Julien_=c3=89LIE?=@21:1/5 to All on Thu Dec 22 22:43:58 2022
    Hi Richard,

    I believe something needs being said about the maximum allowed, so as
    not to be vague.

    Agreed...

    And better put a limit corresponding to the "next
    step" (2^128) instead of the one we consider in 2022 (2^64).

    ...but I’m not so sure that a limit around 2^128 is right.

    There won't be any "right" value :-(



    This way, the problem is closed for version 2 of the protocol. As you
    say, it should normally be enough for several years or maybe forever
    for the needs of NNTP. (As a side note, future 128-bit processors
    will probably treat "long long" as 128-bit integers, and existing
    implementations built on such architectures may advertise that
    length... I don't know how current 256-bit GPU treat "long long".)

    I’d note that there is a practical advantage to a 63-bit (or 64-bit)
    limit: parsing integers up to 64 bits is straightforward in current C implementations

    Even with, say, a limit of 2^128, you can also use strtoull() to easily
    parse numbers.
    In fact, an implementation needs checking the syntax.

    If a news client sends:

    MAXARTNUM 000001514649446464645978796145345048644068460496262

    You're not required to parse it as an integer.
    First you split "words" (separated by whitespace), check the number of arguments, and see whether they are correct. Is every character of the
    second argument a digit? Does it have less than XX digits?
    Is it in the range of numbers I can cope with?
    ...

    All of these questions cannot be answered in a straightforward way by strtoull(), can it?
    Won't it return ULLONG_MAX and you'll still have to compare the string
    to ULLONG_MAX (removing leading zeroes) to check it was not exactly
    ULLONG_MAX?

    If strtoull() can do all of that without any manual parsing, then
    there's no problem in having a 2^128 limit (or any other limit).
    If it can't, then anyway it would be the same with a 2^64 limit. You'll
    still have to validate somehow the input.


    Unless I am missing something?



    And it is the same thing if a news server adversises a very large number
    in MAXARTNUM capability.



    If article numbers above 64 bits were realistic then this consideration
    would be irrelevant. But they aren’t; as such I don’t think we should make life harder for implementors to handle an implausible contingency.

    Clive explicitly warned us in 2005: https://lists.eyrie.org/pipermail/ietf-nntp/2005-July/005802.html

    Firstly, let us learn from the past and *not* call this - or make
    it - a "64 bit" facility.


    We don't know what the future will be...

    (And, yes, I am also annoyed by 2^128 or any other value... the problem
    is that a limit is required in version 2 of NNTP, so better take the
    maximum we can have. 2^256 with 78 digits may be possible; but 2^512
    with 155 digits, definitively not, because a GROUP command with 3
    numbers and a newsgroup name may not fit with such extremely large numbers.)

    --
    Julien ÉLIE

    « Le temps change toute chose ; il n'y a aucune raison pour que la
    langue échappe à cette loi universelle. » (Saussure)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Franck@21:1/5 to All on Fri Dec 23 07:11:33 2022
    Hello Julien,

    Does it sound good to you with these explanations?

    And I forgot to say that if no article number is > 2^31-1 on the server, MAXARTNUM is NOT advertised in CAPABILITIES and the command is not
    recognized by the server nor displayed in HELP.

    Have nice day,
    Franck

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Urs =?UTF-8?Q?Jan=C3=9Fen?=@21:1/5 to All on Fri Dec 23 11:34:16 2022
    Julien ÉLIE wrote:
    Does it sound good to you with these explanations?

    yes

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Adam H. Kerman@21:1/5 to Julien on Fri Dec 23 17:04:36 2022
    Julien wrote:

    I would like to submit you this first draft about MAXARTNUM, as recently >discussed.
    Comments welcome :-)

    I have no contribution to make. I did want to comment that I was
    enjoying observing this review process take place publicly here on
    Usenet. It appears that Julien has gotten useful feedback on his
    proposal a whole lot more quickly than certain other review processes I
    shall not name.

    Usenet remains of great value even if ignored by kids today.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Richard Kettlewell@21:1/5 to iulius@nom-de-mon-site.com.invalid on Fri Dec 23 21:50:55 2022
    Julien ÉLIE <iulius@nom-de-mon-site.com.invalid> writes:
    I believe something needs being said about the maximum allowed, so as
    not to be vague.
    Agreed...

    And better put a limit corresponding to the "next
    step" (2^128) instead of the one we consider in 2022 (2^64).
    ...but I’m not so sure that a limit around 2^128 is right.

    There won't be any "right" value :-(

    Honestly, 64 bits is plenty. This isn’t a ‘640k is enough for
    anyone’. There will never be a newsgroup with 2^64 articles in it. See
    my earlier post.

    [strtoull]
    Unless I am missing something?

    No, I think you’re right.

    If article numbers above 64 bits were realistic then this consideration
    would be irrelevant. But they aren’t; as such I don’t think we should
    make life harder for implementors to handle an implausible contingency.

    Clive explicitly warned us in 2005: https://lists.eyrie.org/pipermail/ietf-nntp/2005-July/005802.html

    Firstly, let us learn from the past and *not* call this - or make
    it - a "64 bit" facility.


    We don't know what the future will be...

    In this case I think we can make an adequate prediction.

    --
    https://www.greenend.org.uk/rjk/

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?Julien_=c3=89LIE?=@21:1/5 to All on Sat Dec 24 13:13:53 2022
    Hi Richard,

    I believe something needs being said about the maximum allowed, so as
    not to be vague.
    Agreed...

    OK, we both agree to define a limit.


    Honestly, 64 bits is plenty. This isn’t a ‘640k is enough for
    anyone’. There will never be a newsgroup with 2^64 articles in it. See
    my earlier post.

    Yes, I understand.
    On second thoughts, I think Clive's point was to have a scalable
    capability. That is to say, not define a simple BIGNUM command without argument saying that the implementation handles 64-bit article numbers,
    but a command with an explicit article number.

    This way, I agree that 2^64 is a good choice for the limit to define in
    the MAXARTNUM extension because:
    - as you say, it is a tremendously high number for the usage of Usenet
    or Netnews in general;
    - it is unlikely that there are existing implementations which can
    already cope with higher article numbers, and that won't probably be the
    cases for years;
    - NNTP version 2 enforces limits in the length of command lines and
    initial response lines, so we cannot define for now a command allowing
    article numbers of an arbitrary size;
    - and if after all it appears that Netnews needs higher article numbers
    than 2^64-1, then the MAXARTNUM command is already prepared to handle
    the case - to be proper, we would then have to standardize an official
    NNTP version 3 with a few adjustements to allow greater limits for
    commands and responses, and possible other stuff to fit the usage of
    Netnews at that future time, and integrate MAXARTNUM in the core v3 spec.

    Sounds good to you?


    We don't know what the future will be...

    In this case I think we can make an adequate prediction.

    :)


    I'll update the proposal in January with all we came up with during this discussion.
    Thanks again for your valuable input.

    Happy Christmas all!

    --
    Julien ÉLIE

    « O fortunatos nimium, sua si bona norint, agricolas. » (Virgile)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Richard Kettlewell@21:1/5 to iulius@nom-de-mon-site.com.invalid on Sat Dec 24 15:19:34 2022
    Julien ÉLIE <iulius@nom-de-mon-site.com.invalid> writes:
    I believe something needs being said about the maximum allowed, so as >>>>> not to be vague.
    Agreed...

    OK, we both agree to define a limit.


    Honestly, 64 bits is plenty. This isn’t a ‘640k is enough for
    anyone’. There will never be a newsgroup with 2^64 articles in it. See
    my earlier post.

    Yes, I understand.
    On second thoughts, I think Clive's point was to have a scalable
    capability. That is to say, not define a simple BIGNUM command
    without argument saying that the implementation handles 64-bit article numbers, but a command with an explicit article number.

    Possibly.

    This way, I agree that 2^64 is a good choice for the limit to define
    in the MAXARTNUM extension because:

    Just to be pedantic - 2^64-1 as the maximum, so that it unambiguously
    fits in a 64-bit word.

    - as you say, it is a tremendously high number for the usage of Usenet
    or Netnews in general;
    - it is unlikely that there are existing implementations which can
    already cope with higher article numbers, and that won't probably be
    the cases for years;
    - NNTP version 2 enforces limits in the length of command lines and
    initial response lines, so we cannot define for now a command
    allowing article numbers of an arbitrary size;
    - and if after all it appears that Netnews needs higher article
    numbers than 2^64-1, then the MAXARTNUM command is already prepared
    to handle the case - to be proper, we would then have to standardize
    an official NNTP version 3 with a few adjustements to allow greater
    limits for commands and responses, and possible other stuff to fit the
    usage of Netnews at that future time, and integrate MAXARTNUM in the
    core v3 spec.

    Sounds good to you?

    Yes.

    I'll update the proposal in January with all we came up with during
    this discussion.

    Thanks again for your valuable input.

    Happy Christmas all!

    And to you!

    --
    https://www.greenend.org.uk/rjk/

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Franck@21:1/5 to All on Sun Dec 25 07:19:48 2022
    Salut Julien et joyeux noël!

    This way, I agree that 2^64 is a good choice for the limit to define in
    the MAXARTNUM extension because:
    - as you say, it is a tremendously high number for the usage of Usenet
    or Netnews in general;

    I agree.

    - it is unlikely that there are existing implementations which can
    already cope with higher article numbers, and that won't probably be the cases for years;

    It would take me no more than one hour to make the changes to accept
    numeric values up to 38 significant digits.

    Besides being completely useless, I think 2^64-1 is more than enough for
    NNTP's needs.

    - NNTP version 2 enforces limits in the length of command lines and
    initial response lines, so we cannot define for now a command allowing article numbers of an arbitrary size;
    - and if after all it appears that Netnews needs higher article numbers
    than 2^64-1, then the MAXARTNUM command is already prepared to handle
    the case - to be proper, we would then have to standardize an official
    NNTP version 3 with a few adjustements to allow greater limits for
    commands and responses, and possible other stuff to fit the usage of
    Netnews at that future time, and integrate MAXARTNUM in the core v3 spec.

    It sounds like you are looking into the future and considering preparing
    INN to support large numbers, which is good news!

    So what we do? If I understand correctly, we are removing MAXARTNUM 0?

    Have nice day,
    Franck

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?Julien_=c3=89LIE?=@21:1/5 to All on Sun Dec 25 09:43:53 2022
    Salut Franck,

    It sounds like you are looking into the future and considering preparing
    INN to support large numbers, which is good news!

    It has been in the wishlist for INN since a long time. It's quite a
    work though, to make it happen.
    The fact that INN does not support large article numbers is not a reason
    for not standardizing a way for other news servers to support them.

    Besides article numbers, current NNTP implementations will also have to
    make sure and test that the infamous 64-bit epoch time is correctly delt
    with. LIST ACTIVE.TIMES returns an epoch time.


    So what we do? If I understand correctly, we are removing MAXARTNUM 0?

    Exactly.

    Merry Christmas all!

    --
    Julien ÉLIE

    « Être øu ne pås être, telle est lå questiøn… » (Kerøzen)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Bo Lindbergh@21:1/5 to All on Mon Jan 2 10:12:04 2023
    The maximum article number is a capability of the client which is useful
    for the server to know. Is this the only such capability that could exist?
    If there are others, it might be better to instead define a general CLIENTCAP command with a MAXARTNUM sub-command.

    /Bo Lindbergh

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?Julien_=c3=89LIE?=@21:1/5 to All on Thu Jan 5 00:51:57 2023
    Hi Bo,

    The maximum article number is a capability of the client which is useful
    for the server to know. Is this the only such capability that could exist? If there are others, it might be better to instead define a general CLIENTCAP command with a MAXARTNUM sub-command.

    That's a good point.
    We could define a general ENABLE extension, like IMAP has (RFC 5161).
    They defined only 1 extension to start with (CONDSTORE), and afterwards
    only two up to now if I am not mistaken: METADATA (RFC 5464) and UTF8
    (RFC 6855).

    We could do something similar, indeed...
    I am unsure we'll end up with many capabilities, though. Maybe one for
    UTF-8 to advertise that a client cope with internationalized newsgroup
    names?
    I am unsure "enabling UTF-8" for a client is really needed for NNTP as
    RFC 3977 says clients SHOULD match newsgroups names octet by octet. A
    news server may already send non-ASCII newsgroup names, and I don't
    think it should not do that unless the client asks for it. But I may be
    wrong. Any point of view about that?

    From RFC 3977:

    o Although this specification allows UTF-8 for newsgroup names, they
    SHOULD be restricted to US-ASCII until a successor to RFC 1036
    [RFC1036] standardises another approach. 8-bit encodings SHOULD
    NOT be used because they are likely to cause interoperability
    problems.

    Until such specifications
    are published, implementations SHOULD match newsgroup names octet by
    octet. It is anticipated that any approved scheme will be applied
    "at the edges", and therefore octet-by-octet comparison will continue
    to apply to most, if not all, uses of newsgroup names in NNTP.



    If we're going to have only a very few, if any, more general "ENABLE"
    commands, is it worth the complexification of defining its parsing
    instead of just having top-level commands?

    ENABLE MAXARTNUM=18446744073709551615 UTF8=ACCEPT NULCHARS=ACCEPT MAXRESPLINES=UNLIMITED

    Just an example; apart from MAXARTNUM, the other keywords aren't
    probably the right way to extend the requirements of NNTP Version 2.

    --
    Julien ÉLIE

    « Il ne faut jamais parler sèchement à un Numide. » (Astérix)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Michael =?ISO-8859-1?Q?B=E4uerle?=@21:1/5 to All on Thu Jan 5 10:32:54 2023
    Julien ÉLIE wrote:

    [...]
    From RFC 3977:

    o Although this specification allows UTF-8 for newsgroup names, they
    SHOULD be restricted to US-ASCII until a successor to RFC 1036
    [RFC1036] standardises another approach. 8-bit encodings SHOULD
    NOT be used because they are likely to cause interoperability
    problems.

    Until such specifications
    are published, implementations SHOULD match newsgroup names octet by
    octet. It is anticipated that any approved scheme will be applied
    "at the edges", and therefore octet-by-octet comparison will continue
    to apply to most, if not all, uses of newsgroup names in NNTP.

    If we're going to have only a very few, if any, more general "ENABLE" commands, is it worth the complexification of defining its parsing
    instead of just having top-level commands?

    ENABLE MAXARTNUM=18446744073709551615 UTF8=ACCEPT NULCHARS=ACCEPT MAXRESPLINES=UNLIMITED

    There are potential problems if not all of the features are supported
    by the server. The command likely will be rejected with an error code,
    no feature will be enabled and the client doesn't know which subcommand
    was responsible for the problem. The client has to parse the response
    for information or test all features individually.
    In theory this should not happen, if the client has parsed CAPABILITIES carefully, but in real world there may be bugs somewhere.

    I would prefer simplicity, this means only one feature per command.
    But maybe the toplevel command "ENABLE", to enable/negotiate features,
    is useful to share the same return codes for all features.

    Like this it would work similar to LIST or COMPRESS:

    C CAPABILITIES
    S 101 Capability list:
    S VERSION 2
    S ENABLE MAXARTNUM UTF8 NULCHARS MAXRESPLINES
    S .

    C ENABLE MAXARTNUM=18446744073709551615
    S 2xx
    C ENABLE UTF-8=ACCEPT
    S 500
    C ENABLE NULCHARS=ACCEPT
    S 2xx
    C ENABLE MAXRESPLINES=UNLIMITED
    S 2xx

    This has to be done only once per session and the additional roundtrip
    times should be acceptable if the number of features is small.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Urs =?UTF-8?Q?Jan=C3=9Fen?=@21:1/5 to All on Thu Jan 5 11:17:15 2023
    In <AABjtplGZckAAAqY.A3.flnews@WStation7.micha.freeshell.org> on Thu, 05 Jan 2023 10:32:54,
    Michael Bäuerle wrote:
    I would prefer simplicity, this means only one feature per command.

    ACK

    S ENABLE MAXARTNUM UTF8 NULCHARS MAXRESPLINES
    S .

    but now the server does not tell the client it's MAXARTNUM limit anymore

    C ENABLE MAXARTNUM=18446744073709551615

    and that might be beyond the servers limit of e.g. 2^63-1 and thus
    leading to 4xx ...

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Michael =?ISO-8859-1?Q?B=E4uerle?=@21:1/5 to All on Thu Jan 5 14:02:39 2023
    Urs Janßen wrote:
    In <AABjtplGZckAAAqY.A3.flnews@WStation7.micha.freeshell.org> on Thu, 05 Jan 2023 10:32:54,
    Michael Bäuerle wrote:

    I would prefer simplicity, this means only one feature per command.

    ACK

    S ENABLE MAXARTNUM UTF8 NULCHARS MAXRESPLINES
    S .

    but now the server does not tell the client it's MAXARTNUM limit anymore

    There is a similar situation for AUTHINFO with SASL in CAPABILITIES:
    |
    | AUTHINFO USER SASL
    | SASL CRAM-MD5 PLAIN

    C ENABLE MAXARTNUM=18446744073709551615

    and that might be beyond the servers limit of e.g. 2^63-1 and thus
    leading to 4xx ...

    In our example it could look like:

    ENABLE MAXARTNUM UTF8 NULCHARS MAXRESPLINES
    MAXARTNUM 18446744073709551615

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Urs =?UTF-8?Q?Jan=C3=9Fen?=@21:1/5 to All on Thu Jan 5 13:59:10 2023
    In <AABjtspvADYAABbL.A3.flnews@WStation7.micha.freeshell.org> on Thu, 05 Jan 2023 14:02:39,
    Michael Bäuerle wrote:
    In our example it could look like:

    ENABLE MAXARTNUM UTF8 NULCHARS MAXRESPLINES
    MAXARTNUM 18446744073709551615

    but what's the point in listing MAXARTNUM in ENABLE when it is
    announced with its limit anyway (if available)?

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?Julien_=c3=89LIE?=@21:1/5 to All on Thu Jan 5 19:13:57 2023
    Hi Michael,

    There is a similar situation for AUTHINFO with SASL in CAPABILITIES:
    |
    | AUTHINFO USER SASL
    | SASL CRAM-MD5 PLAIN

    A separate capability list was needed because of:

    In agreement with [SASL], the server MUST continue to advertise the
    SASL capability in response to a CAPABILITIES command with the same
    list of SASL mechanisms that it did before authentication (thereby
    enabling the client to detect a possible active down-negotiation
    attack).

    Whereas AUTHINFO is no longer advertised after a successful authentication.


    In our example it could look like:

    ENABLE MAXARTNUM UTF8 NULCHARS MAXRESPLINES
    MAXARTNUM 18446744073709551615

    It could. And after a successful use of MAXARTNUM, the capability
    disappears in the ENABLE keywords, and the separate MAXARTNUM line.

    --
    Julien ÉLIE

    « Le tennis c'est comme le ping-pong, sauf qu'au tennis, les joueurs
    sont debout sur la table. »

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?Julien_=c3=89LIE?=@21:1/5 to All on Thu Jan 5 19:18:30 2023
    Hi Urs,

    ENABLE MAXARTNUM UTF8 NULCHARS MAXRESPLINES
    MAXARTNUM 18446744073709551615

    but what's the point in listing MAXARTNUM in ENABLE when it is
    announced with its limit anyway (if available)?

    Out of consistency, as it is a capability negotiated with the ENABLE
    command?
    Yet, I agree it is redundant, compared with "ENABLE MAXARTNUM=xxx".
    Both the approaches seem to make sense, though.

    --
    Julien ÉLIE

    « – Frappe quelqu'un !
    – Mais je ne suis fâché avec personne ! » (Astérix)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?Julien_=c3=89LIE?=@21:1/5 to All on Thu Jan 5 20:02:37 2023
    Hi Michael,

    If we're going to have only a very few, if any, more general "ENABLE"
    commands, is it worth the complexification of defining its parsing
    instead of just having top-level commands?

    ENABLE MAXARTNUM=18446744073709551615 UTF8=ACCEPT NULCHARS=ACCEPT
    MAXRESPLINES=UNLIMITED

    There are potential problems if not all of the features are supported
    by the server. The command likely will be rejected with an error code,
    no feature will be enabled and the client doesn't know which subcommand
    was responsible for the problem.

    The ENABLE command in IMAP does:

    - If the argument is not an extension known to the server, the server
    MUST ignore the argument.

    - If the argument is an extension known to the server, and it is not
    specifically permitted to be enabled using ENABLE, the server MUST
    ignore the argument. (Note that knowing about an extension doesn't
    necessarily imply supporting that extension.)

    - If the argument is an extension that is supported by the server and
    that needs to be enabled, the server MUST enable the extension for
    the duration of the connection. At present, this applies only to
    CONDSTORE ([RFC4551]). Note that once an extension is enabled,
    there is no way to disable it.

    The ENABLE command can be issued multiple times in a session. It is
    additive; i.e., "ENABLE a b", followed by "ENABLE c" is the same as a
    single command "ENABLE a b c".

    The server MUST NOT change the CAPABILITY list as a result of
    executing ENABLE; i.e., a CAPABILITY command issued right after an
    ENABLE command MUST list the same capabilities as a CAPABILITY
    command issued before the ENABLE command.


    For the last point about going on advertising the capability, I am
    unsure we should do the same for NNTP, if we use ENABLE. Once the
    maximum article number is negotiated, it cannot be changed a second
    time, so there's no need in advertising the capability.



    The client has to parse the response
    for information or test all features individually.
    In theory this should not happen, if the client has parsed CAPABILITIES carefully, but in real world there may be bugs somewhere.

    I agree that parsing ENABLE commands and results without error will be challenging...
    It is far more work than unique commands, one per feature.

    Capability list:
    [S] ENABLE MAXARTNUM=18446744073709551615 UTF8=NEWSGROUPS,HEADERS
    NULCHARS MAXRESPLINES=4096
    (or a variant on several lines with individual MAXARTNUM, UTF8 and
    MAXRESPLINES capabilities - not needed for NULCHARS)

    Command:
    [C] ENABLE MAXARTNUM=9223372036854775807 UTF8=NEWSGROUPS MAXRESPLINES=2048
    [S] 206 UTF8=NEWSGROUPS MAXARTNUM=9223372036854775807 MAXRESPLINES=2048

    Responding what has been enabled, without trailing comment.

    We may have instead of a 501 syntax error, not giving useful information:
    [C] ENABLE MAXARTNUM=badnumber UTF8=NONE MAXRESPLINES=2048
    [S] 506 MAXARTNUM UTF8
    (new error code to indicate syntax error in recognized commands, listed
    after 506, and MAXRESPLINES is not enabled)
    Or we can just use 501 without that complexity.

    A bit of work...



    I would prefer simplicity, this means only one feature per command.
    But maybe the toplevel command "ENABLE", to enable/negotiate features,
    is useful to share the same return codes for all features.

    Defining shared return codes, and enabling all features in only 1
    command, may be the only advantages for ENABLE.
    I don't know if that's worth the extra work in complexity of coding/debugging/getting it right.
    Notably if we end up only having MAXARTNUM as a feature to enable...



    C ENABLE UTF-8=ACCEPT
    S 500

    Might be 503 (feature not supported) in case the extension is unknown to
    the server. (500 is for the base command.)



    This has to be done only once per session and the additional roundtrip
    times should be acceptable if the number of features is small.

    We already have many roundtrips in sessions like this one, where the
    client sends CAPABILITIES several times:

    CAPABILITIES (gosh, MODE-READER is advertised)
    MODE READER
    CAPABILITIES (gosh, there is no argument to AUTHINFO)
    STARTTLS
    CAPABILITIES (yes, I can now authenticate)
    AUTHINFO USER
    AUTHINFO PASS
    CAPABILITIES (I now have more capabilities available)
    COMPRESS DEFLATE
    ENABLE MAXARTNUM=18446744073709551615

    And now after 10 commands (sic), I can send LIST ACTIVE & co.

    Using implicit TLS on port 563, which is the most wide-spread situation,
    we have 4 less commands.
    Seems reasonable after all to have separate commands (MAXARTNUM, and
    other possible future extensions) instead of ENABLE...

    --
    Julien ÉLIE

    « On appelle ça une insula. C'est une maison où les gens habitent les
    uns au-dessus des autres… » (Astérix)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Michael =?ISO-8859-1?Q?B=E4uerle?=@21:1/5 to All on Sat Jan 7 09:38:51 2023
    Julien ÉLIE wrote:
    Urs Janßen wrote:
    Michael Bäuerle wrote:

    ENABLE MAXARTNUM UTF8 NULCHARS MAXRESPLINES
    MAXARTNUM 18446744073709551615

    but what's the point in listing MAXARTNUM in ENABLE when it is
    announced with its limit anyway (if available)?

    Out of consistency, as it is a capability negotiated with the ENABLE
    command?

    Yes, because other commands use a similar scheme.
    But this does not mean that I would prefer the variant with ENABLE.

    Yet, I agree it is redundant, compared with "ENABLE MAXARTNUM=xxx".
    Both the approaches seem to make sense, though.

    Without ENABLE it is simpler to implement.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Michael =?ISO-8859-1?Q?B=E4uerle?=@21:1/5 to All on Sat Jan 7 10:10:32 2023
    Julien ÉLIE wrote:
    Michael Bäuerle wrote:
    Julien ÉLIE wrote:

    If we're going to have only a very few, if any, more general "ENABLE" commands, is it worth the complexification of defining its parsing instead of just having top-level commands?

    ENABLE MAXARTNUM=18446744073709551615 UTF8=ACCEPT NULCHARS=ACCEPT MAXRESPLINES=UNLIMITED

    There are potential problems if not all of the features are supported
    by the server. The command likely will be rejected with an error code,
    no feature will be enabled and the client doesn't know which subcommand
    was responsible for the problem.

    The ENABLE command in IMAP does:

    - If the argument is not an extension known to the server, the server
    MUST ignore the argument.

    - If the argument is an extension known to the server, and it is not
    specifically permitted to be enabled using ENABLE, the server MUST
    ignore the argument. (Note that knowing about an extension doesn't
    necessarily imply supporting that extension.)

    Anything unknown or not permitted is ignored.

    - If the argument is an extension that is supported by the server and
    that needs to be enabled, the server MUST enable the extension for
    the duration of the connection. At present, this applies only to
    CONDSTORE ([RFC4551]). Note that once an extension is enabled,
    there is no way to disable it.

    The ENABLE command can be issued multiple times in a session. It is
    additive; i.e., "ENABLE a b", followed by "ENABLE c" is the same as a
    single command "ENABLE a b c".

    Could be simplified by only allowing one extension per command:

    ENABLE a
    ENABLE b
    ENABLE c

    The server MUST NOT change the CAPABILITY list as a result of
    executing ENABLE; i.e., a CAPABILITY command issued right after an
    ENABLE command MUST list the same capabilities as a CAPABILITY
    command issued before the ENABLE command.


    For the last point about going on advertising the capability, I am
    unsure we should do the same for NNTP, if we use ENABLE. Once the
    maximum article number is negotiated, it cannot be changed a second
    time, so there's no need in advertising the capability.

    For IMAP this would match the case "not specifically permitted to be
    enabled using ENABLE" (should be ignored, if enabled again).

    The client has to parse the response
    for information or test all features individually.
    In theory this should not happen, if the client has parsed CAPABILITIES carefully, but in real world there may be bugs somewhere.

    I agree that parsing ENABLE commands and results without error will be challenging...
    It is far more work than unique commands, one per feature.

    Capability list:
    [S] ENABLE MAXARTNUM=18446744073709551615 UTF8=NEWSGROUPS,HEADERS
    NULCHARS MAXRESPLINES=4096
    (or a variant on several lines with individual MAXARTNUM, UTF8 and MAXRESPLINES capabilities - not needed for NULCHARS)

    Command:
    [C] ENABLE MAXARTNUM=9223372036854775807 UTF8=NEWSGROUPS MAXRESPLINES=2048 [S] 206 UTF8=NEWSGROUPS MAXARTNUM=9223372036854775807 MAXRESPLINES=2048

    Responding what has been enabled, without trailing comment.

    We may have instead of a 501 syntax error, not giving useful information:
    [C] ENABLE MAXARTNUM=badnumber UTF8=NONE MAXRESPLINES=2048
    [S] 506 MAXARTNUM UTF8
    (new error code to indicate syntax error in recognized commands, listed
    after 506, and MAXRESPLINES is not enabled)
    Or we can just use 501 without that complexity.

    A bit of work...

    Because adoption of new features is already so slow for Usenet, I think
    we should look for a simpler solution.

    I would prefer simplicity, this means only one feature per command.
    But maybe the toplevel command "ENABLE", to enable/negotiate features,
    is useful to share the same return codes for all features.

    Defining shared return codes, and enabling all features in only 1
    command, may be the only advantages for ENABLE.
    I don't know if that's worth the extra work in complexity of coding/debugging/getting it right.
    Notably if we end up only having MAXARTNUM as a feature to enable...

    Maybe for VERSION 3, if there would really be a lot of extensions.

    C ENABLE UTF-8=ACCEPT
    S 500

    Might be 503 (feature not supported) in case the extension is unknown
    to the server. (500 is for the base command.)

    Yes, 500 only if ENABLE itself would be not supported.

    This has to be done only once per session and the additional roundtrip times should be acceptable if the number of features is small.

    We already have many roundtrips in sessions like this one, where the
    client sends CAPABILITIES several times:

    CAPABILITIES (gosh, MODE-READER is advertised)
    MODE READER
    CAPABILITIES (gosh, there is no argument to AUTHINFO)
    STARTTLS
    CAPABILITIES (yes, I can now authenticate)
    AUTHINFO USER
    AUTHINFO PASS
    CAPABILITIES (I now have more capabilities available)
    COMPRESS DEFLATE
    ENABLE MAXARTNUM=18446744073709551615

    And now after 10 commands (sic), I can send LIST ACTIVE & co.

    Using implicit TLS on port 563, which is the most wide-spread
    situation, we have 4 less commands.

    For your server I already have 8 commands even with implicit TLS and
    without MODE-READER:

    CAPABILITIES
    AUTHINFO USER
    AUTHINFO PASS
    CAPABILITIES
    COMPRESS DEFLATE
    CAPABILITIES
    LIST OVERVIEW.FMT
    LIST DISTRIB.PATS

    Would be 9 commands with MAXARTNUM.

    The capability refresh after COMPRESS may be required because flnews
    supports immediate and 480-triggered authentication (the order listed
    above is not guaranteed).

    <https://www.rfc-editor.org/rfc/rfc8054#section-2.2.2>
    |
    | [...], a server MUST either list the AUTHINFO capability with
    | no arguments or not advertise it at all, in response to a
    | CAPABILITIES command received from an unauthenticated client after a
    | successful use of the COMPRESS command, [...]

    Seems reasonable after all to have separate commands (MAXARTNUM, and
    other possible future extensions) instead of ENABLE...

    Agreed.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?Julien_=c3=89LIE?=@21:1/5 to All on Sat Jan 7 11:33:30 2023
    Hi Michael,

    We may have instead of a 501 syntax error, not giving useful information:
    [C] ENABLE MAXARTNUM=badnumber UTF8=NONE MAXRESPLINES=2048
    [S] 506 MAXARTNUM UTF8
    (new error code to indicate syntax error in recognized commands, listed
    after 506, and MAXRESPLINES is not enabled)
    Or we can just use 501 without that complexity.

    A bit of work...

    Because adoption of new features is already so slow for Usenet, I think
    we should look for a simpler solution.

    Agreed.


    For your server I already have 8 commands even with implicit TLS and
    without MODE-READER:

    CAPABILITIES
    AUTHINFO USER
    AUTHINFO PASS
    CAPABILITIES
    COMPRESS DEFLATE
    CAPABILITIES
    LIST OVERVIEW.FMT
    LIST DISTRIB.PATS

    Would be 9 commands with MAXARTNUM.

    Oh yes, I had forgotten about these useful generic LIST requests.
    When and if flnews supports automatic display of the message of the day
    (when changed from a previous session), it will also be another LIST
    MOTD command to send...
    You may also have generic LIST SUBSCRIPTIONS (only on 1st connection on
    a news server probably) so it does not really count, and LIST HEADERS if
    you use HDR commands.

    Also wondering whether a news reader couldn't periodically check for new newsgroups and propose them to the user? (with an option like "check
    and advertise new newsgroups every XX days", which would refresh the
    list and propose new newsgroups from NEWGROUPS or LIST ACTIVE.TIMES
    commands, or maybe direct check in LIST ACTIVE because a news server may
    not maintain the information)


    The capability refresh after COMPRESS may be required because flnews
    supports immediate and 480-triggered authentication (the order listed
    above is not guaranteed).

    | [...], a server MUST either list the AUTHINFO capability with
    | no arguments or not advertise it at all, in response to a
    | CAPABILITIES command received from an unauthenticated client after a
    | successful use of the COMPRESS command, [...]

    OK, then it would be 9 commands if COMPRESS is sent before AUTHINFO.
    Otherwise, only 8 commands because sending CAPABILITIES after COMPRESS
    when the user is already authenticated is normally not useful, is it?

    --
    Julien ÉLIE

    « – Ben où est l'eau ?
    – Elle est sortie quand tu es entré Obélix. Il n'y a pas de place pour
    vous deux là dedans ! » (Astérix)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Michael =?ISO-8859-1?Q?B=E4uerle?=@21:1/5 to All on Sat Jan 7 14:03:18 2023
    Julien ÉLIE wrote:
    Michael Bäuerle wrote:

    [...]
    For your server I already have 8 commands even with implicit TLS and without MODE-READER:

    CAPABILITIES
    AUTHINFO USER
    AUTHINFO PASS
    CAPABILITIES
    COMPRESS DEFLATE
    CAPABILITIES
    LIST OVERVIEW.FMT
    LIST DISTRIB.PATS

    Would be 9 commands with MAXARTNUM.

    Oh yes, I had forgotten about these useful generic LIST requests.
    When and if flnews supports automatic display of the message of the day
    (when changed from a previous session), it will also be another LIST
    MOTD command to send...

    MOTD is supported, but only from the menu (must be explicitly requested
    by the user).

    You may also have generic LIST SUBSCRIPTIONS (only on 1st connection on
    a news server probably) so it does not really count,

    This is supported too.

    and LIST HEADERS if
    you use HDR commands.

    I have never used HDR.

    Also wondering whether a news reader couldn't periodically check for new newsgroups and propose them to the user? (with an option like "check
    and advertise new newsgroups every XX days", which would refresh the
    list and propose new newsgroups from NEWGROUPS or LIST ACTIVE.TIMES
    commands, or maybe direct check in LIST ACTIVE because a news server may
    not maintain the information)

    Sounds useful. But currently flnews does not store lists of "all"
    groups, neither former nor current.

    The group list is loaded again for the subscription window every time
    (via LIST in RFC 977 mode or LIST ACTIVE).

    Normally only the subscribed groups are used.

    The capability refresh after COMPRESS may be required because flnews supports immediate and 480-triggered authentication (the order listed
    above is not guaranteed).

    | [...], a server MUST either list the AUTHINFO capability with
    | no arguments or not advertise it at all, in response to a
    | CAPABILITIES command received from an unauthenticated client after a
    | successful use of the COMPRESS command, [...]

    OK, then it would be 9 commands if COMPRESS is sent before AUTHINFO. Otherwise, only 8 commands because sending CAPABILITIES after COMPRESS
    when the user is already authenticated is normally not useful, is it?

    The current logic is not smart enough for that and the refresh is always executed.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Franck@21:1/5 to All on Sat Jan 28 08:04:26 2023
    Hi all,

    After many changes to make the MAXARTNUM command compatible with the
    published specification and comments, it is now available again for
    everyone (including anonymous) on my server.

    On news.spitfire-nntp.fr:119, four groups with large numbers are
    available for testing purpose:

    maxartnum.test1
    maxartnum.test2
    maxartnum.test3 => This one is full
    maxartnum.test4

    Please let me know if something is wrong.
    Franck

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)