• STUMP, spam and Newsgroups headers

    From Owen Rees@21:1/5 to All on Mon Nov 1 19:49:02 2021
    A recent discussion in uk.net.news.moderation led me to investigate how
    STUMP handles incoming messages that do not have a Newsgroups header.

    I would expect that if the submission address for a moderated group gets
    onto email spam lists, the spambots will send email to that address but
    it is unlikely to include a Newsgroups header. I would hope that
    anything sent from a news server or a news client configured to send
    directly for moderated groups should include a Newsgroups header.

    Is anyone running STUMP (with or without Webstump) suffering from spam
    mailed directly to the submission address?

    That question may be difficult to answer because STUMP adds a Newsgroups
    header if it is missing. If the header is present but does not name the
    target group then STUMP adds the group. If all you see is the mail
    forwarded by STUMP then it may not be obvious that it was emailed
    directly without a Newsgroups header.

    I expect the idea was that you can post to a moderated group by sending
    an email to the submission address. Does any non-spammer ever do that?

    I have been playing around with my test instance of STUMP/Webstump which
    is based on the versions in use in the uk.* hierarchy to have STUMP
    record the bad header and pass that to Webstump where the administrator
    for the group can choose what to do with such messages. Since I do not
    actually run a moderated group I cannot do any live tests but anyone who
    is suffering from this problem is welcome to adopt/adapt what I have
    done. It can be found here:

    https://github.com/owenrees/stump/tree/handle-bad-header

    I think it should not be too difficult to adapt that to other versions.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Eli the Bearded@21:1/5 to orees@hotmail.com on Tue Nov 2 04:14:54 2021
    In news.admin.moderation, Owen Rees <orees@hotmail.com> wrote:
    I would expect that if the submission address for a moderated group gets
    onto email spam lists, the spambots will send email to that address but
    it is unlikely to include a Newsgroups header. I would hope that
    anything sent from a news server or a news client configured to send
    directly for moderated groups should include a Newsgroups header.

    I would think 100% of spam to the submission address will lack a
    Newgroups header. And I think 100% of posts submitted to news servers
    will have a Newsgroups header. I believe the RFC1036 authors would not
    have imagined _requiring_ separate email addresses for separate groups,
    so include the Newsgroups header with all posts. (Further, I bet 100%
    of the posts taking the news server route will have a Path header and
    none of the pure email ones will.)

    That question may be difficult to answer because STUMP adds a Newsgroups header if it is missing. If the header is present but does not name the target group then STUMP adds the group. If all you see is the mail
    forwarded by STUMP then it may not be obvious that it was emailed
    directly without a Newsgroups header.

    Very often I think you could guess based on the purported user-agent.
    A careful reading of the Received: headers would probably work, too.

    I expect the idea was that you can post to a moderated group by sending
    an email to the submission address. Does any non-spammer ever do that?

    Yes. Not often, but it happens. I've seen newsgroups in 2021 still
    advertising the submission address and I saw in the last year someone
    post to misc.legal.moderated saying they used that after having trouble
    posting regularly. (m.l.m seems to be quirky. I have moderated a group
    in the distant past[*], and _I've_ found posting there tricky.)

    I suspect it may (be|have been) more common with cron posts of FAQ
    messages, but a quick look at everything in alt.answers and comp.answers
    on my local server looks like cron to news server not cron to mail.
    Those are the only groups I can think of that are moderated and getting
    FAQ posts from someone other than the moderator.

    Elijah
    ------
    [*] late 1990s alt.sex.stories.moderated, using home grown software not STUMP

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Thomas Hochstein@21:1/5 to Owen Rees on Tue Nov 2 04:14:53 2021
    Owen Rees schrieb:

    I expect the idea was that you can post to a moderated group by sending
    an email to the submission address. Does any non-spammer ever do that?

    Yes, of course, but not for moderated discussion groups. For those
    groups, it's a red flag.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Owen Rees@21:1/5 to All on Tue Nov 2 12:31:22 2021
    On Tue, 2 Nov 2021 04:14:54 EDT, Eli the Bearded <*@eli.users.panix.com>
    wrote in <eli$2111012233@qaz.wtf>:

    In news.admin.moderation, Owen Rees <orees@hotmail.com> wrote:
    I would expect that if the submission address for a moderated group gets
    onto email spam lists, the spambots will send email to that address but
    it is unlikely to include a Newsgroups header. I would hope that
    anything sent from a news server or a news client configured to send
    directly for moderated groups should include a Newsgroups header.

    I would think 100% of spam to the submission address will lack a
    Newgroups header. And I think 100% of posts submitted to news servers
    will have a Newsgroups header. I believe the RFC1036 authors would not
    have imagined _requiring_ separate email addresses for separate groups,
    so include the Newsgroups header with all posts. (Further, I bet 100%
    of the posts taking the news server route will have a Path header and
    none of the pure email ones will.)

    RFC1036 does not describe how messages get to the moderators of
    moderated groups. RFC5537 does have a description of what I believe has
    been the standard practice for a long time (except for its preferred encapsulation method). Every moderated group has its own submission
    email address and there is an established formula for deriving a
    submission address from the newsgroup name. Those addresses go via a
    forwarding service which has a list of the actual addresses (or further forwarders for some hierarchies).

    If everyone is doing things properly, the messsages arriving for
    moderation for a target group should have the target group as the first
    named moderated group in the Newsgroups header. There may also be other moderated groups named in the header and there is a known process for
    dealing with that which involves rearranging the Newsgroups header and
    sending in on to the next moderated group submission address if it is
    approved.


    That question may be difficult to answer because STUMP adds a Newsgroups
    header if it is missing. If the header is present but does not name the
    target group then STUMP adds the group. If all you see is the mail
    forwarded by STUMP then it may not be obvious that it was emailed
    directly without a Newsgroups header.

    Very often I think you could guess based on the purported user-agent.
    A careful reading of the Received: headers would probably work, too.

    Anything that needs human scrutiny is probably unnecessary. The messages
    will probably be obvious spam to a human moderator.


    I expect the idea was that you can post to a moderated group by sending
    an email to the submission address. Does any non-spammer ever do that?

    Yes. Not often, but it happens. I've seen newsgroups in 2021 still >advertising the submission address and I saw in the last year someone
    post to misc.legal.moderated saying they used that after having trouble >posting regularly. (m.l.m seems to be quirky. I have moderated a group
    in the distant past[*], and _I've_ found posting there tricky.

    I do not use one myself but I believe that some news clients can be
    configured to send posts to moderated groups directly to the submission address. I would expect that a news client would include the Newsgroups
    header if it is doing that.

    People running news servers may also configure them to send directly to
    the real submission address for a moderated group rather than relying on
    the forwarding.


    I suspect it may (be|have been) more common with cron posts of FAQ
    messages, but a quick look at everything in alt.answers and comp.answers
    on my local server looks like cron to news server not cron to mail.
    Those are the only groups I can think of that are moderated and getting
    FAQ posts from someone other than the moderator.

    I would hope that anyone writing a script to submit posts to a moderated
    group would include the Newsgroups header if sending it by email to the submission address.


    Elijah
    ------
    [*] late 1990s alt.sex.stories.moderated, using home grown software not STUMP

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jason Evans@21:1/5 to Owen Rees on Wed Nov 3 17:43:36 2021
    On Mon, 1 Nov 2021 19:49:02 EDT, Owen Rees wrote:

    Since I do not
    actually run a moderated group I cannot do any live tests but anyone who
    is suffering from this problem is welcome to adopt/adapt what I have
    done.

    Hi Owen,

    I think we've fixed that in our STUMP repo. I've emailed you directly
    with info and we've got a live session that we can use for tests.

    For anyone else who's interested in the bug fixes that we've done, our
    project page is here:

    https://savannah.gnu.org/projects/stump/ https://savannah.gnu.org/projects/webstump

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Owen Rees@21:1/5 to Jason Evans on Fri Nov 5 18:30:22 2021
    On Wed, 3 Nov 2021 17:43:36 EDT, Jason Evans <board@big-8.org> wrote in <sluvor$mbp$2@theuse.news.theuse.net>:

    On Mon, 1 Nov 2021 19:49:02 EDT, Owen Rees wrote:

    Since I do not
    actually run a moderated group I cannot do any live tests but anyone who
    is suffering from this problem is welcome to adopt/adapt what I have
    done.

    Hi Owen,

    I think we've fixed that in our STUMP repo. I've emailed you directly
    with info and we've got a live session that we can use for tests.

    I have sent you a reply. Let me know if you have not received it.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)