(*) Except for alt.* and free.*, to the extent that anyone honors them.
Hi all,
I'm considering a policy change for the newsgroup lists maintained at >ftp.isc.org to only honor PGP-signed control messages except for alt.* and >free.* and wanted to run them by everyone.
Historically, control.ctl has included entries for large numbers of local, >regional, and language hierarchies that predate control message signing or >that didn't go to the trouble of creating PGP keys and setting up signing.
Since we didn't want to break anything when control message signing was >introduced, those entries were only changed if there was an abuse problem. >Many of those hierarchies are too small and obscure for anyone to have >bothered to forge control messages for them, even back in the heyday of >control message vandalism.
This has been bothering me for a while, though, since I have a rather
strong interest in making this system as automated as possible since I
have very little time to fix things manually. Vandalism would be easy to >manually repair, but it would require I go do something about it, which is >unappealing.
Possibly more relevantly, I have not seen anyone who in theory is
maintaining any of those non-PGP hierarchies issue a valid control message
in years (probably more than ten years). In practice, I don't believe
anyone is sending unsigned control messages except for alt.* and free.* >(which are intended to be a free-for-all left to each individual site to >manage), and I believe all of those legacy entries are effectively
defunct.
I am therefore proposing removing all non-PGP entries from control.ctl or, >alternately, leaving them there but commented out. I'm kind of leaning >towards the former since if anyone cares about the history for some reason >they can get it from old versions of control.ctl in the INN repository or >from <https://github.com/rra/control-archive/> (and I have no reason to >believe that the people identified with those email addresses still exist
or feel in any way responsible for those hierarchies), but I could be >convinced to leave them there commented out.
Thoughts?
chi.*, for instance, hasn't had a hierarchy administrator since Gerry
Swetsky moved away. He never sent a newgroup message to start a new
group that I recall, all groups were started before he was the
administrator. But if a group were proposed, we were supposed to get
together for an in-person meeting, probably called as Uniforum Chicago
or a successor if it's still meeting. It was pretty informal and mostly
an excuse to drink beer, if it ever happened. If people wanted a new
group, Gerry would have sent a newgroup message.
Historically, control.ctl has included entries for large numbers of
local, regional, and language hierarchies that predate control message
signing or that didn't go to the trouble of creating PGP keys and
setting up signing.
Unless any of the massive attacks included bogus newgroup messages in
any of these hierarchies, why would they have bothered to have
implemented authenticated control messages in the past?
I haven't reviewed the documents in years, but rone's unified
control.ctl used to list a dozen local hierarchies with a note as to
which institution or News server provider they were for. I thought once
you took over the document, you purged them as they aren't Usenet, or
you moved the list to hierarchy-notes.
Well, yeah. And I would request that you continue to treat them as
"There is no problem to fix."
A lot of nearly dead hierarchies may still have a bit of discussion in
the *.general or equivalent newsgroup. Let's leave the option that if
there's an actual need to propose and create a new group, that there is
no requirement to implement authenticated control messages without a
need for it.
(*) Except for alt.* and free.*, to the extent that anyone honors them.[...]
I'm considering a policy change for the newsgroup lists maintained at ftp.isc.org to only honor PGP-signed control messages except for alt.* and free.* and wanted to run them by everyone.
Thoughts?I have no objection.
I don't believe anyone is sending unsigned control messages exceptOk, though some newsgroup names are questionable (like free.biden.sucks
for alt.* and free.* (which are intended to be a free-for-all left to
each individual site to manage)
I am therefore proposing removing all non-PGP entries from control.ctl or, alternately, leaving them there but commented out.
Ok, though some newsgroup names are questionable (like free.biden.sucks created today) but that's another debate!
Separate active and newsgroups files containing only alt.* and free.* newsgroups may be provided by control-archive and put in ftp.isc.org
(just a thought, to better enhance the difference).
Private hierarchies like bofh.* or szaf.*, and historic hierarchies like net.* or eug.*, which have a PGP key, will remain if I understand well.
As well as reserved hierarchies (control.*, example.*, to.* ...) for technical reasons.
Removing them from control.ctl doesn't remove the newsgroups. It just
means no changes will be honored, and the existing newsgroup list will be kept as-is. That seems fine? If someone wants to change it, they would
have to create a PGP key and set up some software to issue the control messages, which is a bit higher of a bar, but in practice this seems to happen rarely and I'm sure a bunch of people here would be happy to help
if it came up.
I'm considering a policy change for the newsgroup lists maintained at ftp.isc.org to only honor PGP-signed control messages except for alt.* and free.* and wanted to run them by everyone.
I am therefore proposing removing all non-PGP entries from control.ctl or, alternately, leaving them there but commented out. I'm kind of leaning towards the former since if anyone cares about the history for some reason they can get it from old versions of control.ctl in the INN repository or from <https://github.com/rra/control-archive/> [...]
"Adam H. Kerman" <ahk@chinet.com> writes:
chi.*, for instance, hasn't had a hierarchy administrator since Gerry >>Swetsky moved away. He never sent a newgroup message to start a new
group that I recall, all groups were started before he was the >>administrator. But if a group were proposed, we were supposed to get >>together for an in-person meeting, probably called as Uniforum Chicago
or a successor if it's still meeting. It was pretty informal and mostly
an excuse to drink beer, if it ever happened. If people wanted a new
group, Gerry would have sent a newgroup message.
The thing is, though is that none of this has happened. Even ten years
ago, legitimate unsigned control messages basically don't exist. So far
as I can tell, the last change to chi.* was Hipcrime sabotage that we had
to manually reverse because we still had this unauthenticated control
message policy. In fact, nearly all chi.* control messages that are
archived are abusive sabotage. Thankfully that hasn't happened since
2002, but if it happened again, it would be a giant mess and a huge pain
for me to clean up.
Historically, control.ctl has included entries for large numbers of >>>local, regional, and language hierarchies that predate control message >>>signing or that didn't go to the trouble of creating PGP keys and
setting up signing.
It turns out that I was probably wrong about this and David Lawrence
instead did tons of manual cleanup. There are a bunch of forged control >messages for chi.*, for example, from back when this was common.
Unless any of the massive attacks included bogus newgroup messages in
any of these hierarchies, why would they have bothered to have
implemented authenticated control messages in the past?
With the above correction, I can note that this did happen, and yet they >still didn't implement authenticated control messages, unfortunately. I >suspect in most cases that's because these folks are no longer using
Usenet, and in most cases (such as with Gary Swetsky) no longer have the >email addresses that they were using to send these messages (and in some >cases may no longer be alive; it's been 30 years in many cases).
I haven't reviewed the documents in years, but rone's unified
control.ctl used to list a dozen local hierarchies with a note as to
which institution or News server provider they were for. I thought once
you took over the document, you purged them as they aren't Usenet, or
you moved the list to hierarchy-notes.
I don't *think* I removed anything unless I could confirm that it was >defunct. But lots of these hierarchies are just unmaintained and in use
but not changing the newsgroup list.
I see that what I did for wpg.* was replace the entry with:
## WPG (Winnipeg, Manitoba, Canada)
#
# This hierarchy is still in use, but it has no active maintainer.
# Control messages for this hierarchy should not be honored without
# confirming that the sender is the new hierarchy maintainer.
I could do something similar for the others, which would avoid losing the
URL if it still works.
Well, yeah. And I would request that you continue to treat them as
"There is no problem to fix."
The problem with doing this from my perspective is that at any point it
could turn into a giant problem for me to fix, and should that happen, the >amount of time I'd have to spend on it would be way higher than the amount
of time it would take for me to prevent this proactively now.
A lot of nearly dead hierarchies may still have a bit of discussion in
the *.general or equivalent newsgroup. Let's leave the option that if >>there's an actual need to propose and create a new group, that there is
no requirement to implement authenticated control messages without a
need for it.
I think that's what my proposal does?
Removing them from control.ctl doesn't remove the newsgroups. It just
means no changes will be honored, and the existing newsgroup list will be >kept as-is. That seems fine?
If someone wants to change it, they would
have to create a PGP key and set up some software to issue the control >messages, which is a bit higher of a bar, but in practice this seems to >happen rarely and I'm sure a bunch of people here would be happy to help
if it came up.
Russ Allbery <eagle@eyrie.org> wrote:
The thing is, though is that none of this has happened. Even ten years
ago, legitimate unsigned control messages basically don't exist. So
far as I can tell, the last change to chi.* was Hipcrime sabotage that
we had to manually reverse because we still had this unauthenticated
control message policy. In fact, nearly all chi.* control messages
that are archived are abusive sabotage. Thankfully that hasn't
happened since 2002, but if it happened again, it would be a giant mess
and a huge pain for me to clean up.
But weren't these sent as a massive denial-of-service attack and not individually? Doesn't that allow you to thwart the attack?
Removing them from control.ctl doesn't remove the newsgroups. It just
means no changes will be honored, and the existing newsgroup list will
be kept as-is. That seems fine?
I'm saying if there is a need to send a newgroup message, you would
rather complicate matters.
If someone wants to change it, they would have to create a PGP key and
set up some software to issue the control messages, which is a bit
higher of a bar, but in practice this seems to happen rarely and I'm
sure a bunch of people here would be happy to help if it came up.
How would you know it's not a troll?
I don't have a good suggestion that doesn't require manual intervention either way. If no one is maintaining a hierarchy, not even an
occassional checkgroups, maybe an exchange of emails is necessary should someone issue a newgroup message before you archive it.
What if you delayed processing the messages for archiving to prevent a
denial of service attack?
BTW, for full information, I did just see an unsigned but apparently valid checkgroups message today, so apparently at least one hierarchy is sending them (greenend.*).
Russ Allbery schrieb:
BTW, for full information, I did just see an unsigned but apparently
valid checkgroups message today, so apparently at least one hierarchy
is sending them (greenend.*).
A private hierachy that was not part of the last Master List I could
find - and I don't doubt that the people of
<http://www.greenend.org.uk/> would be able to sign control messages
if they cared. :)
Thomas Hochstein <thh@thh.name> writes:
Russ Allbery schrieb:
BTW, for full information, I did just see an unsigned but apparently
valid checkgroups message today, so apparently at least one hierarchy
is sending them (greenend.*).
A private hierachy that was not part of the last Master List I could
find - and I don't doubt that the people of
<http://www.greenend.org.uk/> would be able to sign control messages
if they cared. :)
Yeah, I'm pretty sure this is just unintentional leakage from some private peering and not all that relevant to this discussion.
Hi Russ,
(*) Except for alt.* and free.*, to the extent that anyone honors them.
I'm considering a policy change for the newsgroup lists maintained at >>ftp.isc.org to only honor PGP-signed control messages except for alt.* and >>free.* and wanted to run them by everyone.
[...]
Thoughts?
I have no objection.
I reckon it is the right move to do.
I don't believe anyone is sending unsigned control messages except
for alt.* and free.* (which are intended to be a free-for-all left to
each individual site to manage)
Ok, though some newsgroup names are questionable (like free.biden.sucks >created today) but that's another debate!
Separate active and newsgroups files containing only alt.* and free.* >newsgroups may be provided by control-archive and put in ftp.isc.org
(just a thought, to better enhance the difference).
I am therefore proposing removing all non-PGP entries from control.ctl or, >>alternately, leaving them there but commented out.
I'm fine with removing non-PGP entries, including private, local,
historic and defunct hierarchies.
The main argument would be that the control.ctl file is used as a >configuration file, not as the memory of Usenet history.
. . .
Under the current scheme, invalid mailboxes (and even a NULL string) are accepted for some control messages where they shouldn't be. Only the
"drop" action (in file "control.ctl") can have the mailbox match be "*", because that is a "don't care" case.
The 'from' field in control.ctl should be changed from "*" (where that appears) to "?*@?*.??*" to make certain that a legal mailbox is
accepted. "*" by itself will match 0 characters, so the adjacent "?"s
make certain that each component has at least one character. The domain
side basically needs two components with an intervening dot. Although "localhost" is an acceptable domain, it is not useful in this context,
so I intentionally suggest a syntactic pattern match that excludes it.
Some patterns with matching text may also need "*" changed to "?*" for positive actions (i.e. not drop).
Under the current scheme, invalid mailboxes (and even a NULL string) are accepted for some control messages where they shouldn't be. Only the
"drop" action (in file "control.ctl") can have the mailbox match be "*", because that is a "don't care" case.
The 'from' field in control.ctl should be changed from "*" (where that appears) to "?*@?*.??*" to make certain that a legal mailbox is
accepted. "*" by itself will match 0 characters, so the adjacent "?"s
make certain that each component has at least one character. The domain
side basically needs two components with an intervening dot. Although "localhost" is an acceptable domain, it is not useful in this context,
so I intentionally suggest a syntactic pattern match that excludes it.
Some patterns with matching text may also need "*" changed to "?*" for positive actions (i.e. not drop).
1) I did not limit my suggestion to just alt/free.*. It applies to ALL hierarchies where the mailbox field has a wildcard and the action is not "drop."
2) It will eliminate many of the poorly formatted fake control messages
which do not use syntactically valid mailboxes.
I examined the past control message archive.
Out of about 85,000 groups, only about 35,000 have valid newgroup
messages. The other 50,000 fell into three categories: Bad newsgroup
names, bad from mailboxes, and omitting the "For your newsgroups file:"
line followed by the group description on the next line.
What does it save: It saves e-mailing the usenet site administrator
with bogus messages as the default action for newgroup, rmgroup, and checkgroups is "mail."
"D. Stussy" <spam@spam.org> writes:
1) I did not limit my suggestion to just alt/free.*. It applies to ALL >>hierarchies where the mailbox field has a wildcard and the action is not >>"drop."
It's irrelevant to anything other than alt.* and free.* because those are
the only ones that allow wildcard control messages, no? I think there may
be a few other minor exceptions, but nothing that I've seen in any >significant numbers in many years.
2) It will eliminate many of the poorly formatted fake control messages >>which do not use syntactically valid mailboxes.
What fake control messages are you seeing that aren't for alt.* and
free.*?
I examined the past control message archive.
What goes into control.ctl is irrelevant to the control message archive.
Out of about 85,000 groups, only about 35,000 have valid newgroup
messages. The other 50,000 fell into three categories: Bad newsgroup >>names, bad from mailboxes, and omitting the "For your newsgroups file:" >>line followed by the group description on the next line.
Did you look at the dates? This is almost entirely stuff that was
archived 15 or 20 years ago.
If you're saying that I should go through the archive and delete old
invalid control messages from it, that's a whole different argument. But >nothing about control.ctl has any influence on that.
. . .
Waitaminit.
I thought the whole point of this that you were no longer intending to archive control messages that weren't from hierarchies with PGP signing.
Hi Adam,
Waitaminit.
I thought the whole point of this that you were no longer intending to >>archive control messages that weren't from hierarchies with PGP signing.
The subject of this thread is "stop honoring" (that is to say actually >creating and removing newsgroups in the ftp.isc.org active file), not
"stop archiving"...
Waitaminit.
I thought the whole point of this that you were no longer intending to archive control messages that weren't from hierarchies with PGP signing.
"Adam H. Kerman" <ahk@chinet.com> writes:
Waitaminit.
I thought the whole point of this that you were no longer intending to >>archive control messages that weren't from hierarchies with PGP signing.
No, this whole thread was only about the default control.ctl from INN and >ftp.isc.org and the ftp.isc.org newsgroup list.
ISC hosts the archive and I don't feel any particular need to clean it up.
People have spammed it with all sorts of crap. I added some basic >anti-binary filtering to keep from dealing with stupid copyright nonsense, >and otherwise it doesn't take up much space and I don't realy care.
(Please no one do anything that makes me have to care.)
At some point I may go clean up the archived control messages for
literally syntactically invalid groups that would never be archived today >(there are archive files for nonsense like group names containing *), but >realistlcally I'm too busy with other things and probably won't get around
to it.
I am confused, though. I thought the archive was also fed by a subset of
INN processes that parse for control messages, and that a control
message is archived in the same process that the sample newsgroup and
active files are updated in.
"Adam H. Kerman" <ahk@chinet.com> writes:
I am confused, though. I thought the archive was also fed by a subset of >>INN processes that parse for control messages, and that a control
message is archived in the same process that the sample newsgroup and >>active files are updated in.
The code is unrelated to INN, apart from INN providing an article feed and >the tinyleaf program that I use to process that feed. Both things are
done by the same process, yes, but the archiving is done separately from
the checks about whether to honor the message and applies only more basic >sanity checks to throw away syntactically-invalid junk and figure out what >newsgroup would supposedly be affected by the message.
The code is all in <https://github.com/rra/control-archive>.
1) I did not limit my suggestion to just alt/free.*. It applies to ALL hierarchies where the mailbox field has a wildcard and the action is not "drop."
2) It will eliminate many of the poorly formatted fake control messages
which do not use syntactically valid mailboxes.
I examined the past control message archive.
Out of about 85,000 groups, only about 35,000 have valid newgroup
messages. The other 50,000 fell into three categories: Bad newsgroup
names, bad from mailboxes, and omitting the "For your newsgroups file:"
line followed by the group description on the next line.
What does it save: It saves e-mailing the usenet site administrator
with bogus messages as the default action for newgroup, rmgroup, and checkgroups is "mail."
Preventing a (malicious) flood of these is the point.
I did receive one e-mailed newgroup message for the "rtfm" hierarchy
this month, a hierarchy not in the control.ctl file. The greenend.* hierarchy checkgroups message was also emailed because there is
something wrong with its signature. I don't think that's in the control
file either, but I have seen it before so it's in my control.ctl.local
file.
| Sysop: | Keyop |
|---|---|
| Location: | Huddersfield, West Yorkshire, UK |
| Users: | 291 |
| Nodes: | 16 (2 / 14) |
| Uptime: | 143:39:53 |
| Calls: | 6,611 |
| Calls today: | 1 |
| Files: | 12,159 |
| Messages: | 5,310,491 |