• Re: Article retention and archive search

    From The Running Man@21:1/5 to noreply@mixmin.net on Wed Sep 4 04:25:43 2024
    On 03/09/2024 21:17 D <noreply@mixmin.net> wrote:

    some news servers have very long article retention, with terabytes
    of overview and article storage, and some newsgroups with millions
    of articles, from historical usenet archives, commercial providers
    e.g. utzoo-wiseman archive http://annex.retroarchive.org/utzoo and https://archive.org/details/usenethistorical with many hierarchies;
    these archives can also be searched by retrieving overview headers
    online> sample headers from selected groups > download all headers
    (up to 'x'), the total available depending on server and newsgroup,
    or select a smaller number as needed; after downloading be sure to
    set group > selected group options to "never" purge read or unread
    article headers; subscribe to newsgroup, and backup entire program
    folder (c:\40tude dialogue) to removable media; with the newsgroup
    article list pane selected [f6], try subject > has, and enter some
    keyword probably found in that group, e.g. famous historical event
    in news:alt.history.ocean-liners.titanic, type "iceberg" and press
    enter (or click button on right side) to filter the list, which if
    using the popular "news.blueworldhosting.com" server, should yield
    about three dozen articles; select all and press enter to download
    these small article bodies; in edit > find [ctrl+f] > text to find
    in article body pane, try "olympic" in scope > selected group, and
    click ok, total of ten instances may be found using find next [f3];
    but there are about three thousand four hundred (3400) articles in
    this newsgroup, many dating back over twenty years, so it's easier
    to search overview subject headers first for keywords, then filter
    the list before downloading several dozen rather than thousands of
    article bodies (any server may block ip if excessive use detected);

    many free newsservers store articles for only about one year, some
    even less, but others have longer retention, so the best way is to
    test them; add as many servers as you like, and get complete group
    list for each; at this writing news.alt119.net, paganini.bofh.team, news.usenet.ovh, freenews.netfront.net, news.novabbs.org, and some
    others (see news:alt.free.newsservers) should be added to the list;
    pick one popular newsgroup and download all headers on each server
    for comparison; the server i've been using regularly on "localhost"
    has less than one year of article retention, but connects securely
    via omnimix and using tor browser, socks 5, implicit ssl, port 563;

    while some newsgroups have remained popular over the years, others
    have not or had long ago succumbed to troll farm infestation which
    overran that formerly active discussion forum--perhaps hundreds of
    newsgroups fit that description? but sometimes the public interest
    in a particular subject like the sinking of a luxury liner in 1912
    waxes and wanes over time; sometimes the subject of a newsgroup is
    too obscure or lacks common definition, e.g. politics and religion
    are not alone in that category; anything other-worldly, paranormal,
    occult, ufos, aliens, ghosts, crop circles, tarot, astrology, etc.,
    is also too far removed from this weaponized world of human beings
    who are united in but one common purpose, namely the love of money;

    even so, there are many newsgroups that still attract some modicum
    of on-topic discussion and that makes archive searches worth while;
    also, recommended reading about how articles are stored on servers:

    https://www.eyrie.org/~eagle/software/inn/docs/install.html
    Choosing an Article Storage Format >>https://www.eyrie.org/~eagle/software/inn/docs/install.html#S6
    Choosing an Overview Storage Mechanism >>https://www.eyrie.org/~eagle/software/inn/docs/install.html#S7


    Which servers have the longest retention? AFAIK only Google had 100%
    retention since the advent of Usenet and they bought those archives from another company. Now that Google has severed itself from Usenet those
    archives are gone (or inaccessible).

    The only thing that's keeping Usenet alive is servers run by volunteers, like Eternal September and Cybernews.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From yeti@21:1/5 to The Running Man on Wed Sep 4 06:38:28 2024
    The Running Man <running_man@writeable.com> writes:

    The only thing that's keeping Usenet alive is servers run by volunteers, like Eternal September and Cybernews.

    Maybe the data <https://olduse.net/> uses can be restored somewhere else
    too?

    --
    I do not bite, I just want to play.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Scott Dorsey@21:1/5 to snipeco.1@gmail.com on Wed Sep 4 14:23:42 2024
    Sn!pe <snipeco.1@gmail.com> wrote:
    yeti <yeti@tilde.institute> wrote:

    The Running Man <running_man@writeable.com> writes:

    The only thing that's keeping Usenet alive is servers run by volunteers, >> > like Eternal September and Cybernews.

    Maybe the data <https://olduse.net/> uses can be restored somewhere else
    too?

    I think the Google archive can still be queried but it is no longer
    being updated.

    The indices in the Google archive were broken more than a decade ago and queries on message text haven't worked since then. To my mind this has
    made the Google archive pretty much useless.
    --scott

    --
    "C'est un Nagra. C'est suisse, et tres, tres precis."

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Steve Bonine@21:1/5 to All on Wed Sep 4 14:51:23 2024
    Sn!pe wrote:

    ... In my view Google's attempt to usurp
    Usenet's place on the Internet has been profoundly damaging to
    Usenet from the very outset.

    It has always been a mystery to me why Google got into the business of
    being a portal to Usenet, and why they kept the facility going for so
    long. I don't think they had a "profound" influence on Usenet either
    positively or negatively. They generated spam, but they did introduce
    Usenet to (a relatively small number of) users who became positive contributors. While it worked, their search mechanism was useful; by
    the time it effectively stopped working there wasn't much new material
    to search.

    If you're looking for a villain for Usenet's demise, look at technology.
    The general population decided that web-based facilities that didn't
    require downloading software were "better" than Usenet, ISPs read their
    users' preferences and stopped supporting it, and the useful material
    moved with the users.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From yeti@21:1/5 to Steve Bonine on Wed Sep 4 21:01:26 2024
    Steve Bonine <spb@pobox.com> writes:

    If you're looking for a villain for Usenet's demise, look at
    technology. The general population decided that web-based facilities
    that didn't require downloading software were "better" than Usenet,

    They could only switch to that when it existed. And it existed not to
    serve users. It was just a service more to squeeze more data out of the
    data cattle.

    --
    The Lincoln Project
    Nationalist Geographic (Impotus Americanus) https://www.youtube.com/watch?v=8x1-CVsoEBU

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From The Running Man@21:1/5 to spb@pobox.com on Wed Sep 4 21:05:47 2024
    On 04/09/2024 21:51 Steve Bonine <spb@pobox.com> wrote:
    Sn!pe wrote:

    ... In my view Google's attempt to usurp
    Usenet's place on the Internet has been profoundly damaging to
    Usenet from the very outset.


    If you're looking for a villain for Usenet's demise, look at technology.
    The general population decided that web-based facilities that didn't require downloading software were "better" than Usenet, ISPs read their users' preferences and stopped supporting it, and the useful material
    moved with the users.

    I'm not sure that's the reason. Users download apps on smartphones all the
    time to access a service. Why would a Usenet app suddenly be a problem?

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Steve Bonine@21:1/5 to The Running Man on Fri Sep 6 10:07:01 2024
    The Running Man wrote:
    On 04/09/2024 21:51 Steve Bonine <spb@pobox.com> wrote:
    Sn!pe wrote:

    ... In my view Google's attempt to usurp
    Usenet's place on the Internet has been profoundly damaging to
    Usenet from the very outset.


    If you're looking for a villain for Usenet's demise, look at technology.
    The general population decided that web-based facilities that didn't
    require downloading software were "better" than Usenet, ISPs read their
    users' preferences and stopped supporting it, and the useful material
    moved with the users.

    I'm not sure that's the reason. Users download apps on smartphones all the time to access a service. Why would a Usenet app suddenly be a problem?

    The decline of Usenet began before smartphones were an option. For
    example, AOL discontinued their Usenet service in 2005 based on the fact
    that most Usenet users had moved to web-based facilities.

    Obtaining Usenet is much more complicated than opening a web browser and
    going to a website. You need a news reader and a provider. While this
    might not seem "a big deal" to anyone remotely technically inclined, the general population is not remotely technically inclined.

    But why would anyone go to the trouble to access a Usenet newsgroup when
    all the participants have moved to a different platform?

    That's not to say that there aren't a few active Usenet groups these
    days, but Usenet is a shadow of its former self and the groups that are
    left active tend to be niche groups populated by regulars who have been
    there for years. The number of new users is vanishingly small due to
    the barriers to entry, the learning curve, and the lack of motivation to overcome these obstacles.

    Usenet is an important part of the history of the Internet and the
    development of what we call "social media" today. But as a valuable
    tool used in people's everyday life, its golden age has passed.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From D@21:1/5 to Steve Bonine on Fri Sep 6 17:22:10 2024
    On Fri, 6 Sep 2024 10:07:01 -0500, Steve Bonine <spb@pobox.com> wrote:
    The decline of Usenet began before smartphones were an option. For
    example, AOL discontinued their Usenet service in 2005 based on the fact
    that most Usenet users had moved to web-based facilities.
    Obtaining Usenet is much more complicated than opening a web browser and >going to a website. You need a news reader and a provider. While this
    might not seem "a big deal" to anyone remotely technically inclined, the >general population is not remotely technically inclined.
    But why would anyone go to the trouble to access a Usenet newsgroup when
    all the participants have moved to a different platform?
    That's not to say that there aren't a few active Usenet groups these
    days, but Usenet is a shadow of its former self and the groups that are
    left active tend to be niche groups populated by regulars who have been
    there for years. The number of new users is vanishingly small due to
    the barriers to entry, the learning curve, and the lack of motivation to >overcome these obstacles.
    Usenet is an important part of the history of the Internet and the >development of what we call "social media" today. But as a valuable
    tool used in people's everyday life, its golden age has passed.

    yet another case in point that unmoderated newsgroups are the only
    public forum for plain text free speech to reach a global audience;
    social media is moderated no matter what they say but "golden age"?

    https://duckduckgo.com/?q=golden+age
    golden age
    noun
    1. A period of great peace, prosperity, and happiness.
    2. The first age of the world, an untroubled and prosperous
    era during which people lived in ideal happiness.
    3. A happy age of peace and prosperity.
    The American Heritage(r) Dictionary of the English Language,
    5th Edition * More at Wordnik
    ...
    https://en.wikipedia.org/wiki/Golden_age_(metaphor)
    Golden age (metaphor) - Wikipedia
    A golden age is a period of high achievement or harmony in the
    history of a country or people. The term originated from Greek
    and Roman mythology and has been applied to various cultures and
    eras, such as the Athenian, Islamic, and Chinese golden ages.
    ...

    no such thing under the sun . . . only summerland vacation awaits

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From D@21:1/5 to Steve Bonine on Fri Sep 6 18:27:13 2024
    On Fri, 6 Sep 2024 10:07:01 -0500, Steve Bonine <spb@pobox.com> wrote:
    snip
    Obtaining Usenet is much more complicated than opening a web browser and >going to a website. You need a news reader and a provider. While this
    might not seem "a big deal" to anyone remotely technically inclined, the >general population is not remotely technically inclined.

    p.s. the rocksolid light backend server is easily accessible to anyone
    using a web browser: https://news.novabbs.org/common/grouplist.php, e.g. https://news.novabbs.org/arts/thread.php?group=alt.fan.usenet, thread... https://news.novabbs.org/computers/article-flat.php?id=800&group=alt.fan.usenet#800

    rocksolid runs in any web browser, including tor browser which makes it
    more secure...and many usenet news servers carry most of the newsgroups
    listed at https://news.novabbs.org/common/grouplist.php, so posting can
    be done from any newsserver, including gateways and anonymous remailers,
    so anyone visiting the "novabbs" website https://news.novabbs.org/ will
    be able to find articles posted there; and if desired can register with username, email, password: https://news.novabbs.org/common/register.php
    to enable posting using the web browser (no need for usenet newsreader)

    always better to use a selection of nntp newsservers with the dedicated newsreader, especially because there are many thousands of other groups
    that the "novabbs.org" website does not include, even though the server https://news.novabbs.org/common/connect.php has "approximately 42,000",
    and other servers have much longer and more complete article retention, news.blueworldhosting.com/paganini.bofh.team/freenews.netfront.net etc.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Steve Bonine@21:1/5 to All on Fri Sep 6 14:37:46 2024
    D wrote:
    317 words that addressed none of what they were replying to.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From The Running Man@21:1/5 to noreply@mixmin.net on Fri Sep 6 20:34:46 2024
    On 06/09/2024 19:27 D <noreply@mixmin.net> wrote:
    On Fri, 6 Sep 2024 10:07:01 -0500, Steve Bonine <spb@pobox.com> wrote:
    snip
    Obtaining Usenet is much more complicated than opening a web browser and >>going to a website. You need a news reader and a provider. While this >>might not seem "a big deal" to anyone remotely technically inclined, the >>general population is not remotely technically inclined.

    p.s. the rocksolid light backend server is easily accessible to anyone
    using a web browser: https://news.novabbs.org/common/grouplist.php, e.g. https://news.novabbs.org/arts/thread.php?group=alt.fan.usenet, thread... https://news.novabbs.org/computers/article-flat.php?id=800&group=alt.fan.usenet#800

    rocksolid runs in any web browser, including tor browser which makes it
    more secure...and many usenet news servers carry most of the newsgroups listed at https://news.novabbs.org/common/grouplist.php, so posting can
    be done from any newsserver, including gateways and anonymous remailers,
    so anyone visiting the "novabbs" website https://news.novabbs.org/ will
    be able to find articles posted there; and if desired can register with username, email, password: https://news.novabbs.org/common/register.php
    to enable posting using the web browser (no need for usenet newsreader)

    always better to use a selection of nntp newsservers with the dedicated newsreader, especially because there are many thousands of other groups
    that the "novabbs.org" website does not include, even though the server https://news.novabbs.org/common/connect.php has "approximately 42,000",
    and other servers have much longer and more complete article retention, news.blueworldhosting.com/paganini.bofh.team/freenews.netfront.net etc.


    The selection of Usenet newsgroups on NovaBBS is very limited. Too limited
    for my liking.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From A1@21:1/5 to Steve Bonine on Tue Oct 1 16:18:13 2024
    Steve Bonine <spb@pobox.com> wrote:
    Sn!pe wrote:

    ... In my view Google's attempt to usurp
    Usenet's place on the Internet has been profoundly damaging to
    Usenet from the very outset.

    It has always been a mystery to me why Google got into the business of
    being a portal to Usenet, and why they kept the facility going for so
    long.

    Google has always been in the business of easily processable data about
    people. Next time you talk to ChatGPT ask it how many Usenet articles
    it’s been trained on.

    They generated spam

    I suspect that in the last months (or maybe years) Google was doing it
    on purpose. Group sci.crypt was still somewhat alive until recently when
    a big wave of Google spam drove off the last few people who were talking
    there.

    If you're looking for a villain for Usenet's demise, look at technology.
    The general population decided that web-based facilities that didn't
    require downloading software were "better" than Usenet

    I doubt it was exactly that. Creating web interface for Usenet was not
    very hard task even in the days of early web forums.

    There was desire to create gated communities, even if just for the
    purpose of easy moderation.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)