1. Forum
  2. Usenet
  3. COMP.INFOSYSTEMS.GOPHER

  • Gopher mirroring tool

    From andres@21:1/5 to All on Mon Jul 18 11:17:20 2022
    Hello all. Does anyone know some Gopher mirroring tool a
    la "wget --mirror" does on HTTP(S)? I already tried
    pavuk, but it misses some relative links.
    I tried to mirror the whole archive from
    gopher://mozz.us/1/funnybone, but it didn't
    neither download the issues nor the yearly
    items linked as Gopher directories.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Dennis Boone@21:1/5 to All on Mon Jul 18 11:21:29 2022
    Hello all. Does anyone know some Gopher mirroring tool a
    la "wget --mirror" does on HTTP(S)? I already tried
    pavuk, but it misses some relative links.
    I tried to mirror the whole archive from
    gopher://mozz.us/1/funnybone, but it didn't
    neither download the issues nor the yearly
    items linked as Gopher directories.

    The gopher protocol is simple enough you could probably build one
    without too much effort. The perl modules LWP::Protocol::gopher and/or URI::gopher might do a fair amount of the work. Python doesn't seem to
    have much in the way of useful libraries to help.

    De

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From dcb@21:1/5 to All on Sun Jul 24 12:06:54 2022
    Hello all. Does anyone know some Gopher mirroring tool a
    la "wget --mirror" does on HTTP(S)? I already tried
    pavuk, but it misses some relative links.
    I tried to mirror the whole archive from
    gopher://mozz.us/1/funnybone, but it didn't
    neither download the issues nor the yearly
    items linked as Gopher directories.

    The gopher protocol is simple enough you could probably build one
    without too much effort. The perl modules LWP::Protocol::gopher and/or URI::gopher might do a fair amount of the work. Python doesn't seem to
    have much in the way of useful libraries to help.

    De

    I wrote a Python library called Pituophis (https://github.com/dotcomboom/Pituophis) some time ago that would help
    here.
    Recursively parsing each 1-type menu, fetching all the 0-type items and
    saving them shouldn't be too bad. I can write a small script for it if you'd like.

    Also this is my first newspost, sorry if I made a mistake..

    dcb

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From dcb@21:1/5 to dcb on Sun Jul 24 14:43:57 2022
    dcb wrote:
    Recursively parsing each 1-type menu, fetching all the 0-type items and saving them shouldn't be too bad. I can write a small script for it if
    you'd like.

    I wrote a script: https://gist.github.com/dotcomboom/1c216ea08da60d0f1c0810a33963b216

    Should be able to put the starting menu url in the list and it'll download
    it recursively.

    dcb

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Szczezuja.space@21:1/5 to andres on Fri Apr 7 18:55:22 2023
    On 2022-07-18, andres <andres@disroot.org> wrote:
    Hello all. Does anyone know some Gopher mirroring tool a
    la "wget --mirror" does on HTTP(S)? I already tried
    pavuk, but it misses some relative links.
    I tried to mirror the whole archive from
    gopher://mozz.us/1/funnybone, but it didn't
    neither download the issues nor the yearly
    items linked as Gopher directories.

    I'm testing https://sr.ht/~lioploum/offpunk/ and its cache folder for
    the similar usecase. I'd like to create personal "Internet Archive" for
    several Gopherholes and publish it.

    --
    .-=-. Szczezuja; on the small-net:
    ( S\ \ gemini://szczezuja.space/ - gemlog & tinylog
    `--' / gopher://sdf.org:70/0/users/szczezuja/ - phlog

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ben Collver@21:1/5 to andres on Sat Apr 8 14:38:57 2023
    On 2022-07-18, andres <andres@disroot.org> wrote:
    Hello all. Does anyone know some Gopher mirroring tool a
    la "wget --mirror" does on HTTP(S)?

    I wrote a gopher mirroring tool in Tcl.

    I have only used it on Linux.

    https://wiki.tcl-lang.org/page/gopher%2Dget

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)