• Bug#1067160: python3-debian: Difficult to iterate over next-level index

    From Colin Watson@21:1/5 to All on Tue Mar 19 16:00:01 2024
    Package: python3-debian
    Version: 0.1.49
    Severity: wishlist

    I've trying to figure out how to build a specialized repository
    mirroring tool as part of debusine (https://freexian-team.pages.debian.net/debusine/), and I started by
    looking at python-debian. First I ran into
    https://bugs.debian.org/710923, which would be an issue - past
    experience with https://bugs.debian.org/918304 suggests that that's much
    harder to get right than it seems. But, assuming that that could be
    solved somehow, I then started thinking about what would be needed next.

    After verifying the Release file, a mirroring tool needs to fetch the
    next level of index files (typically Packages and Sources).
    python-debian offers very little help with this - you get a parsed
    version of the various checksums fields, but that's about it - and there
    are some non-trivial complications here. For instance, https://deb.debian.org/debian/dists/unstable/InRelease lists checksums
    for main/binary-amd64/Packages, but those are the checksums you get
    after fetching a compressed version of the file and uncompressing it;
    the uncompressed version isn't actually on the mirror, and you have to
    fetch https://deb.debian.org/debian/dists/unstable/main/binary-amd64/Packages.xz
    or similar instead. While I have a rough idea of how to implement this
    and could borrow code from apt, I don't really want to have to write
    more versions of this than necessary.

    Ideally (for me), Release would have some way to list the next-level
    index files, and provide a way to fetch checksum-verified versions of
    them given a base URL, parsed into Packages/Sources instances as
    appropriate, and abstracting over all the details of compression
    algorithms and such.

    I think I may end up using apt or python-apt for now, but python-apt is
    always a slightly awkward dependency in a Python codebase due to its
    tight coupling with apt, and python-debian is much more convenient in
    those terms.

    Thanks,

    --
    Colin Watson (he/him) [cjwatson@debian.org]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)