• Challenges packaging Python for a Linux distro - at Python Lang

    From Luke Kenneth Casson Leighton@21:1/5 to Thomas Goirand on Mon May 17 08:50:01 2021
    (apologies i forgot to say, please do cc me, i am using the list
    archives reply-to
    links at https://lists.debian.org/debian-python/2021/05/msg00036.html)

    Thomas Goirand <zigo@debian.org> wrote:

    All the horrors that you are painting after this paragraph, are due to
    the fact that you aren't doing "apt-get dist-upgrade". I'm having a hard
    time understanding why you're both:
    - not doing "apt-get dist-upgrade"

    because precisely the difficulties encountered, and due to software development that critically requires more recent variants of what is typically in debian/stable
    i.e. usually a minimum of one even three years out-of-date.

    a dist-upgrade to debian / testing - a way to obtain the latest
    variants of critical
    software - frequently resulted in massive breakage.

    i quickly learned never to attempt that again given the massive disruption
    it caused to me being able to earn money as a software engineer.

    - complaining that it's breaking your system

    ah, you're missing the point by focussing on the background and context.
    also, i didn't "complain" (despite it causing distress for some considerable number of years), i would have said, "i have a complaint" and stopped
    at that, rather than moved on to describe the details of what the root cause is.

    Could you care to explain? This makes absolutely no sense.

    i could - but doing so is a distraction. i would prefer to solicit some responses that acknowledge that this is an actual problem that needs
    solving.

    Bryan's message unfortunately is a repeat of prior experiences
    in reporting this issue over the years.

    By the way, since you're never doing "apt-get dist-upgrade", it means
    your system is full of security issues that aren't getting fixed.
    Hopefully, the computer system(s) you're talking about isn't connected
    to internet, right?

    indeed. it's a development laptop on which i am critically reliant for financial income and for furthering free software.

    Brian May wrote:

    A rolling type update might be convenient, but it is not supported by
    Debian, and has not been supported by Debian in sometime. There are complexities in such an arrangement, and it is difficult to test such arrangements will work as expected. Such an arrangement is not
    guaranteed or tested to work.

    Brian: as an experienced debian systems administrator and python
    developer I'm not asking for help with either, and over the past 20 years
    have successfully learned and deployed the techniques required to
    keep a rolling system operational, in order to keep my business
    operational and not lose money due to massive downtime either
    from having to transfer 10 million source code files from laptop to
    laptop, or from having to perform disaster recovery due to breakage.

    like Thomas, you are missing the point by focussing on the context
    and background material rather than focussing on the problem.

    this was also the tack taken by others when i explained the problem:
    "it's your own fault, this isn't supported, therefore we don't have to
    do anything, therefore it's not a bug, therefore the bugreport is
    summarily dismissed".

    in my report, if you read it again carefully, i specifically stated that *multiple debian maintainers* are receiving *multiple complaints* of
    breakage and disruption due to the standard debhelper-python
    build system not following the "safe" practice outlined by
    numpy and sci-py.

    i can only surmise that they probably don't want to say anything
    because the message being sent to them on summary closure
    of bugreports is that, well, "nobody cares".

    what i can say is that they're getting pretty fed up of constantly
    having to close bugreports, every time someone raises this
    issue, that's pretty clear from what they didn't say, if you know
    what i mean.

    once this is accepted as actually being a problem that needs solving,
    rather than being avoided because "it's not supported, you're on your
    own, not our problem", i am happy to help advise and discuss
    solutions.

    given that this has been ongoing for 10 years now i have been giving
    it some thought for a long, long time.

    however before getting to a discussion of solutions i would prefer
    to see acknowledgement that it is recognised as a problem that
    people actively wish to see fixed. otherwise i will have to wait patiently
    for the next disaster situation to occur, or, sadly, after 20 years of
    using debian, and remaining loyal to it despite maltreatment, i will
    have to find an alternative distro.

    what is stopping me from doing that is the severe financial consequences involved and risk to myself and my family due to my income being
    far below average. the downtime that would result is a huge risk,
    plus learning a new distro also compromises my effectiveness which
    also results in further lost income.

    what i am saying is: this is serious - i am effectlvely financially
    trapped and critically dependent on the decisions that you make,
    even though i am not paying you money for the work that you do.

    l.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jeremy Stanley@21:1/5 to Luke Kenneth Casson Leighton on Mon May 17 14:30:02 2021
    On 2021-05-17 07:10:39 +0100 (+0100), Luke Kenneth Casson Leighton wrote:
    (apologies i forgot to say, please do cc me
    [...]

    Done.

    a dist-upgrade to debian / testing - a way to obtain the latest
    variants of critical software - frequently resulted in massive
    breakage.

    i quickly learned never to attempt that again given the massive
    disruption it caused to me being able to earn money as a software
    engineer.
    [...]

    You're probably just going to see this as further confirmation of
    your opinion, or yet another person telling you that you're doing it
    wrong, but as someone who also writes rather a lot of Python
    programs for a living I learned long ago to not develop against the
    "system" on any platform. I use sid/unstable for my development
    systems, but I consider the python3 package in it to have two uses:
    Testing new packages of software which are targeting a future Debian
    stable release, and running other packaged things which are already
    part of Debian.

    For software development work, I compile my own Python interpreters
    and libraries, because I need to develop against a variety of
    different versions of these, sometimes in chroots, to be able to
    target other distros and releases thereof. I keep all these in my
    homedir or otherwise outside of normal system-wide installation
    paths. Bear in mind that this isn't just a Debian problem, for
    decades Red Hat has also advised programmers not to rely on their /usr/bin/python for custom development because it is patched and
    tuned specifically for running system applications packaged for
    their distro and not intended as a general-purpose Python
    distribution.
    --
    Jeremy Stanley

    -----BEGIN PGP SIGNATURE-----

    iQKTBAABCgB9FiEEl65Jb8At7J/DU7LnSPmWEUNJWCkFAmCiX+VfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDk3 QUU0OTZGQzAyREVDOUZDMzUzQjJFNzQ4Rjk5NjExNDM0OTU4MjkACgkQSPmWEUNJ WCkOCA//WduEtU/MxoXIv3pZ6b7dJ5Rx6kwzddrBcY0CwH9w4C0omkE7TtPqwXoq jCSzsLoeFim8I2xtzMu7dRjfK0bWye5EH+iH65LuLCZCrezYro9JDSVYfW/x7fIA wMfHx0U3FsLKGdIF9MlSqVRxODBCWm9ahbxwLMPb3vV+oN0VitcS1qyWFD9vGA+/ XXTQ8WsXQQ3hm3MbVYT5CXZKx4PWQMuRox7VMRnbqpT+PAipPjf69PcD1PaAbHt7 XjuWuoniQoNBZ+gvdwRXkr8hAy3jxlqfGz05LsSpr+FKsYOjNU/oUIhCH6Byq2VX 9sfVjxBZkg4NP91f7Qn9J0d5fCX5e8R0OdVggis1ThjXOuV8hVtRQnMf/5kjnBz7 dJTUIRMeKwbic/Jx0KwkdXnn5CGxFpEgiFfhAgqBKxuuruJL7Tgp3vpov1KSowb3 uDp6LFQnz80vfN9PEWbY5jo8K0npIkZIf1GPG0Q+VFXDQtfg/4wvobyglzEt9iAe LtUff01+TN38HzJglgZuCcQXl0DYtWhBByZl1E6wmAYquxXB1Dq6t7LswuUmRUL6 lXxpNatCUgshsw6h/L/zrva53u0xLwk4Kr5D6KVjTT68dbHWlR+VEAu2EWlWdh1S UMdjJuiXK8y3QLslRkg7FIwBX+3AKYtN0KHfPye+lv/PY+hIln4=
    =1nZA
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32
  • From Brian May@21:1/5 to Jeremy Stanley on Tue May 18 00:00:02 2021
    Jeremy Stanley <fungi@yuggoth.org> writes:

    For software development work, I compile my own Python interpreters
    and libraries, because I need to develop against a variety of
    different versions of these, sometimes in chroots, to be able to
    target other distros and releases thereof. I keep all these in my
    homedir or otherwise outside of normal system-wide installation
    paths. Bear in mind that this isn't just a Debian problem, for
    decades Red Hat has also advised programmers not to rely on their /usr/bin/python for custom development because it is patched and
    tuned specifically for running system applications packaged for
    their distro and not intended as a general-purpose Python
    distribution.

    There are tools that will help you do this. e.g. pyenv, asdf, etc.

    But, yes, for my own software development, I gave up trying to use
    Debian python packages years ago, because I found I was spending way
    more time trying to package the required dependencies (and dependencies
    of dependencies of dependencies) then I spending developing my
    application. This gets even worse if you want to support rpm based distributions. Plus the fact that I was becoming increasingly concerned
    that if I upgraded library python3-L for application A, I might
    accidentally break applications B, C, and D also, and testing/fixing
    everything at the same time not really feasible. So now I use asdf + poetry/virtualenvs now, plus Docker for distribution.

    As a result, I can upgrade the dependencies in each application one
    application at a time. If an application breaks, I just need to fix that
    one application, without breaking everything else at the same time. I
    can upgrade the OS distribution, and not be concerned (mostly anyway)
    that it will break my applications in weird and wonderful ways that I
    could not predict.

    This isn't so useful for distribution of Python based desktop
    applications, but I don't do a lot of that (and probably would be
    looking for a good off-the-shelf solution if I did).
    --
    Brian May <bam@debian.org>

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)