I have a couple of systems which used to have python2 as well as
python3 but as Ubuntu and Debian verions have moved on they have
finally eliminated all dependencies on python2.
So they now have only python3 and there is no python executable in
PATH.
There's still both /usr/bin/pip and /usr/bin/pip3 but they're
identical so presuably I can now simply use pip and it will be a
python3 pip.
So, going on from this, how do I do the equivalent of "apt update; apt upgrade" for my globally installed pip packages?
...
So, going on from this, how do I do the equivalent of "apt update; apt >upgrade" for my globally installed pip packages?
Chris Green wrote at 2023-11-2 10:58 +0000:
...
So, going on from this, how do I do the equivalent of "apt update; apt >>upgrade" for my globally installed pip packages?
`pip list -o` will tell you for which packages there are upgrades
available.
`pip install -U ...` will upgrade packages.
Be careful, though.
With `apt`, you usually have (`apt`) sources representing a consistent package universe. Someone tests that package upgrades in this
universe do not break other packages (in this universe).
Because of this, upgrading poses low risk.
`PyPI` does not guarantes consistency. A new package version
may be incompatible to a previous one -- and with other
package you have installed.
I do not think that you would want to auto-upgrade all installed
packages.
On 2023-11-02, Chris Green <cl@isbd.net> wrote:
I have a couple of systems which used to have python2 as well as
python3 but as Ubuntu and Debian verions have moved on they have
finally eliminated all dependencies on python2.
So they now have only python3 and there is no python executable in
PATH.
There's still both /usr/bin/pip and /usr/bin/pip3 but they're
identical so presuably I can now simply use pip and it will be a
python3 pip.
So, going on from this, how do I do the equivalent of "apt update; apt upgrade" for my globally installed pip packages?
I'm not sure what that question has to do with everything that preceded
it, but you don't want to install python packages globally using pip.
Either install them with 'apt', or install them in a virtual environment.
On 2023-11-02, Dieter Maurer <dieter@handshake.de> wrote:
Chris Green wrote at 2023-11-2 10:58 +0000:
...
So, going on from this, how do I do the equivalent of "apt update; apt >>upgrade" for my globally installed pip packages?
`pip list -o` will tell you for which packages there are upgrades available.
`pip install -U ...` will upgrade packages.
Be careful, though.
With `apt`, you usually have (`apt`) sources representing a consistent package universe. Someone tests that package upgrades in this
universe do not break other packages (in this universe).
Because of this, upgrading poses low risk.
`PyPI` does not guarantes consistency. A new package version
may be incompatible to a previous one -- and with other
package you have installed.
I do not think that you would want to auto-upgrade all installed
packages.
Indeed. What you're describing is a very unfortunate failing of pip. 'Upgrade' doesn't even follow requirements when you tell it what to
upgrade - e.g. if you do "pip install foo" and foo requires "bar<2"
so you end up with:
Package Version
---------------------- ---------
foo 1.0.0
bar 1.2.0
and then a new version 1.3.0 of bar comes out and you do
"pip install -U foo", pip will not upgrade bar even though it could
and should, because foo is already at the latest version so pip won't
even look at its dependencies.
Indeed there is no way of knowing that you should upgrade bar without manually following all the dependency graphs. ("pip list -o" will tell
you there's a newer version, but that isn't the same - e.g. if the new version of bar was 2.0.0 then "pip list -o" will list it, but you should
not upgrade to it.)
You can do "pip install -I foo", which will pointlessly reinstall foo
and then presumably upgrade bar as well, thus probably getting to the
right result via a rather roundabout route, but I'm not sure if that
does indeed work properly and if it is a reliable and recommended way
of doing things.
Jon Ribbens <jon+usenet@unequivocal.eu> wrote:
On 2023-11-02, Chris Green <cl@isbd.net> wrote:
I have a couple of systems which used to have python2 as well as
python3 but as Ubuntu and Debian verions have moved on they have
finally eliminated all dependencies on python2.
So they now have only python3 and there is no python executable in
PATH.
There's still both /usr/bin/pip and /usr/bin/pip3 but they're
identical so presuably I can now simply use pip and it will be a
python3 pip.
So, going on from this, how do I do the equivalent of "apt update; apt
upgrade" for my globally installed pip packages?
I'm not sure what that question has to do with everything that preceded
it, but you don't want to install python packages globally using pip.
Either install them with 'apt', or install them in a virtual environment.
Why in a virtual environment? When I install a package whether from
apt or from pip I want everyone/everything on my system to be able to
use it.
I do only install a few things using pip.
Jon Ribbens <jon+usenet@unequivocal.eu> wrote:
On 2023-11-02, Dieter Maurer <dieter@handshake.de> wrote:
Chris Green wrote at 2023-11-2 10:58 +0000:
...
So, going on from this, how do I do the equivalent of "apt update; apt
upgrade" for my globally installed pip packages?
`pip list -o` will tell you for which packages there are upgrades
available.
`pip install -U ...` will upgrade packages.
Be careful, though.
With `apt`, you usually have (`apt`) sources representing a consistent
package universe. Someone tests that package upgrades in this
universe do not break other packages (in this universe).
Because of this, upgrading poses low risk.
`PyPI` does not guarantes consistency. A new package version
may be incompatible to a previous one -- and with other
package you have installed.
I do not think that you would want to auto-upgrade all installed
packages.
Indeed. What you're describing is a very unfortunate failing of pip.
'Upgrade' doesn't even follow requirements when you tell it what to
upgrade - e.g. if you do "pip install foo" and foo requires "bar<2"
so you end up with:
Package Version
---------------------- ---------
foo 1.0.0
bar 1.2.0
and then a new version 1.3.0 of bar comes out and you do
"pip install -U foo", pip will not upgrade bar even though it could
and should, because foo is already at the latest version so pip won't
even look at its dependencies.
Indeed there is no way of knowing that you should upgrade bar without
manually following all the dependency graphs. ("pip list -o" will tell
you there's a newer version, but that isn't the same - e.g. if the new
version of bar was 2.0.0 then "pip list -o" will list it, but you should
not upgrade to it.)
You can do "pip install -I foo", which will pointlessly reinstall foo
and then presumably upgrade bar as well, thus probably getting to the
right result via a rather roundabout route, but I'm not sure if that
does indeed work properly and if it is a reliable and recommended way
of doing things.
It is a bit of a minefield isn't it. I try to minimise my use of
packages installed using pip for this very reason. Maybe the safest
route would simply be to uninstall everything and then re-install it.
I have a couple of systems which used to have python2 as well as
python3 but as Ubuntu and Debian verions have moved on they have
finally eliminated all dependencies on python2.
So they now have only python3 and there is no python executable in
PATH.
So, going on from this, how do I do the equivalent of "apt update; apt upgrade" for my globally installed pip packages
On 2023-11-02, Chris Green <cl@isbd.net> wrote:
Jon Ribbens <jon+usenet@unequivocal.eu> wrote:
On 2023-11-02, Chris Green <cl@isbd.net> wrote:
I have a couple of systems which used to have python2 as well as
python3 but as Ubuntu and Debian verions have moved on they have
finally eliminated all dependencies on python2.
So they now have only python3 and there is no python executable in
PATH.
There's still both /usr/bin/pip and /usr/bin/pip3 but they're
identical so presuably I can now simply use pip and it will be a
python3 pip.
So, going on from this, how do I do the equivalent of "apt update; apt >> > upgrade" for my globally installed pip packages?
I'm not sure what that question has to do with everything that preceded
it, but you don't want to install python packages globally using pip.
Either install them with 'apt', or install them in a virtual environment.
Why in a virtual environment? When I install a package whether from
apt or from pip I want everyone/everything on my system to be able to
use it.
Because pip barely plays well by itself, let alone with other package managers at the same time.
I do only install a few things using pip.
Are they not available in your system's package manager?
I guess you might get away with "sudo -H pip install -U foo"
for a couple of things, if they don't have many dependencies.
Because pip barely plays well by itself, let alone with other package managers at the same time.
I do only install a few things using pip.
Are they not available in your system's package manager?
Am Thu, Nov 02, 2023 at 09:35:43PM -0000 schrieb Jon Ribbens via Python-list:
Regardless of ...
Because pip barely plays well by itself, let alone with other package
managers at the same time.
... being true ...
I do only install a few things using pip.
Are they not available in your system's package manager?
... this clearly often answers to "no" for applications of
any complexity.
Is there a suggested proper path to deal with that (Debian is
of interest to me here) ?
...
Are they not available in your system's package manager?
... this clearly often answers to "no" for applications of
any complexity.
Is there a suggested proper path to deal with that (Debian is
of interest to me here) ?
So they now have only python3 and there is no python executable in
PATH.
FWIW, for this you install the little stub package python-is-python3. Especially if you
want to keep a python2 installation around - "python" will still be python3 in this
case.
Am Thu, Nov 02, 2023 at 04:07:33PM -0600 schrieb Mats Wichmann via Python-list:
So they now have only python3 and there is no python executable in
PATH.
FWIW, for this you install the little stub package python-is-python3.
Especially if you want to keep a python2 installation around -
"python" will still be python3 in this case.
Since you seem knowledgeable in this area: Do you know of a
resource for learning the *canonical* way of packaging a
Python application for installation via apt which
- needs some packages available via apt
- needs some packages only available via pip
- needs some packages newer than what is available via apt
?
On 2023-11-03, Karsten Hilbert <Karsten.Hilbert@gmx.net> wrote:
Am Thu, Nov 02, 2023 at 04:07:33PM -0600 schrieb Mats Wichmann via Python-list:
So they now have only python3 and there is no python executable in
PATH.
FWIW, for this you install the little stub package python-is-python3.
Especially if you want to keep a python2 installation around -
"python" will still be python3 in this case.
Since you seem knowledgeable in this area: Do you know of a
resource for learning the *canonical* way of packaging a
Python application for installation via apt which
- needs some packages available via apt
- needs some packages only available via pip
- needs some packages newer than what is available via apt
?
I suspect the answer to that is that you would have to:
* create packages yourself for the unpackaged dependencies
* create a dependency graph of *every* Python package in the package
repository (whether or not the package is relevant to what you're doing)
* work out what versions of every Python package are required in order
to have a dependency graph that can be successfully resolved, taking
into account the requirements of your new package also
* contact every single maintainer of every single one of the packages
that needs updating and persuade them to update their packages and
reassure them that you are getting all the other package maintainers
to update their packages accordingly and that you have a plan and
that you know what you're doing
... screen fades to black, title card "3 years later", fade in to ...
* publish your package
Jon Ribbens <jon+usenet@unequivocal.eu> wrote:
On 2023-11-03, Karsten Hilbert <Karsten.Hilbert@gmx.net> wrote:Surely it's not that bad, the vast bulk of Debian, Ubuntu and other distributions are installed via systems that sort out dependencies once
Am Thu, Nov 02, 2023 at 04:07:33PM -0600 schrieb Mats Wichmann via Python-list:
So they now have only python3 and there is no python executable in
PATH.
FWIW, for this you install the little stub package python-is-python3.
Especially if you want to keep a python2 installation around -
"python" will still be python3 in this case.
Since you seem knowledgeable in this area: Do you know of a
resource for learning the *canonical* way of packaging a
Python application for installation via apt which
- needs some packages available via apt
- needs some packages only available via pip
- needs some packages newer than what is available via apt
?
I suspect the answer to that is that you would have to:
* create packages yourself for the unpackaged dependencies
* create a dependency graph of *every* Python package in the package
repository (whether or not the package is relevant to what you're doing) >> * work out what versions of every Python package are required in order
to have a dependency graph that can be successfully resolved, taking
into account the requirements of your new package also
* contact every single maintainer of every single one of the packages
that needs updating and persuade them to update their packages and
reassure them that you are getting all the other package maintainers
to update their packages accordingly and that you have a plan and
that you know what you're doing
... screen fades to black, title card "3 years later", fade in to ...
* publish your package
given a particular package's requirements. Python is surely not
unique in its dependency requirements.
Are they not available in your system's package manager?
... this clearly often answers to "no" for applications of
any complexity.
Is there a suggested proper path to deal with that (Debian is
of interest to me here) ?
Yes, as previously mentioned, use virtual environments.
These days they don't even need to be "activated". For package 'foo'
for example you could create /usr/local/lib/foo, under which you would
create a virtual environment and install the 'foo' package inside it,
and then you could do:
ln -s /usr/local/lib/foo/env/bin/foo /usr/local/bin/foo
and then you could just type 'foo' to run it.
* contact every single maintainer of every single one of the packages
that needs updating and persuade them to update their packages and
reassure them that you are getting all the other package maintainers
to update their packages accordingly and that you have a plan and
that you know what you're doing
... screen fades to black, title card "3 years later", fade in to ...
* publish your package
Surely it's not that bad, the vast bulk of Debian, Ubuntu and other distributions are installed via systems that sort out dependencies once
given a particular package's requirements. Python is surely not
unique in its dependency requirements.
Karsten Hilbert wrote at 2023-11-3 14:47 +0100:
...
Are they not available in your system's package manager?
... this clearly often answers to "no" for applications of
any complexity.
Is there a suggested proper path to deal with that (Debian is
of interest to me here) ?
Complex applications may maintain a set of "known workable versions" associated with the application's releases.
They may describe those "known workable versions" in a `pip` constraint file. In this case, you can upgrade to a new application release
by using this constraint file.
Am Fri, Nov 03, 2023 at 01:53:32PM -0000 schrieb Jon Ribbens via Python-list:
Are they not available in your system's package manager?
... this clearly often answers to "no" for applications of
any complexity.
Is there a suggested proper path to deal with that (Debian is
of interest to me here) ?
Yes, as previously mentioned, use virtual environments.
These days they don't even need to be "activated". For package 'foo'
for example you could create /usr/local/lib/foo, under which you would
create a virtual environment and install the 'foo' package inside it,
and then you could do:
ln -s /usr/local/lib/foo/env/bin/foo /usr/local/bin/foo
and then you could just type 'foo' to run it.
This all being nice and well, but:
How does one "fill" that venv with packages from pip during
apt-get install python3-app-of-interest
?
Is the suggested way really to pip-install into this venv
during apt-get install ?
...
do you happen to know where to read up on how to fit a pip
constraint file into a Debian package creation workflow ?
How does one "fill" that venv with packages from pip duringAre they not available in your system's package manager?
... this clearly often answers to "no" for applications of
any complexity.
Is there a suggested proper path to deal with that (Debian is
of interest to me here) ?
Yes, as previously mentioned, use virtual environments.
apt-get install python3-app-of-interest
?
Is the suggested way really to pip-install into this venv
during apt-get install ?
I don't know what you mean by that. But if you install the apt packages
and then create your venv with --system-site-packages then I believe
your venv should be able to see the apt packages and import them.
I know that debian packagers create debian packages
from Python distributions not using the approach sketched above
and likely they have their reasons.
You might want to discuss this on an `apt` related mailing list.
I had just hoped someone here might have a handy pointer for
how to deal with modules having to be installed from pip for
use with an apt-installed python-based application.
I had just hoped someone here might have a handy pointer for
how to deal with modules having to be installed from pip for
use with an apt-installed python-based application.
That just shouldn't happen - such packages are supposed to be dependency-complete within
the packaging universe in question.
Am Mon, Nov 06, 2023 at 02:43:47PM -0700 schrieb Mats Wichmann via Python-list:
I had just hoped someone here might have a handy pointer for
how to deal with modules having to be installed from pip for
use with an apt-installed python-based application.
That just shouldn't happen - such packages are supposed to be dependency-complete within
the packaging universe in question.
Yep, that's the preferable ideal world.
Which doesn't happen (but that's not the fault of anyone
around here, no harm intended).
.From all the posts I gather the answer to my question is
"simply": unpackaged-but-needed modules need to be packaged.
Sysop: | Keyop |
---|---|
Location: | Huddersfield, West Yorkshire, UK |
Users: | 300 |
Nodes: | 16 (2 / 14) |
Uptime: | 76:59:25 |
Calls: | 6,716 |
Files: | 12,247 |
Messages: | 5,357,726 |