Hello,
I wonder that today was the first day I stumbled over PEP668 / pipx and
the "externally-managed-environment" problem. I migrated from Debian 11
to 12.

I'm developing some Python packages; applications and libraries. I never
used virtual environments and wouldn't like to start with it.
But I do use "--editable" for my packages. I wonder how I should go on
with using "pipx" and "--editable" mode because I had some problems with
it I couldn't fix following PEP668 but only with using "--break-system-packages". And PEP668 do not mention editable-mode.

The two example packages I use here are public available and clonable
(is this a word?) via git.

Let's start with a command line application named "hyperorg" (https://codeberg.org/buhtz/hyperorg).
I tried to install it via "pipx install -e .[develop]". It's
pyproject.toml has a bug: A missing dependency "dateutil". But
"dateutil" is not available from PyPi for Python 3.11 (the default in
Debian 12). But thanks to great Debian they have a "python3-dateutil"
package. I installed it.
But "hyperorg" do ignore it and can't import it. Of course I
re-installed hyperorg via pipx.
Installing it via "pip" and "--break-system-packages" it works well and
do see the "python3-dateutil" package.

The second example is a "library" named "buhtzology" (https://codeberg.org/buhtz/buhtzology).
Here it seems that "pipx" do refuse to install it in the first place
("pipx install -e .[develop]") and gave me this message.

"No apps associated with package buhtzology. Try again with
'--include-deps' to include apps of dependent packages, which are listed above. If you are attempting to install a library, pipx should not be
used. Consider using pip or a similar tool instead."

What is the difference between an "app" (it is a google/apple marketing
term used for software with user interaction. Just name it application.)
and a "library" in context of pip(x)? Why does it count?
I tried again with "pipx install --include-deps -e .[develop]" and got
an output I'm not sure if this is an error or a success message. I
assume it is an error because I'm not able to "import buhtzology". It
seems like that the package was not installed.

⚠️ Note: normalizer was already on your PATH at /usr/bin/normalizer
⚠️ Note: f2py was already on your PATH at /usr/bin/f2py
⚠️ Note: f2py3 was already on your PATH at /usr/bin/f2py3
⚠️ Note: f2py3.11 was already on your PATH at /usr/bin/f2py3.11
⚠️ Note: py.test was already on your PATH at /usr/bin/py.test
⚠️ Note: pytest was already on your PATH at /usr/bin/pytest
⚠️ File exists at /home/user/.local/bin/pycodestyle and points to /home/user/.local/bin/pycodestyle, not /home/user/.local/pipx/venvs/buhtzology/bin/pycodestyle. Not modifying.
⚠️ File exists at /home/user/.local/bin/ruff and points to /home/user/.local/bin/ruff, not /home/user/.local/pipx/venvs/buhtzology/bin/ruff. Not modifying.
installed package buhtzology 0.2.0.dev0, installed using Python 3.11.2
These apps are now globally available
- f2py
- f2py3
- f2py3.11
- fonttools
- normalizer
- py.test
- pyftmerge
- pyftsubset
- pytest
- tabulate
- ttx
- pycodestyle (symlink missing or pointing to unexpected location)
- ruff (symlink missing or pointing to unexpected location)
done! ✨ 🌟 ✨

BUT!

$ python3 -c "import buhtzology"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'buhtzology'

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

I never
used virtual environments and wouldn't like to start with it.

There's your problem - everything else is a result of this. There is
just no nice way to work with a combination of pypi, apt-get and
system-wide python.

Everyone uses virtual environments.

Sorry.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Dear Rimu,

thanks for your reply. I'm willing to learn for sure.

On 2023-09-16 14:17 Rimu Atkinson via Python-list
<python-list@python.org> wrote:

There's your problem - everything else is a result of this. There is
just no nice way to work with a combination of pypi, apt-get and
system-wide python.

Everyone uses virtual environments.

It is nothing bad about using virtual environments but also not about
not using them. In my own work I haven't see a use case where I needed
them. And I expect that some day I'll encounter a use case for it. This
here is not about pro and cons of virtual environments.

Please explain how the two problems I explained are influenced by not
using virtual environments.

As I explained I tried to use "pipx" (the X at the end!) because it
seems to be the recommended way today because of PEP668. And to my
knowledge pipx do use a virtual environment in the back. Nice.

But it did not work the way I expected.

I don't say that pipx is wrong or buggy. I assume I misunderstood
something in context of PEP668 and in the use of pipx. That is why I'm
asking here.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Am Sat, Sep 16, 2023 at 02:17:19PM +1200 schrieb Rimu Atkinson via Python-list:

Everyone uses virtual environments.

Umm, like, no.

Karsten
--
GPG 40BE 5B0E C98E 1713 AFA6 5BC0 3BEA AC80 7D4F C89B

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

It is nothing bad about using virtual environments but also not about
not using them. In my own work I haven't see a use case where I needed
them. And I expect that some day I'll encounter a use case for it. This
here is not about pro and cons of virtual environments.

You are in a use case where you need them, right now :) When you
understand the benefits of virtual environments you will understand what
I meant by that.

Please explain how the two problems I explained are influenced by not
using virtual environments.

The first problem can be avoided because virtual environments can use a different version of python than the system one. If you need an earlier
version of python then you can use it instead.

The second problem can be avoided because virtual environments exist in
a part of the file system that you have write access to, so you don't
need to use sudo to install packages. Your main user account does not
have write access to /usr/bin.

Also when a virtual environment is activated the path to it's packages
is a part of that environment so your code will always be able to import
the packages you want.

It's much easier to understand if you try it for yourself. Google has
many excellent resources, here is one https://www.freecodecamp.org/news/how-to-setup-virtual-environments-in-python/

Best of luck :)

R

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 9/16/2023 7:57 PM, Rimu Atkinson via Python-list wrote:

It is nothing bad about using virtual environments but also not about
not using them. In my own work I haven't see a use case where I needed
them. And I expect that some day I'll encounter a use case for it. This
here is not about pro and cons of virtual environments.

You are in a use case where you need them, right now :) When you
understand the benefits of virtual environments you will understand what
I meant by that.

Please explain how the two problems I explained are influenced by not
using virtual environments.

The first problem can be avoided because virtual environments can use a different version of python than the system one. If you need an earlier version of python then you can use it instead.

I have multiple versions of Python on both Windows and Linux machines.
I don't have to use the system version. On Windows the "py" launcher
can launch any Python version on your system.

The second problem can be avoided because virtual environments exist in
a part of the file system that you have write access to, so you don't
need to use sudo to install packages. Your main user account does not
have write access to /usr/bin.

And therefore on Linux pip will install packages as --user even if the
option isn't specified. On Windows, one would just use --user routinely.

Also when a virtual environment is activated the path to it's packages
is a part of that environment so your code will always be able to import
the packages you want.

Whenever a version of Python is launched its system path is set up so
that its own packages are used. The one thing that can cause problems
is programs in the Python Scripts directory, which may not be on the
path for a particular version of Python. A virtual environment will
take case of that, but I almost never run scripts in that directory so
it's not an issue for me.

I think there are situations where a venv is useful, but not because of
the points you have asserted here. For myself, I find that after a
while, I tend to forget what I set up the various venvs for, and what
state they are in. So I have stopped using them. I just put up with
having more packages in /site-packages than a particular applications
needs. If I want to, say, work on code in a git clone's directory tree,
I launch Python using a batch file that sets the PYTHONPATH to point there.

It's much easier to understand if you try it for yourself. Google has
many excellent resources, here is one https://www.freecodecamp.org/news/how-to-setup-virtual-environments-in-python/

Best of luck :)

R

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Thanks for your reply. btw: I do know what a virtual environment is and
how its works. No need to explain. And again: I do expect "pipx" to use
virtual environments. So this thread is not about pro and cons of using
virtual environments.

On 2023-09-17 11:57 Rimu Atkinson via Python-list
<python-list@python.org> wrote:

The first problem can be avoided because virtual environments can use
a different version of python than the system one. If you need an
earlier version of python then you can use it instead.

But as I explained I used pipx to install it. So virtual environments
are used in that case.
Maybe I made not the problem clear enough: The problem is that the
installed application ("hyperorg") did not see the system package "python3-dateutils".

How can I solve this without(!) using another Python version?
The behavior that the pipx-installed application is not able to see
system python packages (installed via "apt install python3-dateutils")
is unexpected to me.

Maybe there is a good reason for that? Explain it please?
Maybe I did something wrong? How can I make it right?
Maybe it is a bug or a use case that is not covered by current version
of pipx or Debian 12?

The second problem can be avoided because virtual environments exist
in a part of the file system

Also here I assume I didn't make my problem clear enough.
I tried to install a library (from source) with "pipx". But it did not
work somehow (no clear error message). The global python interpreter
did not see that library.

Why?

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

+1 843 620 1451 Hash Online Shop Europe


+1 843 620 1451
Sales@hashandcannabis.com
Los Angeles , California (USA)

Hashish is produced from the resin glands (trichomes) of the cannabis plant. These trichomes are collected, processed, and compacted into a solid form, creating hash. The result is a highly potent substance known for its unique aroma, flavor, and effects.

Why Buy Hash Online?
Purchasing hash online provides several advantages. It offers convenience, a wider selection, and access to reputable suppliers. Moreover, online dispensaries often provide detailed information about their products, allowing you to make informed decision

Legality of Hashish in Europe
Before you buy hash online, it's crucial to be aware of the legal status of hashish in your European country. Laws regarding cannabis products vary, and staying compliant with local regulations is essential.

Traditional Hash
This is the classic form of hash, typically found in brick or slab form. It has a distinctive aroma and a smooth, earthy flavor.

Bubble Hash
Bubble hash is created using ice water and agitation to separate the trichomes from the plant material. It's known for its purity and potency.

Afghan Hash
Afghan hash is renowned for its strong, sedative effects and earthy taste. It's often dark and sticky in texture.

Moroccan Hash
Moroccan hash is known for its sweet, spicy flavor and a mellow, relaxing high. It's usually light in color and has a crumbly texture.

How to Order Hash Online
Ordering hashish from HashandCannabis.com is a straightforward process. Simply browse their selection, add your desired products to the cart, and follow the checkout instructions. They accept various payment methods, making it convenient for customers
worldwide.

https://hashandcannabis.com/product-category/buy-cannabis-oil-online/ https://hashandcannabis.com/product-category/buy-vape-cartridges-online/ https://hashandcannabis.com/product-category/buy-concentrates-online/ https://hashandcannabis.com/product-category/buy-hashish-online/ https://hashandcannabis.com/product-category/buy-hybrid-strains-online/ https://hashandcannabis.com/product-category/buy-indica-strains-online/ https://hashandcannabis.com/product-category/buy-moon-rock-eu-online/ https://hashandcannabis.com/product-category/buy-sativa-strains-online/ https://hashandcannabis.com/product-category/buy-shatter-wax-online/

Hash Culture in Europe
Hash has a long-standing cultural significance in Europe, and it continues to be enjoyed by people from all walks of life. Learning about its cultural context can deepen your appreciation for this substance.

Buying hash online in Europe can be a rewarding experience when approached with knowledge and caution. Remember to research the legalities in your region, choose a reputable seller, and consume responsibly. With these guidelines, you can enjoy the world
of hash safely and responsibly.

buy cheap hash online uk +1 843 620 1451
deep purple cannabis +1 843 620 1451
hash online shop +1 843 620 1451
Hash UK Delivery +1 843 620 1451
how to buy hash +1 843 620 1451
buy real hash online +1 843 620 1451
hash dispensary +1 843 620 1451
hash deep purple +1 843 620 1451
deep purple hash +1 843 620 1451
Hash Online Shop Europe +1 843 620 1451

+1 843 620 1451
Sales@hashandcannabis.com
Los Angeles , California (USA)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2023-09-15 14:15:23 +0000, c.buhtz--- via Python-list wrote:

I tried to install it via "pipx install -e .[develop]". It's pyproject.toml has a bug: A missing dependency "dateutil". But "dateutil" is not available from PyPi for Python 3.11 (the default in Debian 12). But thanks to great Debian they have a "python3-dateutil" package. I installed it.

Sidenote:
PyPI does have several packages with "dateutil" in their name. From the
version number (2.8.2) I guess that "python-dateutil" is the one
packaged in Debian 12.

This can be installed via pip:

% lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux 12 (bookworm)
Release: 12
Codename: bookworm

(dateutil) % pip install python-dateutil
Collecting python-dateutil
Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 247.7/247.7 kB 3.1 MB/s eta 0:00:00
Collecting six>=1.5
Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Installing collected packages: six, python-dateutil
Successfully installed python-dateutil-2.8.2 six-1.16.0

hp

--
_ | Peter J. Holzer | Story must make more sense than reality.
|_|_) | |
| | | hjp@hjp.at | -- Charles Stross, "Creative writing
__/ | http://www.hjp.at/ | challenge!"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEETtJbRjyPwVTYGJ5k8g5IURL+KF0FAmUIBzwACgkQ8g5IURL+ KF17Eg//aeO749A2G0buZ/jtv0dgGO9NYjwa1eUJlqx9nOPQi4KzxHmAV7KqDllh K6T5mK8Vmdtvpxd8axpXAhYqz47VyDCN6TjDo+Y8w1EJ/QMeYg1Jy/sJyIwR7w4l W2ez1mkmtOSH+kh950D4fZVOiejl3l4jEVfYKmGi8Zh6tiWAWTPHzzn8HcSi6Ver XF49lV/8Sxo6OQpcH5gN5/9GRsRbXsUEmsID9shKM9ZN6PyKoOsX63A/oIXMB7WA tgpGY5dD7K5wUUL07GpRuWZQ4oIxyilyRhcjActN5+2oTmi5PcSf37aEwp0EEq86 ckyKaC88KyMFCGwtmLCiZmflwZfyLDvbkMS0cpfZPcErMlwMmNKswOQn9E5d3e4x p8H3NltQYbn+bmBF8jAT9+OFTzP9nFOP4InGWu4TnYNL5QXvaO/JNwxR2rOrDRap vxguDzerXz4aMUhI2Pdpk3IgtNr8nYgtBUL+uh0FmJjOUBKrJQRlvq0U3337qct1 AoH/+IHMasvju0hx3xQ8zWTnts9NgU+xEQlo/RqcCmVCNnFqsEqI6Wgk0wOlfImb 6aO321lt2KDucRtt9ECfiHhgunrp4o97isTB5uasQ6+8dxiSlZIsDhVkm+YZW/Oz RHajQr6p61lFibfsESCWvhAPwSuew8ZHBoLh99T

On 9/18/23 02:16, Peter J. Holzer via Python-list wrote:

On 2023-09-15 14:15:23 +0000, c.buhtz--- via Python-list wrote:

I tried to install it via "pipx install -e .[develop]". It's pyproject.toml >> has a bug: A missing dependency "dateutil". But "dateutil" is not available >> from PyPi for Python 3.11 (the default in Debian 12). But thanks to great
Debian they have a "python3-dateutil" package. I installed it.

Sidenote:
PyPI does have several packages with "dateutil" in their name. From the version number (2.8.2) I guess that "python-dateutil" is the one
packaged in Debian 12.

This can be installed via pip:

It *is* the case that package name is not always equal to importable
name. That certainly occurs in the universe of Python packages on PyPI;
it's if anything much more likely on Linux distributions which have to
share the package name namespace with a lot more than just Python
packages (just for starters, what seems like billions of Perl packages),
so you're even more likely to see names like python-foo or python3-foo
when the thing you import is foo. That has nothing to do virtualenvs,
of course.

The use of a virtualenv for a project actually makes it more likely that
you discover unstated dependencies, which is generally a good thing.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 9/18/23 12:56, c.buhtz--- via Python-list wrote:

On 2023-09-18 10:16 "Peter J. Holzer via Python-list" <python-list@python.org> wrote:

On 2023-09-15 14:15:23 +0000, c.buhtz--- via Python-list wrote:

I tried to install it via "pipx install -e .[develop]". It's
pyproject.toml has a bug: A missing dependency "dateutil". But
"dateutil" is not available from PyPi for Python 3.11 (the default
in Debian 12). But thanks to great Debian they have a
"python3-dateutil" package. I installed it.

This can be installed via pip:

I'm aware of this. But this is not the question.

I would like to know and understand why my via "pipx" installed package "hyperorg" is not able to see the systems packages installed via "apt
install python3-dateutils"?

Is this the usual behavior? Is this correct?

Yes. By default, the virtualenv contains just what you've installed.
It's designed to give you tight control over what's installed, so you
can track dependencies, avoid accidental inclusions, etc. As usual, you
don't have to accept the default. For example, for the venv module:

usage: venv [-h] [--system-site-packages] [--symlinks | --copies] [--clear]
[--upgrade] [--without-pip] [--prompt PROMPT] [--upgrade-deps]
ENV_DIR [ENV_DIR ...]

Creates virtual Python environments in one or more target directories.

positional arguments:
ENV_DIR A directory to create the environment in.

options:
-h, --help show this help message and exit
--system-site-packages
Give the virtual environment access to the system
site-packages dir.
...

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2023-09-18 10:16 "Peter J. Holzer via Python-list"
<python-list@python.org> wrote:

On 2023-09-15 14:15:23 +0000, c.buhtz--- via Python-list wrote:

I tried to install it via "pipx install -e .[develop]". It's
pyproject.toml has a bug: A missing dependency "dateutil". But
"dateutil" is not available from PyPi for Python 3.11 (the default
in Debian 12). But thanks to great Debian they have a
"python3-dateutil" package. I installed it.

This can be installed via pip:

I'm aware of this. But this is not the question.

I would like to know and understand why my via "pipx" installed package "hyperorg" is not able to see the systems packages installed via "apt
install python3-dateutils"?

Is this the usual behavior? Is this correct?
What is the design decision behind it?

Is it really the intention of PEP668 that pipx-installed packages are
not allowed to use system packages?

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 9/18/2023 2:56 PM, c.buhtz--- via Python-list wrote:

On 2023-09-18 10:16 "Peter J. Holzer via Python-list" <python-list@python.org> wrote:

On 2023-09-15 14:15:23 +0000, c.buhtz--- via Python-list wrote:

I tried to install it via "pipx install -e .[develop]". It's
pyproject.toml has a bug: A missing dependency "dateutil". But
"dateutil" is not available from PyPi for Python 3.11 (the default
in Debian 12). But thanks to great Debian they have a
"python3-dateutil" package. I installed it.

This can be installed via pip:

I'm aware of this. But this is not the question.

I would like to know and understand why my via "pipx" installed package "hyperorg" is not able to see the systems packages installed via "apt
install python3-dateutils"?

Is this the usual behavior? Is this correct?
What is the design decision behind it?

Is it really the intention of PEP668 that pipx-installed packages are
not allowed to use system packages?

One way this could happen is if the hyperorg package got installed by a different version of Python than the system version. I've lost track of
just how your installation is set up so this may not apply. But, for
example, if the system is using Python 3.10.x and you installed hyperorg
using Python 3.11, that would be such a situation. The apt-installed
packages will get installed into the system python directories, rather
than to the Python 3.11 directories.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2023-09-18 18:56:35 +0000, c.buhtz--- via Python-list wrote:

On 2023-09-18 10:16 "Peter J. Holzer via Python-list" <python-list@python.org> wrote:

On 2023-09-15 14:15:23 +0000, c.buhtz--- via Python-list wrote:

I tried to install it via "pipx install -e .[develop]". It's pyproject.toml has a bug: A missing dependency "dateutil". But
"dateutil" is not available from PyPi for Python 3.11 (the default
in Debian 12). But thanks to great Debian they have a
"python3-dateutil" package. I installed it.

This can be installed via pip:

I'm aware of this.

You wrote:

"dateutil" is not available from PyPi for Python 3.11

That's quite a curious thing to write if you are aware that dateutil is
in fact available from PyPi for Python 3.11.

But this is not the question.

I know. That's why I labeled my comment as a "Sidenote".

hp

--
_ | Peter J. Holzer | Story must make more sense than reality.
|_|_) | |
| | | hjp@hjp.at | -- Charles Stross, "Creative writing
__/ | http://www.hjp.at/ | challenge!"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEETtJbRjyPwVTYGJ5k8g5IURL+KF0FAmUK6PYACgkQ8g5IURL+ KF3k1Q/+PyYmNrExobH3MMvwwKPhL72XzHGT6nhctTp85dK4QFvqsM3wxf0nyDKQ t8zmHkcShJCwZiI92PdGbSw9kB1XL3s00532NgFRSeT2trscG6RiTXVxCo9GB5X6 1FoywlQ9loSt4pfLgfRdEHr7sqsx8WMB8jAaUwezb7VTQ+qtE8nHTowyrXEZSXwK xTVn23a4F0Ac4B7Q3FaPHdU3m/lQSnYy6foV7ZksH6yfOnc+y0ve5wh5IFnlJnYe IVtP0QsslzpKqJAt0SMVmFkT4cef3raFlc7r1Zd1ZA20CI/AZ347tgK12a63jeYL iKXjdgR13xSK+f3jkPDXw6pVeFUQAC1cSk8UmZ2txGwHFU8Sg3+sQjpovB/LyU0j cHJ3dtkrJBJg9P7TxKTjGuaLKBoCOWBANLuzlYe82Fq0qAQglTh5j6yaLhC7PkVB DU075ppKDEjwizzcg5kbIhDir09puXP8cjk2MOXkTVgRwA+/Y3VuqvhhFgF1rZXI I8weVW8BkZmae2Rus4eHxIO6lPaU3PP3Lxjowz3fo9/5CIZr6EzNA6UhQNIgRgf+ Eh1xoTkB4SU0v9WBOPQoZ1bpXKI08dp9eL6qcVHR1bjlHRIU5lLRIbPlrFxsGoVY h7YXVwcmva7dkob/Aq+g6p0RBSJV8YThymInpIN

Dear Peter,

maybe we have a missunderstanding.

Am 20.09.2023 14:43 schrieb Peter J. Holzer via Python-list:

"dateutil" is not available from PyPi for Python 3.11

That's quite a curious thing to write if you are aware that dateutil is
in fact available from PyPi for Python 3.11.

Do I miss something here?

See https://pypi.org/project/dateutils/ and also the open Issue about
the missing support for Python 3.11 https://github.com/dateutil/dateutil/issues/1233 ?

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)