I'd like to ask for some help. I'm working on packaging pydevd, whichIf it's just a private library and not a Python module it should be called bar.so.
builds a private .so library. Ordinary extensions built using cython
or similar end up being called "foo.cpython-310-x86_64-linux-gnu.so",
but this library, which is not dependent on the Python version, should presumably be called "bar.x86_64-linux-gnu.so".
Question 1: How do I determine (within Python) the triplet to use when building the library?You don't.
Question 2: How do I determine (within Python) the triplet to use when loading the library at runtime?You don't, but also how are you actually loading it?
The normal way for this is putting it intoI'd like to ask for some help. I'm working on packaging pydevd, which builds a private .so library. Ordinary extensions built using cythonIf it's just a private library and not a Python module it should be called bar.so.
or similar end up being called "foo.cpython-310-x86_64-linux-gnu.so",
but this library, which is not dependent on the Python version, should presumably be called "bar.x86_64-linux-gnu.so".
Question 1: How do I determine (within Python) the triplet to use when building the library?You don't.
Question 2: How do I determine (within Python) the triplet to use when loading the library at runtime?You don't, but also how are you actually loading it?
Well, the upstream wanted to compile two versions of the library, one
for 64 bit architectures and one for 32 bit architectures. I don't
really want to build two different arch libraries in a single build,
because that seems very contrary to the way the Debian architectures
work, and would also limit it to the amd64/i386 architectures for no obviously good reason. I had imagined that if there is some sort of multiarch setup, one might have the amd64 and i386 packages installed simultaneously, hence the need for different names.
On Wed, Jun 08, 2022 at 10:43:57PM +0100, Julian Gilbey wrote:
I'd like to ask for some help. I'm working on packaging pydevd, which builds a private .so library. Ordinary extensions built using cythonIf it's just a private library and not a Python module it should be called bar.so.
or similar end up being called "foo.cpython-310-x86_64-linux-gnu.so",
but this library, which is not dependent on the Python version, should presumably be called "bar.x86_64-linux-gnu.so".
Question 1: How do I determine (within Python) the triplet to use when building the library?You don't.
Question 2: How do I determine (within Python) the triplet to use when loading the library at runtime?You don't, but also how are you actually loading it?
[...]
Well, the upstream wanted to compile two versions of the library, oneThe normal way for this is putting it into
for 64 bit architectures and one for 32 bit architectures. I don't
really want to build two different arch libraries in a single build, because that seems very contrary to the way the Debian architectures
work, and would also limit it to the amd64/i386 architectures for no obviously good reason. I had imagined that if there is some sort of multiarch setup, one might have the amd64 and i386 packages installed simultaneously, hence the need for different names.
/usr/lib/<triplet>/pkgname/foo.so, and according to the code below you'll need the full path at the run time so you indeed need the triplet at both build and run time. You can get the triplet in d/rules, not sure how
should you pass it to the build system as that depends on the build system used. For the run time, https://wiki.debian.org/Python/MultiArch suggests sys.implementation._multiarch (note that you cannot use it during the
build as that would break cross-compilation etc.), not sure if there are better ways.
On Thu, 09 Jun 2022 at 13:03:25 +0500, Andrey Rahmatullin wrote:
The normal way for this is putting it into /usr/lib/<triplet>/pkgname/foo.so, and according to the code below you'll need the full path at the run time so you indeed need the triplet at both build and run time.
You can do something like
handle = dlopen("/usr/${LIB}/pkgname/foo.so", flags);
[...]
Then you'd install the private library into what Autotools would refer to
as ${libdir}/pkgname/foo.so (adjust as necessary for other build systems)
and it will usually end up in the correct place. This assumes that
${libdir} is configured to something like
${exec_prefix}/lib/x86_64-linux-gnu or ${exec_prefix}/lib64 as appropriate for the distribution, but that's normally true anyway, and in particular should be true in debhelper.
The normal way for this is putting it into
/usr/lib/<triplet>/pkgname/foo.so, and according to the code below you'll need the full path at the run time so you indeed need the triplet at both build and run time.
It's always DEB_HOST_MULTIARCH. *_TARGET_* are for Canadian cross. See[...]
Well, the upstream wanted to compile two versions of the library, oneThe normal way for this is putting it into /usr/lib/<triplet>/pkgname/foo.so, and according to the code below you'll need the full path at the run time so you indeed need the triplet at both build and run time. You can get the triplet in d/rules, not sure how
for 64 bit architectures and one for 32 bit architectures. I don't really want to build two different arch libraries in a single build, because that seems very contrary to the way the Debian architectures work, and would also limit it to the amd64/i386 architectures for no obviously good reason. I had imagined that if there is some sort of multiarch setup, one might have the amd64 and i386 packages installed simultaneously, hence the need for different names.
should you pass it to the build system as that depends on the build system used. For the run time, https://wiki.debian.org/Python/MultiArch suggests sys.implementation._multiarch (note that you cannot use it during the
build as that would break cross-compilation etc.), not sure if there are better ways.
Thanks for your help!
OK (and yes, it does require the full path at runtime). What triplet
do I use in d/rules? dpkg-architecture offers 6 different ones: DEB_{BUILD,HOST,TARGET}_{GNU_TYPE,MULTIARCH}? I'm guessing DEB_TARGET_MULTIARCH, but I'm really not certain, so it would be great
to confirm that.
About the location, though: why do compiled Python libraries live in /usr/lib/python3/dist-packages/<pkgname> and not /usr/lib/<triplet>/<pkgname>?That's where the import machinery expects them. They also have special
And is there a good reason not to do the same with this Python-package-specific library?Yes, it's not Python-related.
It's not for general use, so I can't see why I shouldn't put it in the python3 directory with the other compiled Python module libraries.It's not a module.
OK (and yes, it does require the full path at runtime). What triplet
do I use in d/rules? dpkg-architecture offers 6 different ones: DEB_{BUILD,HOST,TARGET}_{GNU_TYPE,MULTIARCH}? I'm guessing DEB_TARGET_MULTIARCH, but I'm really not certain, so it would be great
to confirm that.
About the location, though: why do compiled Python libraries live in /usr/lib/python3/dist-packages/<pkgname> and not /usr/lib/<triplet>/<pkgname>?
And is there a good reason not to do
the same with this Python-package-specific library?
It's not for
general use, so I can't see why I shouldn't put it in the python3
directory with the other compiled Python module libraries.
Sure, it's not necessary to know the installation path at the compile andThe normal way for this is putting it into /usr/lib/<triplet>/pkgname/foo.so, and according to the code below you'll need the full path at the run time so you indeed need the triplet at both build and run time.
You can do something like
handle = dlopen("/usr/${LIB}/pkgname/foo.so", flags);
[...]
Then you'd install the private library into what Autotools would refer to as ${libdir}/pkgname/foo.so (adjust as necessary for other build systems) and it will usually end up in the correct place. This assumes that ${libdir} is configured to something like ${exec_prefix}/lib/x86_64-linux-gnu or ${exec_prefix}/lib64 as appropriate for the distribution, but that's normally true anyway, and in particular should be true in debhelper.
Thanks Simon!
The build system here is the standard Python setup.py, except for this library. That is built by the following script:
---
g++ -m64 -shared -o attach_linux_amd64.so -fPIC -nostartfiles attach.cpp
mv attach_linux_amd64.so ../attach_linux_amd64.so
echo Compiled amd64
---
There's not even an attempt at working out ${libdir} or so on.
The build system here is the standard Python setup.py, except for this library. That is built by the following script:
---
g++ -m64 -shared -o attach_linux_amd64.so -fPIC -nostartfiles attach.cpp
mv attach_linux_amd64.so ../attach_linux_amd64.so
echo Compiled amd64
---
There's not even an attempt at working out ${libdir} or so on.Sure, it's not necessary to know the installation path at the compile and link time. It's only needed at the install time.
On Thu, 09 Jun 2022 at 09:56:42 +0100, Julian Gilbey wrote:
OK (and yes, it does require the full path at runtime). What triplet
do I use in d/rules? dpkg-architecture offers 6 different ones: DEB_{BUILD,HOST,TARGET}_{GNU_TYPE,MULTIARCH}? I'm guessing DEB_TARGET_MULTIARCH, but I'm really not certain, so it would be great
to confirm that.
You'd want DEB_HOST_MULTIARCH here (or use ${LIB} as I mentioned in a previous message to this thread).
[...]
About the location, though: why do compiled Python libraries live in /usr/lib/python3/dist-packages/<pkgname> and not /usr/lib/<triplet>/<pkgname>?
The Python jargon for a native C/C++ library that can be loaded to
provide a Python module is an *extension*.
[...]
And is there a good reason not to do
the same with this Python-package-specific library?
If it isn't a Python extension (cannot be loaded into Python with
"import foo" to provide a module API to Python code) then it would seem inappropriate to put it in the directory that is reserved for Python extensions.
On Thu, Jun 09, 2022 at 08:42:28AM +0100, Julian Gilbey wrote:
I'd like to ask for some help. I'm working on packaging pydevd, which builds a private .so library. Ordinary extensions built using cython or similar end up being called "foo.cpython-310-x86_64-linux-gnu.so", but this library, which is not dependent on the Python version, should presumably be called "bar.x86_64-linux-gnu.so".If it's just a private library and not a Python module it should be called
bar.so.
Question 1: How do I determine (within Python) the triplet to use when building the library?You don't.
Question 2: How do I determine (within Python) the triplet to use when loading the library at runtime?You don't, but also how are you actually loading it?
Well, the upstream wanted to compile two versions of the library, oneThe normal way for this is putting it into
for 64 bit architectures and one for 32 bit architectures. I don't
really want to build two different arch libraries in a single build, because that seems very contrary to the way the Debian architectures
work, and would also limit it to the amd64/i386 architectures for no obviously good reason. I had imagined that if there is some sort of multiarch setup, one might have the amd64 and i386 packages installed simultaneously, hence the need for different names.
/usr/lib/<triplet>/pkgname/foo.so, and according to the code below you'll need the full path at the run time so you indeed need the triplet at both build and run time. You can get the triplet in d/rules, not sure how
should you pass it to the build system as that depends on the build system used. For the run time, https://wiki.debian.org/Python/MultiArch suggests sys.implementation._multiarch (note that you cannot use it during the
build as that would break cross-compilation etc.), not sure if there are better ways.
I got all of the triplets working, but was then stymied when I triedActually, #812228, mentioned in the tag description, was fixed in 2021 so
to specify Multi-Arch: same in the control file. I got a lintian
warning: multi-arch-same-package-calls-pycompile
It seems that since the pybuild system (via dh_python3) adds a
py3compile command to the postinst of the package, then I can't safely
use Multi-Arch: same.
[...]
I got all of the triplets working, but was then stymied when I tried
to specify Multi-Arch: same in the control file. I got a lintian
warning: multi-arch-same-package-calls-pycompile
It seems that since the pybuild system (via dh_python3) adds a
py3compile command to the postinst of the package, then I can't safely
use Multi-Arch: same.
I don't know if this is the case for all python3 Arch: any packagesI think the tag desciption has a good step-by-step explanation why does
with compiled extensions;
the tag exists.
But your package is not a "package with compiled extensions", is it?
are they all effectively Multi-Arch: no? Is this worth thinking aboutWhat do you propose?
in the longer term?
On Sun, Jul 24, 2022 at 06:36:57PM +0100, Julian Gilbey wrote:
I got all of the triplets working, but was then stymied when I triedActually, #812228, mentioned in the tag description, was fixed in 2021 so it's possible that this is no longer a problem.
to specify Multi-Arch: same in the control file. I got a lintian
warning: multi-arch-same-package-calls-pycompile
It seems that since the pybuild system (via dh_python3) adds a
py3compile command to the postinst of the package, then I can't safely
use Multi-Arch: same.
On Sun, Jul 24, 2022 at 11:41:56PM +0500, Andrey Rahmatullin wrote:If it actually ships extensions, the "it should usually get a dependency
[...]
I got all of the triplets working, but was then stymied when I tried
to specify Multi-Arch: same in the control file. I got a lintian warning: multi-arch-same-package-calls-pycompile
It seems that since the pybuild system (via dh_python3) adds a
py3compile command to the postinst of the package, then I can't safely use Multi-Arch: same.
I don't know if this is the case for all python3 Arch: any packagesI think the tag desciption has a good step-by-step explanation why does
with compiled extensions;
the tag exists.
But your package is not a "package with compiled extensions", is it?
Yes, it's got compiled extensions (Cython) in addition to this
non-Python .so library file.
are they all effectively Multi-Arch: no? Is this worth thinking aboutWhat do you propose?
in the longer term?
I think the fix to bug #812228 might have done the job nicely ;-)
On Sun, Jul 24, 2022 at 08:30:42PM +0100, Julian Gilbey wrote:
[...]
are they all effectively Multi-Arch: no? Is this worth thinking about in the longer term?What do you propose?
I think the fix to bug #812228 might have done the job nicely ;-)If it actually ships extensions, the "it should usually get a dependency
on the Python interpreter for the same architecture" part should still
apply as far as I understand it.
| Sysop: | Keyop |
|---|---|
| Location: | Huddersfield, West Yorkshire, UK |
| Users: | 384 |
| Nodes: | 16 (2 / 14) |
| Uptime: | 63:05:11 |
| Calls: | 8,174 |
| Calls today: | 6 |
| Files: | 13,113 |
| Messages: | 5,864,696 |