This is all pretty straightforward. If foo.pc in libfoo-dev references
bar.pc that lives in libbar-dev, then libfoo-dev needs a dependency on libbar-dev, and the missing dependency is a serious bug. That has been
the case for as long as I remember, and doesn't require more long
discussions or policy changes IMO.

Libs.private is a bit different because static linking is not typically
used for Debian packages, but Requires and Requires.private are quite
clear cut.

Cheers,
Julien

On Thu, Dec 09, 2021 at 03:24:27PM +0100, Alexander Traud wrote:

Linux distributions, which have separate packages for developers, like Debian, are not really supported [1] by the developer tool pkg-config
[2]. The problems are C/C++ libraries which depend on other libraries at runtime. Let us pick one, a security library for utilizing DNSSEC:

$ sudo apt install libunbound-dev
$ pkg-config --print-errors --short-errors --cflags libunbound
$ pkg-config --print-errors --short-errors --exists libunbound
Package 'hogweed', required by 'libunbound', not found

1) The error message is misleading because pkg-config talks about the *runtime* package 'hogweed', although (in Debian) a *-dev* package is required to get the .pc file. Should I report that, where?

2) In this case, the package name is not simply an added '-dev' like 'hogweed-dev'. Instead, the developer has to search the contents of all packages [3] for the file 'hogweed.pc'. That is part of the package 'nettle-dev'.

3) Are 'nettle-dev' (and 'libevent-dev', by the way) missing
dependencies of 'libunbound-dev'?

The latter seems to be an ongoing question on debian-devel [4] if the
package includes a static library '.a' as libunbound-dev still does. At
least I found no conclusive answer. Position No: It is not a missing dependency because I can use that package perfectly (as a shared
library) without pkg-config. Position Yes: pkg-config has to be
required, recommended, suggested - for each level of dependency, you
find an argument on debian-devel, see [5][6][7][8].

16 years. This topic has come up for 16 years now, again and again on
the mailing list debian-devel, documenting the uncertainty of package maintainers. Furthermore, bug reports come in, again and again for that topic. Again, libunbound-dev as example for the Debian bug report [9]: Upstream, in the pkg-config .pc file, the maintainer moved the libraries
from 'Requires' to 'Requires.private'. That was the correct approach. However, the maintainer closed the Debian bug report because he falsely believed to have fixed the issue that way.

Requires -> -I/subfolder -lfuu, avoid if possible, see [10] Requires.private -> -I/subfolder , a header includes that header
Libs -> -lfoo, lib itself
Libs.private -> -lfuu, exclusively for static linking
Cflags -> -I/subfolder , lib headers itself and/or
header includes that header

Any idea how to approach this?
a) Leave as is. Do not depend on 'Require.private' automatically because
the package can be used without pkg-config. Only depend on other -devs
if one of those headers is included in a header.
b) Fix/convert. Upstream, move everything into 'Requires.private'. Downstream, in the Debian package, remove 'Requires.private' and convert
to 'Libs.private'; again, except if one of those headers is included in
a header.

These uncertainties create repeated frustration, even for core Debian members. Proposal: Why not decide, or at least give a recommendation on
these questions:

i) pkg-config tool itself: When is it a depend (which dep level?), and
when not? My suggestions: Because I can use a '-dev' package without any tool, because of the FHS, just -lfoo should be needed [11].

ii) .pc file and its 'Require': Leave it or change it? My suggestion:
Report upstream that those libs have to be moved to 'Requires.private'.

iii) .pc file and its 'Require[.private]': Depend on each lib? My
suggestion: Only if the headers include it. For Debian, convert all
others to 'Libs.private' because the built '.a' file (for static
linking) has a known dependency tree. This gets a Debian patch in that
source package.

iv) Cflags: If I got it correctly, back in the year 2005, the cause of
this drama were -I flags [12]; if the header included another header,
and that header included further headers but was not in the root but in
a subfolder, an -I flag *might* be required. For example, the package 'libopusfile-dev' has its header file in '/usr/include/opus/opusfile.h'
with #include <opus_multistream.h> which is part of the package
'libopus-dev' and is not within '/usr/include/' but within '/usr/include/opus/'. Therefore,
$ pkg-config --cflags opusfile
-I/usr/include/opus

After reading, understanding, and researching so much, I really wonder
if this was correct. For example, if the header did
#include <opus/opus_multistream.h>
no include directory would be needed to be figured out via pkg-config. Actually, if the world was like that, I could shelf pkg-config and go
just for linking libraries.

Long story short:
After 16 years of back and forth, several hundred bug reports, still new
ones coming in, and existing reports not fixed completely, I wonder if someone in the Debian world is able to decide this finally and provide
that decision either as policy change, or recommendation, or at least as
a hint for future -dev package maintainers. And please, tell me, what I should do about libunbound-dev; re-open that bug report [9]?

[1] <https://gitlab.freedesktop.org/pkg-config/pkg-config/-/issues/7#note_656054>
[2] <https://people.freedesktop.org/~dbn/pkg-config-guide.html>
[3] <https://packages.debian.org>
[4] <https://lists.debian.org/debian-devel/2008/02/msg01189.html>
[5] <https://lists.debian.org/debian-devel/2004/11/msg00319.html>
[6] <https://www.mail-archive.com/debian-devel@lists.debian.org/msg258710.html>
[7] <https://www.mail-archive.com/debian-policy@lists.debian.org/msg21816.html>
[8] <https://www.mail-archive.com/debian-user@lists.debian.org/msg729223.html>
[9] <https://bugs.debian.org/958331>
[10] <https://lists.debian.org/debian-devel/2005/10/msg00959.html>
[11] <https://lists.debian.org/debian-devel-announce/2005/11/msg00016.html> [12] <https://lists.debian.org/debian-devel/2006/09/msg01053.html>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Linux distributions, which have separate packages for developers, like
Debian, are not really supported [1] by the developer tool pkg-config
[2]. The problems are C/C++ libraries which depend on other libraries at runtime. Let us pick one, a security library for utilizing DNSSEC:

$ sudo apt install libunbound-dev
$ pkg-config --print-errors --short-errors --cflags libunbound
$ pkg-config --print-errors --short-errors --exists libunbound
Package 'hogweed', required by 'libunbound', not found

1) The error message is misleading because pkg-config talks about the
*runtime* package 'hogweed', although (in Debian) a *-dev* package is
required to get the .pc file. Should I report that, where?

2) In this case, the package name is not simply an added '-dev' like 'hogweed-dev'. Instead, the developer has to search the contents of all packages [3] for the file 'hogweed.pc'. That is part of the package 'nettle-dev'.

3) Are 'nettle-dev' (and 'libevent-dev', by the way) missing
dependencies of 'libunbound-dev'?

The latter seems to be an ongoing question on debian-devel [4] if the
package includes a static library '.a' as libunbound-dev still does. At
least I found no conclusive answer. Position No: It is not a missing
dependency because I can use that package perfectly (as a shared
library) without pkg-config. Position Yes: pkg-config has to be
required, recommended, suggested - for each level of dependency, you
find an argument on debian-devel, see [5][6][7][8].

16 years. This topic has come up for 16 years now, again and again on
the mailing list debian-devel, documenting the uncertainty of package maintainers. Furthermore, bug reports come in, again and again for that
topic. Again, libunbound-dev as example for the Debian bug report [9]: Upstream, in the pkg-config .pc file, the maintainer moved the libraries
from 'Requires' to 'Requires.private'. That was the correct approach.
However, the maintainer closed the Debian bug report because he falsely believed to have fixed the issue that way.

Requires -> -I/subfolder -lfuu, avoid if possible, see [10] Requires.private -> -I/subfolder , a header includes that header
Libs -> -lfoo, lib itself
Libs.private -> -lfuu, exclusively for static linking
Cflags -> -I/subfolder , lib headers itself and/or
header includes that header

Any idea how to approach this?
a) Leave as is. Do not depend on 'Require.private' automatically because
the package can be used without pkg-config. Only depend on other -devs
if one of those headers is included in a header.
b) Fix/convert. Upstream, move everything into 'Requires.private'.
Downstream, in the Debian package, remove 'Requires.private' and convert
to 'Libs.private'; again, except if one of those headers is included in
a header.

These uncertainties create repeated frustration, even for core Debian
members. Proposal: Why not decide, or at least give a recommendation on
these questions:

i) pkg-config tool itself: When is it a depend (which dep level?), and
when not? My suggestions: Because I can use a '-dev' package without any
tool, because of the FHS, just -lfoo should be needed [11].

ii) .pc file and its 'Require': Leave it or change it? My suggestion:
Report upstream that those libs have to be moved to 'Requires.private'.

iii) .pc file and its 'Require[.private]': Depend on each lib? My
suggestion: Only if the headers include it. For Debian, convert all
others to 'Libs.private' because the built '.a' file (for static
linking) has a known dependency tree. This gets a Debian patch in that
source package.

iv) Cflags: If I got it correctly, back in the year 2005, the cause of
this drama were -I flags [12]; if the header included another header,
and that header included further headers but was not in the root but in
a subfolder, an -I flag *might* be required. For example, the package 'libopusfile-dev' has its header file in '/usr/include/opus/opusfile.h'
with #include <opus_multistream.h> which is part of the package
'libopus-dev' and is not within '/usr/include/' but within '/usr/include/opus/'. Therefore,
$ pkg-config --cflags opusfile
-I/usr/include/opus

After reading, understanding, and researching so much, I really wonder
if this was correct. For example, if the header did
#include <opus/opus_multistream.h>
no include directory would be needed to be figured out via pkg-config. Actually, if the world was like that, I could shelf pkg-config and go
just for linking libraries.

Long story short:
After 16 years of back and forth, several hundred bug reports, still new
ones coming in, and existing reports not fixed completely, I wonder if
someone in the Debian world is able to decide this finally and provide
that decision either as policy change, or recommendation, or at least as
a hint for future -dev package maintainers. And please, tell me, what I
should do about libunbound-dev; re-open that bug report [9]?

[1] <https://gitlab.freedesktop.org/pkg-config/pkg-config/-/issues/7#note_656054>
[2] <https://people.freedesktop.org/~dbn/pkg-config-guide.html>
[3] <https://packages.debian.org>
[4] <https://lists.debian.org/debian-devel/2008/02/msg01189.html>
[5] <https://lists.debian.org/debian-devel/2004/11/msg00319.html>
[6] <https://www.mail-archive.com/debian-devel@lists.debian.org/msg258710.html>
[7] <https://www.mail-archive.com/debian-policy@lists.debian.org/msg21816.html>
[8] <https://www.mail-archive.com/debian-user@lists.debian.org/msg729223.html>
[9] <https://bugs.debian.org/958331>
[10] <https://lists.debian.org/debian-devel/2005/10/msg00959.html>
[11] <https://lists.debian.org/debian-devel-announce/2005/11/msg00016.html> [12] <https://lists.debian.org/debian-devel/2006/09/msg01053.html>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Mon, 13 Dec 2021 at 22:46:43 +0100, Alexander Traud wrote:

Let us assume 'bar.pc' would create a dependency in Debian, then the
question arises: Why does the Debian maintainer not transform it to 'Libs.private', not upstream but via a Debian patch? That would avoid
such .pc-file*only* dependencies.

Libs.private does not follow interdependencies between .pc files, so
converting Requires.private into Libs.private would require copying all
the recursive dependencies of the required library, not just the required library itself.

For example, if you depend on glib-2.0.pc (-lglib-2.0), converting Requires.private: glib-2.0 into Libs.private requires you to know (or more likely, find out at configure time) that it depends on libpcre.

This doesn't scale well, and a previously-correct .pc file can become retroactively incorrect as a result of a lower-level library changing
an implementatation detail. For example, if GLib moves from libpcre to libpcre2, as has been requested, then a GLib-dependent library having -lglib-2.0 -lpcre in Libs.private will no longer be sufficient for
static linking.

Just to repeat: libunbound-dev works great without nettle-dev and libevent-dev

With the change you are asking for, libunbound-dev would work *for dynamic linking* without those.

If you're linking statically, you need to be able to satisfy the recursive dependencies of libunbound (regardless of whether you are using pkg-config
or not), so, no, you will need nettle-dev and libevent-dev either way.

smcv

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Simon McVittie <smcv@debian.org> writes:

If you're linking statically, you need to be able to satisfy the
recursive dependencies of libunbound (regardless of whether you are
using pkg-config or not), so, no, you will need nettle-dev and
libevent-dev either way.

And, specifically, I think we should say as a matter of policy that Debian
does not try to declare the dependencies required for static linking and therefore static linking may require tracking down what additional -dev packages you need and manually installing them. The dependencies for -dev packages should reflect the requirements for dynamic linking.

Static linking of C libraries is rare enough at this point, and (possibly
more relevantly) sufficiently untested by Debian that I don't think we
should try to support it directly in the dependency structure. It would
pull in a whole bunch of -dev packages that wouldn't be used 95% of the
time.

This is what I already do for libremctl-dev, for example. (Well, I list
the libraries required for static linking in Suggests currently, but I'm
not sure that list is complete any more and should probably just drop it completely.)

--
Russ Allbery (rra@debian.org) <https://www.eyrie.org/~eagle/>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Fri, 17 Dec 2021 at 10:05:39 +0100, Alexander Traud wrote:

I do not understand why:
a) pkg-config itself is *not* a dependency as well

Sometimes it is appropriate for it to be, and sometimes it is not.

For some libraries, the only maintainer-supported way to consume the
library is via pkg-config. If that's the case, then a dependency on
pkg-config can be appropriate - although we don't add a dependency on
cc or binutils, which is equally necessary. Libraries with multiple parallel-installable versions, like GTK, often behave like this: you have
to explicitly ask (via pkg-config) to add the version you want to use to
the compiler's search paths, so that you will not get the <gtk/gtk.h>
from GTK 2 or 4 where GTK 3 was expected.

For other libraries, either there are other supported ways to consume
the library (CMake metadata or sdl2-config or similar), or the library
is in the compiler's default search paths (not parallel-installable)
like libjpeg - so you *can* use pkg-config, but you don't *have* to.

b) Lintian was not upgraded in 15 years to check the content of the .pc
file and aid the maintainer in not missing any required dependency.

Lintian cannot know whether the required dependencies are present: that
is outside its scope. Lintian analyzes each package in isolation, without referring to other packages. If your library depends on foo-bar, Lintian
cannot know whether foo-bar.pc is in libfoo-bar-dev, libfoo-bar2-dev, libfoo-dev, libfoo2-dev or something else.

Simon McVittie wrote:

this doesn't scale well

Then, I do not understand static linking. I thought, please correct me
here, static linking is about the '.a' file in the package, the current
one. I have to suffice its links as it was built, not the links when I
build.

Imagine you are packaging a library "libfoo" that makes use of GLib (glib-2.0.pc, from src:glib2.0) but does not expose GLib in its header
files. This is an example of the class of dependency that you want
to move from Requires.private to Libs.private.

glib-2.0 currently depends on libpcre, which it uses to implement GRegex.
It does not include libpcre headers in its header files (the GRegex API completely wraps libpcre), so libpcre is another example of the class of dependency that you want to move from Requires.private to Libs.private.
Imagine that GLib had done this.

When you statically link your library libfoo into a program, you want to
end up with something like this:

gcc -static ... -lfoo -lglib-2.0 -lpcre

libfoo.a provides symbols used by the program, libglib-2.0.a provides
symbols used by libfoo, and libpcre.a provides symbols used by libglib-2.0.a. So far, everything is working.

If we were using Libs.private as you suggest, then glib-2.0.pc and libfoo.pc would have something like:

# glib-2.0.pc
...
Libs: -L/usr/lib/MULTIARCH -lglib-2.0
# generated from: Libs.private: $(pkg-config --static --libs libpcre)
Libs.private: -L/usr/lib/MULTIARCH -lpcre

# libpcre.pc
...
Libs: -L/usr/lib/MULTIARCH -lfoo
# generated from: Libs.private: $(pkg-config --static --libs glib-2.0)
Libs.private: -L/usr/lib/MULTIARCH -lglib-2.0 -lpcre

Now imagine that GLib switches to libpcre2 (libpcre2-8.pc, -lpcre2-8)
as requested in #1000082. This will change the contents of glib-2.0.pc to:

# glib-2.0.pc
...
Libs: -L/usr/lib/MULTIARCH -lglib-2.0
# generated from: Libs.private: $(pkg-config --static --libs libpcre2-8)
Libs.private: -L/usr/lib/MULTIARCH -lpcre2-8

Without changing anything in libfoo, you'll find that libfoo.pc is now
wrong (until it is rebuilt and has the opportunity to update its idea of
what GLib depends on). Programs that depend on libfoo will fail to
link, because the new libglib-2.0.a requires symbols from libpcre2-8.a
instead of the old libpcre.a, but libfoo.pc still says
"Libs.private: ... -lpcre" because that's what it captured at libfoo's build-time.

That's why I think it is best that the information about which libraries
GLib depends on is shipped with GLib and included by reference (via
either Requires.private: glib-2.0, or the Requires.internal: glib-2.0
proposed on https://bugs.freedesktop.org/show_bug.cgi?id=105572), instead
of copying the information about GLib's dependencies into libfoo-dev where
it will become outdated when GLib changes.

smcv

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Thu, 09 Dec 2021 at 15:24:27 +0100, Alexander Traud wrote:

if the header included another header,
and that header included further headers but was not in the root but in
a subfolder, an -I flag *might* be required. For example, the package 'libopusfile-dev' has its header file in '/usr/include/opus/opusfile.h'
with #include <opus_multistream.h> which is part of the package
'libopus-dev' and is not within '/usr/include/' but within '/usr/include/opus/'. Therefore,
$ pkg-config --cflags opusfile
-I/usr/include/opus

This is deliberate, and is a very large part of the purpose of pkg-config.

Users of libraries should #include the library's headers in whatever
way the library maintainer has documented them to be used. The correct
#include pattern is part of the library's API. For example, for GTK,
the documentation says to always #include <gtk/gtk.h>, and using <gtk-3.0/gtk/gtk.h> would be wrong. For SDL2, it's correctly <SDL.h>,
not <SDL2/SDL.h>.

GTK has multiple major versions, which are incompatible and parallel-installable. If your program uses GTK 3, it needs to see GTK 3 headers: using GTK 2 or 4 headers would not work. This is implemented
by keeping the GTK 2, 3 and 4 headers in different non-default
directories, so that you only get the version of <gtk/gtk.h> you asked
for, and not the others. SDL 1.2 and 2 are similar, and there are many
other libraries with a similar design. The canonical text on this is https://ometer.com/parallel.html (and you can tell how old it is because
it talks about GNOME 2 as though that was a new thing - but it's still relevant).

Not doing that would give us the situation we have for things like libjpeg, libcurl and libopenssl, where you can only have one version of the
development files installed system-wide at a time: if you have a program
that needs OpenSSL 1.1, and another program that needs OpenSSL 1.0, you
have to choose which one you can compile (by installing libssl-dev or libssl1.0-dev), and you cannot compile the other one.

Conversely, if the old and new versions of library are *mostly* compatible (like the way some SDL programs such as src:quakespasm can compile against either SDL 1.2 or 2 with a few #ifdefs), we don't want to have to change
every source file to do something like this:

#ifdef USING_MYLIB_3
#include <mylib-3/mylib.h>
#else
#include <mylib-2/mylib.h>
#endif

which explains why GTK and SDL have chosen to document <gtk/gtk.h>
and <SDL.h> respectively, instead of a versioned name appearing in the
source code.

For libraries with many smaller headers (popular in C++, e.g. Qt,
because parsing C++ is more expensive than parsing C), instead of one
big meta-header like the one in GTK, you certainly wouldn't want to do:

#ifdef USING_MYLIB_3
#include <mylib-3/MyFrobnicator>
#include <mylib-3/MyTemplate>
#include <mylib-3/MyWidget>
...
#else
#include <mylib-2/MyFrobnicator>
#include <mylib-2/MyTemplate>
#include <mylib-2/MyWidget>
...
#endif

The other major part of the purpose of pkg-config is that it lets you
use dependencies that are *not* installed system-wide, without having to
make source code changes: if I set the PKG_CONFIG_PATH correctly, it will
find a GTK installation in /opt/gnome-42/{include,lib} and use that,
with my source code still using #include <gtk/gtk.h> as usual.

smcv

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Simon McVittie <smcv@debian.org> writes:

For some libraries, the only maintainer-supported way to consume the
library is via pkg-config. If that's the case, then a dependency on pkg-config can be appropriate - although we don't add a dependency on
cc or binutils, which is equally necessary.

Well, cc and binutils are in build-essential, so this isn't entirely equivalent.

For other libraries, either there are other supported ways to consume
the library (CMake metadata or sdl2-config or similar), or the library
is in the compiler's default search paths (not parallel-installable)
like libjpeg - so you *can* use pkg-config, but you don't *have* to.

Yeah, I think this is the key point: it's entirely possible to use most libraries without pkg-config because they're installed into the default
search paths, so you can just... use them.

If they *require* special flags (non-standard paths, non-standard compiler flags, etc.) such that pkg-config is the only supported interface, then I
think one could make a good argument that pkg-config should be a
dependency of the -dev package.

That's why I think it is best that the information about which libraries
GLib depends on is shipped with GLib and included by reference (via
either Requires.private: glib-2.0, or the Requires.internal: glib-2.0 proposed on https://bugs.freedesktop.org/show_bug.cgi?id=105572),
instead of copying the information about GLib's dependencies into
libfoo-dev where it will become outdated when GLib changes.

Yes.

--
Russ Allbery (rra@debian.org) <https://www.eyrie.org/~eagle/>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)