in 2020 there was a brief discussion on debian-devel@ about trimming changelogs [1,2].
* The `--no-trim` option allows package maintainers that want to ship
the whole changelog a way to do so.
* The full changelogs are preserved in the source packages and thus available via `apt changelog` and similar mechanisms.
Does anybody have objective objections against activating automatic changelog trimming in binary packages?
Does anybody have objective objections against activating automatic changelog trimming in binary packages?
My objections from that time still stand:
<https://lists.debian.org/debian-devel/2020/03/msg00482.html>
I would also like to highlight David Kalnischkies reply:
<https://lists.debian.org/debian-devel/2020/03/msg00309.html>
Doing that trimming globally would also mean that this applies evenMaintainers of packages _meant_ to be installed locally (i.e.,
to packages that are for local or derived use where something like
«apt changelog» will in most cases not work.
Assuming the repository supports it. I have yet to encounter aSimilar to the point above, if the third-party developers are
third-party which does, so if Debian would trim e.g. in debhelper by
default some care might need to be applied so that this happens only
to packages which end up in Debians repositories… which could
complicate reproducibility as its clear for a buildd, but my local
sbuild…
The benefit of treating all packages the same is that tools working(I read this as a comment _in favor_ of automatic trimming, but maybe
with changelogs can handle the grunt work: "apt{,-get} changelog pkg"
prefers the changelog on disk if available – except for repositories
which identify as "Ubuntu" for which it will always download the
online changelog for display.
A proposal I've been floating around from time to time has been toThat is indeed a good idea. And, by implementing it in addition to `dh_installchangelogs` auto-trimming, it will even further reduce the
instead make the changelog and copyright files deb/dpkg metadata
(which they really are anyway IMO).
Il 19/08/2022 03:01, Paul Wise ha scritto:
On Thu, 2022-08-18 at 21:18 +0200, Gioele Barabucci wrote:
Does anybody have objective objections against activatingBefore we consider enabling this by default, first we need a way
automatic
changelog trimming in binary packages?
for
`apt changelog` to download the full changelog rather than loading
the
changelog from /usr/share/doc in the currently installed package.
Otherwise people who want to look at the full changelog for the
currently installed version of the package will have no easy way to
do
so. They will have to manually find it instead, which isn't exactly
an
easy process if you do not know where the changelogs are stored
online.
Hi, I also used the changelog many times both in the packages I
maintain
and in the one I only use, in some cases a very old changelog entries
was needed.
On Thu, 2022-08-18 at 21:18 +0200, Gioele Barabucci wrote:
Does anybody have objective objections against activating automatic
changelog trimming in binary packages?
Before we consider enabling this by default, first we need a way for
`apt changelog` to download the full changelog rather than loading the changelog from /usr/share/doc in the currently installed package.
Otherwise people who want to look at the full changelog for the
currently installed version of the package will have no easy way to do
so. They will have to manually find it instead, which isn't exactly an
easy process if you do not know where the changelogs are stored online.
P.S. BTW the change Guillem suggests seems like a good idea anyway: Â Â Â Â Â Â treating changelogs as control files.
How about making the end of the trimmed file be a standard footer
including a hint about how one can get hold of the rest of the
changelog, and then have `apt changelog` look out for that footer in the local copies in order to know that they've been trimmed, and that it
should therefore try to download the full version.
If we turn on changelog trimming by default, we should probably also
turn on apt downloading the changelogs by default.
I guess this could be solved by dpkg creating symlinks from the "master
copy" which is per-source to /usr/share/doc/$binpkg/
- Having to spawn an external command ("dpkg --show-changelog") just
to access a file is more complicated.
Before we consider enabling this by default, first we need a way for
`apt changelog` to download the full changelog rather than loading the changelog from /usr/share/doc in the currently installed package.
Am 19.08.22 um 10:35 schrieb Philip Hands:
Paul Wise <pabs@debian.org> writes:
On Thu, 2022-08-18 at 21:18 +0200, Gioele Barabucci wrote:
Does anybody have objective objections against activating automatic
changelog trimming in binary packages?
Before we consider enabling this by default, first we need a way for
`apt changelog` to download the full changelog rather than loading the
changelog from /usr/share/doc in the currently installed package.
Otherwise people who want to look at the full changelog for the
currently installed version of the package will have no easy way to do
so. They will have to manually find it instead, which isn't exactly an
easy process if you do not know where the changelogs are stored online.
How about making the end of the trimmed file be a standard footer
including a hint about how one can get hold of the rest of the
changelog, and then have `apt changelog` look out for that footer in the
local copies in order to know that they've been trimmed, and that it
should therefore try to download the full version.
If we enable trimming by default (for all packages), then apt can just
go directly to query the changelog online.
What would be the benefit for "apt changelog" to look for such a footer?
Ansgar <ansgar@43-1.org> writes:
 - Having to spawn an external command ("dpkg --show-changelog") just
  to access a file is more complicated.
The fact that it currently needs to dug out of the main data archive
rather than the control archive to show the user changes at install time seemed like a decent reason to move it, but that was assuming that the
file would still end up being visible in the same place in the end.
If that were not the case then I think it would be more trouble than
it's worth.
Hello,Could you stress on the mapage that some license ask to document the change done by downstream in binary distribution and that in this case this tool should be disable
in 2020 there was a brief discussion on debian-devel@ about trimming changelogs [1,2].
Now there is a working implementation of said functionality in `dh_installchangelogs` [3].
This implementation, combined with the evolution of the apt/dpkg
ecosystem that happened in the meantime, provides now all the benefits
of changelog trimming (less wasted space and bandwidth worldwide,
reduced processing time) without the downsides discussed at the time.
## In detail
* `dh_installchangelogs` is modified install in binary packages the
trimmed changelogs, i.e. changelogs that contain only entries more
recent than the release date of oldstable.
* The trimming is done automatically in compat >= 14.
* The `--no-trim` option allows package maintainers that want to ship
the whole changelog a way to do so.
* The full changelogs are preserved in the source packages and thus
available via `apt changelog` and similar mechanisms.
* The trimming process happens at build time and requires no
modification to the changelogs stored in the VCS repos, nor changes to
the packaging.
## Data on file size reduction
On a sample of ~13.000 packages, the median reduction in size of
gzip-9'ed changelogs is 56% (mean 50%).
Ancient packages or heavily developed packages gain a lot from trimming
the changelogs. Some examples (gzipped → trimmed+gzipped):
* debian-keyring: 664k → 47k (-92%)
* dpkg (essential): 223k → 22k (-90%)
* apt (essential): 156k → 14k (-90%)
* systemd: 110k → 23k (-78%)
* gcc-12: 189k → 18k (-90%)
* python3.9: 48k → 4k (-90%)
* e2fsprogs: 68k → 7k (-89%)
## Consensus
Does anybody have objective objections against activating automatic
changelog trimming in binary packages?
Regards,
[1] https://lists.debian.org/debian-devel/2020/03/msg00299.html
[2] https://bugs.debian.org/954313
[3] https://salsa.debian.org/debian/debhelper/-/merge_requests/80
--
Gioele Barabucci
Le jeudi 18 août 2022, 19:18:35 UTC Gioele Barabucci a écrit :
Hello,Could you stress on the mapage that some license ask to document the change done by downstream in binary distribution and that in this case this tool should be disable
in 2020 there was a brief discussion on debian-devel@ about trimming
changelogs [1,2].
Now there is a working implementation of said functionality in
`dh_installchangelogs` [3].
Le jeudi 18 août 2022, 19:18:35 UTC Gioele Barabucci a écrit :Like GPLv2's "You must cause the modified files to carry prominent notices stating that you changed the files and the date of any change"? We already don't do this.
Hello,
in 2020 there was a brief discussion on debian-devel@ about trimming changelogs [1,2].
Now there is a working implementation of said functionality in `dh_installchangelogs` [3].Could you stress on the mapage that some license ask to document the change done by downstream in binary distribution and that in this case this tool should be disable
Does anybody have objective objections against activating automatic
changelog trimming in binary packages?
* Packages not meant to be included in Debian (and without access to a changelog server): Creators that want to preserve the full history can
use the `--no-trim` option to disable the trimming.
* Third-party repositories: Their administrators can provide access to
the full changelogs by setting `Changelogs:` in the `Release` file, or
can use `--no-trim` in the packages that they distribute.
While all looks good and feels sound from many aspects, I have some reservations against treating changelogs as metadata.Assuming you are talking about making changelogs available at a dpkg
Current changelogs as files have a well known place, can be used by anything and everything, and they are local.
Stuffing them behind a command, possibly making them online only in the process will arguably make system troubleshooting and administration harder, esp. if the system has connectivity issues.I don't think removing recent-ish changelogs from the disk is proposed
If something critical breaks, I can boot to recovery and look at the logs
and changelogs of recently updated packages. Having recent-ish changelogs on the disk arguably accelerates the fixing process.
On Tue, 2022-09-06 at 07:13 +0200, Gioele Barabucci wrote:
* Packages not meant to be included in Debian (and without access to a
changelog server): Creators that want to preserve the full history can
use the `--no-trim` option to disable the trimming.
Most derivatives aren't going to have a changelog server so I don't
think it makes sense to enable it by default on all derivatives.
Perhaps the dpkg vendor could be used as guidance for when the trimming should occur by default? Unless Debian/Ubuntu, disable by default.
Then have a way for vendors to enable it by default if they want.
* Third-party repositories: Their administrators can provide access to
the full changelogs by setting `Changelogs:` in the `Release` file, or
can use `--no-trim` in the packages that they distribute.
Modifying every single package to add --no-trim would be tedious for
some repositories, perhaps allow a config option to disable it?
On Tue, 2022-09-06 at 07:13 +0200, Gioele Barabucci wrote:
* Packages not meant to be included in Debian (and without access to a
changelog server): Creators that want to preserve the full history can
use the `--no-trim` option to disable the trimming.
Most derivatives aren't going to have a changelog server so I don't
think it makes sense to enable it by default on all derivatives.
Perhaps the dpkg vendor could be used as guidance for when the trimming should occur by default? Unless Debian/Ubuntu, disable by default.
On Tue, Sep 06, 2022 at 12:11:38PM +0300, Hakan Bayındır wrote:
Stuffing them behind a command, possibly making them online only in theI don't think removing recent-ish changelogs from the disk is proposed
process will arguably make system troubleshooting and administration harder, >> esp. if the system has connectivity issues.
If something critical breaks, I can boot to recovery and look at the logs
and changelogs of recently updated packages. Having recent-ish changelogs on >> the disk arguably accelerates the fixing process.
here.
On 11 Sep 2022, at 14:59, Andrey Rahmatullin <wrar@debian.org> wrote:
On Sun, Sep 11, 2022 at 02:41:24PM +0300, Hakan Bayındır wrote:
It didn't create anything because that was how it worked from theOn Tue, Sep 06, 2022 at 12:11:38PM +0300, Hakan Bayındır wrote:
While all looks good and feels sound from many aspects, I have someAssuming you are talking about making changelogs available at a dpkg
reservations against treating changelogs as metadata.
Current changelogs as files have a well known place, can be used by anything
and everything, and they are local.
command, as in the RPM world, it's actually making the way to get a
package changelog more well-known, not less.
If this created the opposite effect in RPM world w.r.t. my theory, then I stand corrected.
beginning.
But yes, it's easier to run -q --changelog than write a full file path.
Even this isn't "making them online only".Stuffing them behind a command, possibly making them online only in the >>>> process will arguably make system troubleshooting and administration harder,I don't think removing recent-ish changelogs from the disk is proposed
esp. if the system has connectivity issues.
If something critical breaks, I can boot to recovery and look at the logs >>>> and changelogs of recently updated packages. Having recent-ish changelogs on
the disk arguably accelerates the fixing process.
here.
Again, I may have misread, then. Because what I understood is
Trim changelogs:
1. Convert them to metadata
2. Ship untrimmed part in package DB
3. Get remaining part from network
Effectively eliminating /usr/share/doc/*changelog* files.
--
WBR, wRAR
On 6 Sep 2022, at 12:42, Andrey Rahmatullin <wrar@debian.org> wrote:
On Tue, Sep 06, 2022 at 12:11:38PM +0300, Hakan Bayındır wrote:
While all looks good and feels sound from many aspects, I have someAssuming you are talking about making changelogs available at a dpkg
reservations against treating changelogs as metadata.
Current changelogs as files have a well known place, can be used by anything >> and everything, and they are local.
command, as in the RPM world, it's actually making the way to get a
package changelog more well-known, not less.
Stuffing them behind a command, possibly making them online only in theI don't think removing recent-ish changelogs from the disk is proposed
process will arguably make system troubleshooting and administration harder, >> esp. if the system has connectivity issues.
If something critical breaks, I can boot to recovery and look at the logs
and changelogs of recently updated packages. Having recent-ish changelogs on >> the disk arguably accelerates the fixing process.
here.
--
WBR, wRAR
It didn't create anything because that was how it worked from theOn Tue, Sep 06, 2022 at 12:11:38PM +0300, Hakan Bayındır wrote:
While all looks good and feels sound from many aspects, I have someAssuming you are talking about making changelogs available at a dpkg command, as in the RPM world, it's actually making the way to get a
reservations against treating changelogs as metadata.
Current changelogs as files have a well known place, can be used by anything
and everything, and they are local.
package changelog more well-known, not less.
If this created the opposite effect in RPM world w.r.t. my theory, then I stand corrected.
Even this isn't "making them online only".Stuffing them behind a command, possibly making them online only in theI don't think removing recent-ish changelogs from the disk is proposed here.
process will arguably make system troubleshooting and administration harder,
esp. if the system has connectivity issues.
If something critical breaks, I can boot to recovery and look at the logs >> and changelogs of recently updated packages. Having recent-ish changelogs on
the disk arguably accelerates the fixing process.
Again, I may have misread, then. Because what I understood is
Trim changelogs:
1. Convert them to metadata
2. Ship untrimmed part in package DB
3. Get remaining part from network
Effectively eliminating /usr/share/doc/*changelog* files.
I assume that cases when your dpkg database is broken and cases when youEven this isn't "making them online only".Stuffing them behind a command, possibly making them online only in the >>>> process will arguably make system troubleshooting and administration harder,I don't think removing recent-ish changelogs from the disk is proposed >>> here.
esp. if the system has connectivity issues.
If something critical breaks, I can boot to recovery and look at the logs
and changelogs of recently updated packages. Having recent-ish changelogs on
the disk arguably accelerates the fixing process.
Again, I may have misread, then. Because what I understood is
Trim changelogs:
1. Convert them to metadata
2. Ship untrimmed part in package DB
3. Get remaining part from network
Effectively eliminating /usr/share/doc/*changelog* files.
Yes, you’re right. However, my reservation is whether dpkg is more prone to breaking in disaster recovery scenarios. Reading a gzipped file is always simpler than querying a DB via more abstraction.
Yes, you’re right. However, my reservation is whether dpkg is more prone to breaking in disaster recovery scenarios. Reading a gzipped file is always simpler than querying a DB via more abstraction.
On 14 Sep 2022, at 10:37, Wouter Verhelst <wouter@debian.org> wrote:
On Sun, Sep 11, 2022 at 03:09:07PM +0300, Hakan Bayındır wrote:
Yes, you’re right. However, my reservation is whether dpkg is more prone to
breaking in disaster recovery scenarios. Reading a gzipped file is always
simpler than querying a DB via more abstraction.
Honestly though, the way to track down a regression is to read /var/log/dpkg.log, not changelogs.
--
w@uter.{be,co.za}
wouter@{grep.be,fosdem.org,debian.org}
I will have a Tin-Actinium-Potassium mixture, thanks.
On 14 Sep 2022, at 10:37, Wouter Verhelst <wouter@debian.org> wrote:
On Sun, Sep 11, 2022 at 03:09:07PM +0300, Hakan Bayındır wrote:
Yes, you’re right. However, my reservation is whether dpkg is more prone to
breaking in disaster recovery scenarios. Reading a gzipped file is always >> simpler than querying a DB via more abstraction.
Honestly though, the way to track down a regression is to read /var/log/dpkg.log, not changelogs.
Does /var/log/dpkg.log contain why my VPNs suddenly don’t connect or
my daemon behaves differently after the last update? I thought they’re delivered through news and changelogs.
Actually, OpenVPN’s changing defaults are nicely delivered through
NEWS and changelogs, so I know why things broke at the first place.
On 14 Sep 2022, at 10:37, Wouter Verhelst <wouter@debian.org> wrote:
On Sun, Sep 11, 2022 at 03:09:07PM +0300, Hakan Bayındır wrote:
Yes, you’re right. However, my reservation is whether dpkg is more prone to
breaking in disaster recovery scenarios. Reading a gzipped file is always >> simpler than querying a DB via more abstraction.
Honestly though, the way to track down a regression is to read /var/log/dpkg.log, not changelogs.
Does /var/log/dpkg.log contain why my VPNs suddenly don’t connect or my daemon behaves differently after the last update? I thought they’re delivered
through news and changelogs.
On 18/08/22 21:18, Gioele Barabucci wrote:
Does anybody have objective objections against activating automatic changelog trimming in binary packages?
Hi,
a couple of weeks since the initial email (thanks everybody for the input), my reading is that there is now consensus in going ahead with the proposed automatic trimming of changelogs.
| Sysop: | Keyop |
|---|---|
| Location: | Huddersfield, West Yorkshire, UK |
| Users: | 300 |
| Nodes: | 16 (2 / 14) |
| Uptime: | 24:57:17 |
| Calls: | 6,707 |
| Calls today: | 1 |
| Files: | 12,239 |
| Messages: | 5,352,379 |