Hi,
I occasionally run into the situation that a package has been NMU'd or otherwise updated directly into the Debian repositories,
bypassing/ignoring that a packaging git repository existed. I was
wondering what techniques other DDs use to
1) detect that the git packaging repository was bypassed/diverged?
2) bring the git repository back in sync with minimal effort?
So far, I have settled on having the deb-src for Debian sid configured
on my laptop (even though it is not running sid) and then inside the
git packaging checkout directory run for example:
gbp import-dsc --verbose --pristine-tar apt:j4-dmenu-desktop/sid
This will both check if there are any newer uploads than what git
knows about, and if needed, also download, apply and commit the
uploads using the uploader as the git author. The result of this
particular example is visible in https://salsa.debian.org/otto/j4-dmenu-desktop/-/commit/62ea4cd6df973138e3a452b09250510da5279db4
I wanted to check if anyone knows any better techniques or mechanisms
than trying to remember to run this command manually before working on
a package?
- Otto
Hi,
I occasionally run into the situation that a package has been NMU'd or otherwise updated directly into the Debian repositories,
bypassing/ignoring that a packaging git repository existed. I was
wondering what techniques other DDs use to
1) detect that the git packaging repository was bypassed/diverged?
2) bring the git repository back in sync with minimal effort?
So far, I have settled on having the deb-src for Debian sid configured
on my laptop (even though it is not running sid) and then inside the
git packaging checkout directory run for example:
gbp import-dsc --verbose --pristine-tar apt:j4-dmenu-desktop/sid
I would very much prefer if it was possible in Debian to not allow
the archive to get out of sync with packaging git repo (for example
when it lives under salsa.debian.org/debian which uploaders should have access to already).
That would probably also require some "tag to upload" solution to be implemented first I presume.
I wanted to check if anyone knows any better techniques or mechanisms
than trying to remember to run this command manually before working on
a package?
I would very much prefer if it was possible in Debian to not allow
the archive to get out of sync with packaging git repo (for example
when it lives under salsa.debian.org/debian which uploaders should have access to already).
That would probably also require some "tag to upload" solution to be implemented first I presume.
If my understanding is correct, then it sounds wrong for DDs to be
granted access to all Salsa projects.
On 24/10/2024 21:36, Otto Kekäläinen wrote:
Hi,
I occasionally run into the situation that a package has been NMU'd or otherwise updated directly into the Debian repositories,
bypassing/ignoring that a packaging git repository existed. <snip>
Hi Otto,
Are any of these packages team maintained?
I understand there is an issue whereby although uploading DDs
have unrestricted access to the archive,
they cannot update Salsa team repos if they are not a team member.
Maybe this ought to be fixed?
I can understand restricting access for DMs, but does it make sense for DDs?
Hi,
I occasionally run into the situation that a package has been NMU'd or otherwise updated directly into the Debian repositories,
bypassing/ignoring that a packaging git repository existed. <snip>
I understand there is an issue whereby although uploading DDs
have unrestricted access to the archive,
they cannot update Salsa team repos if they are not a team member.
Maybe this ought to be fixed?
I can understand restricting access for DMs, but does it make sense for DDs?
I occasionally run into the situation that a package has been NMU'd or otherwise updated directly into the Debian repositories,
bypassing/ignoring that a packaging git repository existed. <snip>
Hi Otto,
Are any of these packages team maintained?
I understand there is an issue whereby although uploading DDs
have unrestricted access to the archive,
they cannot update Salsa team repos if they are not a team member.
"the archive is the truth"
"git is the truth"
I would very much prefer if it was possible in Debian to not allow
the archive to get out of sync with packaging git repo (for example
when it lives under salsa.debian.org/debian which uploaders should have access to already).
That would probably also require some "tag to upload" solution to be implemented first I presume.
Not sure what's the logic here, but I feel like what you thought about may require some "tag to upload" solution not to be just implemented but also mandated, which won't happen.
Hi,
I occasionally run into the situation that a package has been NMU'd or otherwise updated directly into the Debian repositories,
bypassing/ignoring that a packaging git repository existed. I was
wondering what techniques other DDs use to
1) detect that the git packaging repository was bypassed/diverged?
2) bring the git repository back in sync with minimal effort?
I would very much prefer if it was possible in Debian to not allow
the archive to get out of sync with packaging git repo (for example
when it lives under salsa.debian.org/debian which uploaders should have access to already).
That would probably also require some "tag to upload" solution to be implemented first I presume.
Honestly I'd be happy if we could just establish some expectation that
the NMUer open a merge request for their changes.
Honestly I'd be happy if we could just establish some expectation that
the NMUer open a merge request for their changes. It can be merged
later without losing anything or requiring additional work. Enforcement
of this expectation would be even better, of course.
Honestly I'd be happy if we could just establish some expectation that
the NMUer open a merge request for their changes. It can be merged
later without losing anything or requiring additional work. Enforcement
of this expectation would be even better, of course.
the current expectation is that an NMU bug is opened, which contains
the debdiff.
https://www.debian.org/doc/manuals/developers-reference/developers-reference.en.html#when-and-how-to-do-an-nmu
"... Then, you must send a patch with the differences between the current
package and your proposed NMU to the BTS. The nmudiff script in the
devscripts package might be helpful...."
Honestly I'd be happy if we could just establish some expectation that the NMUer open a merge request for their changes. It can be merged
later without losing anything or requiring additional work. Enforcement of this expectation would be even better, of course.
the current expectation is that an NMU bug is opened, which contains
the debdiff.
https://www.debian.org/doc/manuals/developers-reference/developers-reference.en.html#when-and-how-to-do-an-nmu
"... Then, you must send a patch with the differences between the current
package and your proposed NMU to the BTS. The nmudiff script in the
devscripts package might be helpful...."
Right, and that's not a whole lot more helpful than requiring me to
download the sourcepackage and generate the debdiff myself. Sure all
the content is there, but it's still a tedious amount of work that's
easily forgotten. Further, it loses a little bit of metadata, in that
the git commit now comes from me, rather than the person doing the
actual NMU.
Yes, I know this is trivial, and yes I know I can fix it with more work;
I don't want NMUs to make more work for me. It makes me not like NMUs.
Thanks for all the comments!
Trying to summarize and expand on the points raised:
Seems this is still the most optimal way to ensure git is correct:
gbp import-dsc --verbose --pristine-tar apt:j4-dmenu-desktop/sid
Also, dgit pull can be used to get the latest source automatically, but unfortunately those
git commits are made as a custom "Debian as a git repo" representation, and is not
compatible with using CI testing and code review before upload in the way many of us
are doing on Salsa currently.
the current expectation is that an NMU bug is opened, which contains
the debdiff.
https://www.debian.org/doc/manuals/developers-reference/developers-reference.en.html#when-and-how-to-do-an-nmu
"... Then, you must send a patch with the differences between the current
package and your proposed NMU to the BTS. The nmudiff script in the
devscripts package might be helpful...."
Seems this is still the most optimal way to ensure git is correct:
gbp import-dsc --verbose --pristine-tar apt:j4-dmenu-desktop/sid
Also, dgit pull can be used to get the latest source automatically, but unfortunately those
git commits are made as a custom "Debian as a git repo" representation, and is not
compatible with using CI testing and code review before upload in the way many of us
are doing on Salsa currently.
'dgit pull' integrates the NMU automatically, when it can. It doesn't
just fetch the source. I don't follow how it's different from 'gbp import-dsc'. Could you say more?
Hi!
Seems this is still the most optimal way to ensure git is correct:
gbp import-dsc --verbose --pristine-tar apt:j4-dmenu-desktop/sid
Also, dgit pull can be used to get the latest source automatically, but unfortunately those
git commits are made as a custom "Debian as a git repo" representation, and is not
compatible with using CI testing and code review before upload in the way many of us
are doing on Salsa currently.
'dgit pull' integrates the NMU automatically, when it can. It doesn't
just fetch the source. I don't follow how it's different from 'gbp
import-dsc'. Could you say more?
In a gbp checkout of git@salsa.debian.org:debian/j4-dmenu-desktop.git,
how would you invoke 'dgit pull sid' to import the NMU?
Without any parameters, it will create branch 'dgit/sid' which has
unrelated history and patches are applied and nothing can be merged or cherry-picked to the git-buildpackage master branch. Perhaps I am just missing something on how this should work, or perhaps https://manpages.debian.org/unstable/dgit/dgit-maint-gbp.7.en.html#INCORPORATING_NMUS
implies the functionality isn't yet there?
I would very much prefer if it was possible in Debian to not allow
the archive to get out of sync with packaging git repo (for example
when it lives under salsa.debian.org/debian which uploaders should have access to already).
Hello,
On Sun 27 Oct 2024 at 05:29pm GMT, Otto Kekäläinen wrote:
Hi!
Seems this is still the most optimal way to ensure git is correct:
gbp import-dsc --verbose --pristine-tar apt:j4-dmenu-desktop/sid
Also, dgit pull can be used to get the latest source automatically, but unfortunately those
git commits are made as a custom "Debian as a git repo" representation, and is not
compatible with using CI testing and code review before upload in the way many of us
are doing on Salsa currently.
'dgit pull' integrates the NMU automatically, when it can. It doesn't
just fetch the source. I don't follow how it's different from 'gbp
import-dsc'. Could you say more?
In a gbp checkout of git@salsa.debian.org:debian/j4-dmenu-desktop.git,
how would you invoke 'dgit pull sid' to import the NMU?
Without any parameters, it will create branch 'dgit/sid' which has unrelated history and patches are applied and nothing can be merged or cherry-picked to the git-buildpackage master branch. Perhaps I am just missing something on how this should work, or perhaps https://manpages.debian.org/unstable/dgit/dgit-maint-gbp.7.en.html#INCORPORATING_NMUS
implies the functionality isn't yet there?
Ah, I was thinking that you had already been using 'dgit --gbp push' to upload the package. In that case the histories would be related, just
with some additional commits on top, and a manual merge would be
possible.
I can (and I did test already) do a merge with --allow-unrelated
histories, but dgit history always has patches applied as separate
commits that get rebased and thus there is no quilt/gbp pq -compatible
git history to merge from. If I later do a 'dgit --gbp push' to
upload, how do I push a development version to Salsa for review and CI testing?
Based on docs and your previous replies it isn't possible, thus `gbp import-dsc --verbose --pristine-tar apt:j4-dmenu-desktop/sid` still
seems to stand as the optimal way to import NMUs onto a Debian
packaging git repo?
The man pages of dgit are extensive and explains well how to interact
with the Debian repository as if it was a git repository, but I am
unable to find descriptions of how to use dgit in team maintained
packages with testing and reviews prior to uploads.
Andrey has already said much of what I could add to the thread, but I
think I can slightly clarify the needs of NMUers.
On Fri, Oct 25, 2024 at 08:45:16AM +0200, Andreas Henriksson wrote:
I would very much prefer if it was possible in Debian to not allow
the archive to get out of sync with packaging git repo (for example
when it lives under salsa.debian.org/debian which uploaders should have
access to already).
There are three quite fundamental pieces missing to achieve this.
There needs to be simple way to turn a git commit into a source package.
If the source of truth ever is to become git, the .dsc becomes an export format and then this becomes a hard requirement. We can turn git commits
into source packages. The problem is that there is not one way to do
this, but about a hundred and you need to know which package uses which.
That does not scale.
There needs to be a simple way to figure out the commit that corresponds
to an upload. This problem has been approached in two ways. For one
thing, there is DEP14 recommending a particular tag layout, but I think
this is backwards. It assumes that the git repository is trusted, but in reality git repositories allow for much wider access than Debian
uploads. What we really needs is a source package to know the commit id
it was generated from.
These operations need to round-trip. If you take a source package,
identify the git commit and export it to .dsc, it must be functionality equivalent to what you started with. Timestamps may differ, but file
content or contained files very much not.
To me, these are hard requirements for using maintainer git
repositories for performing NMUs.
Now the dgit users among us will be grinning already as what I have
written here, very much reads like a specification of (parts of) dgit.
Once again, I question whether salsa as we use it now is the solution or
the problem. I note that it is practically possible to push your dgit
history to salsa and then NMUers can easily do meaningful MRs for their uploads even when your maintainer git has changes that have not yet been uploaded.
On Fri, Oct 25, 2024 at 03:03:53PM +0000, Holger Levsen wrote:
the current expectation is that an NMU bug is opened, which contains
the debdiff.
https://www.debian.org/doc/manuals/developers-reference/developers-reference.en.html#when-and-how-to-do-an-nmu
"... Then, you must send a patch with the differences between the current
package and your proposed NMU to the BTS. The nmudiff script in the
devscripts package might be helpful...."
FWIW, I think this should stay the default when doing NMUs but I also think it should be (spelled out that it's) equally fine to open a MR on salsa
*if* the specific package somehow specifies this is ok.
I also think that currently no package should be able to opt-out from
getting NMUdiffs via the BTS, because it's good to have one workflow which works for *all* packages.
| Sysop: | Keyop |
|---|---|
| Location: | Huddersfield, West Yorkshire, UK |
| Users: | 429 |
| Nodes: | 16 (2 / 14) |
| Uptime: | 116:03:26 |
| Calls: | 9,056 |
| Calls today: | 3 |
| Files: | 13,395 |
| Messages: | 6,016,442 |