Hi,

when I try to visit

https://alkesgroup.broadinstitute.org/BOLT-LMM/downloads/

with any browser I get a list of files. However, the watch file
of bolt-lmm[1] pointing to that web page returns:

$ uscan --verbose --report
uscan info: uscan (version 2.21.3) See uscan(1) for help
uscan info: Scan watch files in .
uscan info: Check debian/watch and debian/changelog in .
uscan info: package="bolt-lmm" version="2.3.5+dfsg-2" (as seen in debian/changelog)
uscan info: package="bolt-lmm" version="2.3.5+dfsg" (no epoch/revision)
uscan info: ./debian/changelog sets package="bolt-lmm" version="2.3.5+dfsg" uscan info: Process watch file at: debian/watch
package = bolt-lmm
version = 2.3.5+dfsg
pkg_dir = .
uscan info: opts: repacksuffix=+dfsg,dversionmangle=s/\+dfsg//g,repack,compression=xz
uscan info: line: https://alkesgroup.broadinstitute.org/BOLT-LMM/downloads/ .*/BOLT-LMM_v(?:[-_]?(\d[\-+\.:\~\da-zA-Z]*))(?i)(?:\.(?:tar\.xz|tar\.bz2|tar\.gz|zip|tgz|tbz|txz))
uscan info: Parsing repacksuffix=+dfsg
uscan info: Parsing dversionmangle=s/\+dfsg//g
uscan info: Parsing repack
uscan info: Parsing compression=xz
uscan info: line: https://alkesgroup.broadinstitute.org/BOLT-LMM/downloads/ .*/BOLT-LMM_v(?:[-_]?(\d[\-+\.:\~\da-zA-Z]*))(?i)(?:\.(?:tar\.xz|tar\.bz2|tar\.gz|zip|tgz|tbz|txz))
uscan info: Last orig.tar.* tarball version (from debian/changelog): 2.3.5+dfsg uscan info: Last orig.tar.* tarball version (dversionmangled): 2.3.5
uscan info: Requesting URL:
https://alkesgroup.broadinstitute.org/BOLT-LMM/downloads/
uscan warn: In watchfile debian/watch, reading webpage
https://alkesgroup.broadinstitute.org/BOLT-LMM/downloads/ failed: 404 Not Found
uscan info: Scan finished

Any idea how to help uscan reading that web page?

Kind regards

Andreas.


[1] https://salsa.debian.org/med-team/bolt-lmm/-/blob/master/debian/watch

--
http://fam-tille.de

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi,

On Sat, Oct 09, 2021 at 09:46:51PM +0200, Andreas Tille wrote:

Hi,

when I try to visit

https://alkesgroup.broadinstitute.org/BOLT-LMM/downloads/

with any browser I get a list of files. However, the watch file
of bolt-lmm[1] pointing to that web page returns:

You don't get a list of files. You get the same 404. Only there is
JavaScript on the page that in turn loads and renders the list of
files…

Any idea how to help uscan reading that web page?

https://storage.googleapis.com/broad-alkesgroup-public?delimiter=/&prefix=BOLT-LMM/downloads/
gets you a step further.


It's a brave new world!

-nik

-----BEGIN PGP SIGNATURE-----

iKcEABYKAE8WIQSk6zxRYJYchegBkTEK5VTlRg4b3QUCYWHzhjEaaHR0cHM6Ly93 d3cuZG9taW5pay1nZW9yZ2UuZGUvZ3BnLXBvbGljeS50eHQuYXNjAAoJEArlVOVG DhvdjQUBAJDdR+kwYe5DMw9Z+l4poaflM93DNyP5h9FNLuAIMS3LAQDJOTIZ2X4m BaBRB8OsxNZJB84MNZUBcXK6fbohE+YFAQ==
=2KiL
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

--an72wnjfd6xp6zdc
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Any idea how to help uscan reading that web page?

https://storage.googleapis.com/broad-alkesgroup-public?delimiter=/&prefix=BOLT-LMM/downloads/
gets you a step further.

Digging a bit, there is actually an example for this in the uscan man
page (grep for AWS).

Find attached a patch for your package ☺.

*snip*
uscan info: => Package is up to date from:
=> https://storage.googleapis.com/BOLT-LMM/downloads/BOLT-LMM_v2.3.5.tar.gz
uscan info: => Forcing download as requested
uscan info: Downloading upstream package: BOLT-LMM_v2.3.5.tar.gz
*snip*

HTH,
Nik

--an72wnjfd6xp6zdc
Content-Type: text/x-diff; charset=utf-8
Content-Disposition: attachment;
filename="0001-Correctly-scan-S3-bucket-in-d-watch.patch" Content-Transfer-Encoding: quoted-printable

From 8179eb9c8fc787d8d32d5fb5cf9ac9d604af3658 Mon Sep 17 00:00:00 2001
From: Dominik George <natureshadow@debian.org>
Date: Sat, 9 Oct 2021 22:07:39 +0200
Subject: [PATCH] Correctly scan S3 bucket in d/watch

---
debian/changelog | 4 +++-
debian/watch | 4 ++--
2 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/debian/changelog b/debian/changelog
index 879f2a0..d2abc9c 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,7 +1,9 @@
bolt-lmm (2.3.5+dfsg-2) UNRELEASED; urgency=medium

* Team upload.
- TODO: Try to fix watch file ... but failed
+
+ [ Dominik George ]
+ * Scan and mangle S3 bucket list in d/watch.

-- Andreas Tille <tille@debian.org> Sat, 09 Oct 2021 08:29:36 +0200

diff --git a/debian/watch b/debian/watch
index e1e1447..761a3a6 100644
--- a/debian/watch
+++ b/debian/watch
@@ -1,3 +1,3 @@
version=4 -opts="repacksuffix=+dfsg,dversionmangle=s/\+dfsg//g,repack,compression=xz" \
- https://alkesgroup.broadinstitute.org/BOLT-LMM/downloads/ .*/BOLT-LMM_v@ANY_VERSION@@ARCHIVE_EXT@
+opts="repacksuffix=+dfsg,dversionmangle=s/\+dfsg//g,pagemangle=s%<Key>([^<]*)</Key>%<Key><a href="$1">$1</a></Key>%g,repack,compression=xz" \
+ https://storage.goo

Hi Dominik,

Am Sat, Oct 09, 2021 at 10:11:23PM +0200 schrieb Dominik George:

Digging a bit, there is actually an example for this in the uscan man
page (grep for AWS).

Find attached a patch for your package ☺.

Thanks a lot for the really quick and welcome help

Andreas.

--
http://fam-tille.de

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)