• Build an initial Debian source package with py2dsp from a GitHub projec

    From Sandro Tosi@21:1/5 to All on Sun Apr 25 06:30:01 2021
    Hello,
    recently i've been making some enhancements to py2dsp (part of
    pypi2deb[1] ); for those who dont know what that is, py2dsp is a tool
    that, given a PyPI project, will create an (initial) Debian source
    package.

    [1] https://packages.qa.debian.org/p/pypi2deb.html

    I've just finished a patch that extend py2dsp to fetch the upstream
    tarball from GitHub instead; nowadays this is my preferred source for
    upstream tarballs, given it contains all the project files (not only
    the one published on pypi via sdist, often missing important files
    like tests, or doc sources, etc).

    it's currently available at the git branch at [2] (there's a PR open at [3]):

    [2] https://github.com/sandrotosi/pypi2deb/tree/morph
    [3] https://github.com/p1otr/pypi2deb/pull/27

    once you cloned/checkout that branch, you can run:

    $ ./py2dsp --profile dpt --distribution unstable --revision 1 --github https://github.com/USER/PROJECT

    alternatively, you can specify an additional argument `<source name>`,
    if PROJECT is not the source name you want to use in Debian:

    $ ./py2dsp --profile dpt --distribution unstable --revision 1 --github https://github.com/USER/PROJECT <source name>

    and it will create the source package in the `result/` directory.

    Let me know if you find this useful and if there are
    issues/enhancement you'd like to see added/fixed.

    Regards,
    --
    Sandro "morph" Tosi
    My website: http://sandrotosi.me/
    Me at Debian: http://wiki.debian.org/SandroTosi
    Twitter: https://twitter.com/sandrotosi

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Antonio Terceiro@21:1/5 to Sandro Tosi on Sun Apr 25 14:40:01 2021
    On Sun, Apr 25, 2021 at 12:22:19AM -0400, Sandro Tosi wrote:
    Let me know if you find this useful and if there are
    issues/enhancement you'd like to see added/fixed.

    That's very useful, thanks!

    It would be useful if it could also be run against a tarball or git
    repository (a directory) that one has already downloaded.

    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCAAdFiEEst7mYDbECCn80PEM/A2xu81GC94FAmCFYQcACgkQ/A2xu81G C94tZQ/8C5b9iuEoHsFYLAdUD8cjR8+PXRPNAt5t2psueRJ/HmjJ6e0gKFfiuUCC nWhNUtjpY+d38kzGFJLJmuIzggY4A2qJhi/l5FrWbkl1Bxmg9HrO0Wc+0omCctvY OgXtfLiA6rkLF1icKBMBFrPJj06eLrQ743YB+3C3KIjJUW/WvxP+L7xqHgKpjNiS /zoqtUulcKrjtbNN6VOWnaS1+9kM/LuE6Nsv/FntACIMBHaxNW2ccKFgrQvWeIRr ZDasqcYFUtGMx3ncDvE7eA+lkAuQi6WRxFZcpS5m8CaQOD+MYd5dRC/ukHt+cnO5 V6pwOuvXePMZJa2zEgWGS2qw6uAp/VWpM/ssE7WBi94cKzaghqgTHrTcKo+TqP9B I6jpzzG5rGC5SUi0a7M4UdkciLPJ6OUl/jEPDrcDpqAjcFSgsk/QXAf4SJRajfB0 lyy+hUE1CXOM0y77TXrxSwRbHXJNy5kw+Z77laIkAF+gQcvaUqNjzjd1AoK2MALU C92l/vUgWsBt1Tdms2zptWeE2vB7Q3B/WwYrsb0oIn7PUe3FhoNJnIqz4GxAokGv gE34VC7r2APw32RBRXaU1u1Bguzjryw/JcuJ/CkRwxWoqiEzYzCA41eMDMScjxDX Mm9zeMCTfw6ydJIugCokQ21qpAuTTLOuRQEjHQeZab4/UmaAi9o=
    =IYCp
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Sandro Tosi@21:1/5 to All on Mon Apr 26 00:30:01 2021
    It would be useful if it could also be run against a tarball

    this is already supported (but in general by py2dsp and in the context
    of --github), f.e.:

    $ ./py2dsp --profile dpt --distribution unstable --revision 1 --gh https://github.com/indygreg/python-zstandard
    ./zstandard_0.14.1.orig.tar.gz

    uses the locally available zstandard_0.14.1.orig.tar.gz tarball (which
    is not the latest available on gh) to create the source pkg with the
    github customizations

    or git repository (a directory) that one has already downloaded.

    i dont see how starting from a git repo is useful, can you expand?

    Regards,
    --
    Sandro "morph" Tosi
    My website: http://sandrotosi.me/
    Me at Debian: http://wiki.debian.org/SandroTosi
    Twitter: https://twitter.com/sandrotosi

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Antonio Terceiro@21:1/5 to Sandro Tosi on Mon Apr 26 01:20:01 2021
    On Sun, Apr 25, 2021 at 06:19:18PM -0400, Sandro Tosi wrote:
    It would be useful if it could also be run against a tarball

    this is already supported (but in general by py2dsp and in the context
    of --github), f.e.:

    $ ./py2dsp --profile dpt --distribution unstable --revision 1 --gh https://github.com/indygreg/python-zstandard
    ./zstandard_0.14.1.orig.tar.gz

    uses the locally available zstandard_0.14.1.orig.tar.gz tarball (which
    is not the latest available on gh) to create the source pkg with the
    github customizations

    or git repository (a directory) that one has already downloaded.

    i dont see how starting from a git repo is useful, can you expand?

    instead of generating a .dsc first and then importing it into a git
    repository, it's more logical to me to import an upstream tarball into a
    git repository first (gbp import-orig), and then generate the debian
    packaging on top of that.

    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCAAdFiEEst7mYDbECCn80PEM/A2xu81GC94FAmCF9xIACgkQ/A2xu81G C96uRA//UXRpObhqPgAfGXYxuy9JIXpMG1SqhMKIFgR9jSwY4lSQ9D8NjeHwhc6a 4Etr3f/+amSsdBi8HGeY75ZSGTZpGdYLdsp8dC4jI8Vi+2LT2A5OiIxKOvf9Oocp lbRBCfXMUXC2uoUGsF6zUxjTi77IjWxT8OQE68N10GvdNw5ydXXD2kCP3ktRDZvH 4kfv4sMirGBog83QiuuyWWyzPnq9zF5h82Q9d8d6DyvsLyIJVh0zunuYsfI3PnUG CIk89CKN3mD+yn1Mm9PncRXJ9fk3pZYykdl0gLNdr36/ui2AbSFzNrFzV/mMK2Nf 4Aj7hxCU1tBexzc+RgXZ+jKTY2TD91gJFsGkyG0H02gwOxKuj4h9g9/fWArIQ1WR IQJ7vf8ezfAR/wfa5k9+CMdoBGLtP2gh7JtACV1UrahlBGuLWNGJcwg5+vtttsSO MdePAl6AOMsRNk2FaHHvbatgTuwudE9t6SLl4/+/5h7uW0myWTbI1OGog2vsXRru V3m9upAnn30P1tf6WG2oKjqXsTW8HchO3UHAKAF1kxvlasmQvd+51GVT8oiF/8j6 Pw94CYcUikz+abn6drHhq+/hpKxZX89kpRnRAtHsdI7VkL9uWyGyRF34tG/LPZNf IFbJXkslpWXZ27Em54GyKff6fz65qI34yGQaBywSauLTOYrANRg=
    =zw0C
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Sandro Tosi@21:1/5 to All on Mon Apr 26 03:30:02 2021
    or git repository (a directory) that one has already downloaded.

    i dont see how starting from a git repo is useful, can you expand?

    instead of generating a .dsc first and then importing it into a git repository, it's more logical to me to import an upstream tarball into a
    git repository first (gbp import-orig), and then generate the debian packaging on top of that.

    py2dsp does that for you: it creates a git repo, along with a source
    package, with the DPT settings (note i used the `--profile dpt`) ready
    to be pushed to salsa (the repo creation on salsa is still manual)

    Regards,
    --
    Sandro "morph" Tosi
    My website: http://sandrotosi.me/
    Me at Debian: http://wiki.debian.org/SandroTosi
    Twitter: https://twitter.com/sandrotosi

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Utkarsh Gupta@21:1/5 to morph@debian.org on Wed May 5 20:40:02 2021
    Hi Sandro,

    On Sun, Apr 25, 2021 at 9:53 AM Sandro Tosi <morph@debian.org> wrote:
    Let me know if you find this useful and if there are
    issues/enhancement you'd like to see added/fixed.

    This looks great! Thank you!
    However, I am running into an issue (or I guess I am just not doing it correctly).
    Whilst trying to package from the g/h source (https://github.com/keylime/keylime), it fails like this: http://paste.debian.net/1195339/

    Am I missing something?


    - u

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Sandro Tosi@21:1/5 to wrar@debian.org on Wed May 5 21:10:01 2021
    On Wed, May 5, 2021 at 2:58 PM Andrey Rahmatullin <wrar@debian.org> wrote:

    On Thu, May 06, 2021 at 12:08:06AM +0530, Utkarsh Gupta wrote:
    However, I am running into an issue (or I guess I am just not doing it correctly).
    Whilst trying to package from the g/h source (https://github.com/keylime/keylime), it fails like this: http://paste.debian.net/1195339/

    Am I missing something?
    As far as I can see, the change is only about getting the tarball, it
    still needs metadata from PyPI.

    that's correct, the package still needs to be on PyPI, as that's the
    place where py2dsp obtains most of the package metadata

    --
    Sandro "morph" Tosi
    My website: http://sandrotosi.me/
    Me at Debian: http://wiki.debian.org/SandroTosi
    Twitter: https://twitter.com/sandrotosi

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Andrey Rahmatullin@21:1/5 to Utkarsh Gupta on Wed May 5 21:00:02 2021
    On Thu, May 06, 2021 at 12:08:06AM +0530, Utkarsh Gupta wrote:
    However, I am running into an issue (or I guess I am just not doing it correctly).
    Whilst trying to package from the g/h source (https://github.com/keylime/keylime), it fails like this: http://paste.debian.net/1195339/

    Am I missing something?
    As far as I can see, the change is only about getting the tarball, it
    still needs metadata from PyPI.

    --
    WBR, wRAR

    -----BEGIN PGP SIGNATURE-----

    iQJhBAABCgBLFiEEolIP6gqGcKZh3YxVM2L3AxpJkuEFAmCS6q0tFIAAAAAAFQAP cGthLWFkZHJlc3NAZ251cGcub3Jnd3JhckBkZWJpYW4ub3JnAAoJEDNi9wMaSZLh WkUP/0qUSAovtCDgHDlCSyADnJHKJU2pGwPsuRNi6Dl9MoG0XpgLkQtlwgA+5tts uOBuK6XEqFaa9Oxft8arFBO6g6YmFxGjn671Wo9VWb/tzuKvjrTjN3cnAG9Edm9y 8ZZ0Ifaxt+z27LdlbMdc7/iZm5NTgaRQCYnU8LQBtMtt9e47i996Bxk2S7jT9gdI 8eCChEiwZd0moFnRCjxTS2tQxWOo+mFG4o2GD+QazzHWJrJBex1mvGNAyaotl96e B1ImSMkneuPOpTJ8IvlI+1FUGmS+kIUcNmF7H4AdXcVzPGLXFq7fJMzIH4KDb1xZ cp9VoJtG7Tj5HWfePhOfSnBw21QiU+GfuW7TNE+I4Haa8GFHK0/7rVXS1b+Ockke /dyOyMo+LZFJirJI8XEKPq2hbBKVOS7NqYsy3nywONmlFuW/P4PjxUrfltq9QOYf yyKawxQjrL2Ex2VF5XKezlY09usMnCgrN+EhQPnHtE7t735fPSRq3aGiKR3Ipo7E GIeH+KyfRYHvMI2umLqxWL8OJDEoUvdeuz5FTnpzMyofyYzu81ybBfb4LBMuBA/K +T7BLpMIL6LyLuv+Kg9lxDvUGD55hx71dNrJslBiqcYHzSfxDee864U19uUSL6iq lE9F9pq1QMDOzzA4G8hPbl5rBwW9jZimB1X4kYci2lx/iU/6
    =RCpJ
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Utkarsh Gupta@21:1/5 to morph@debian.org on Wed May 5 21:50:01 2021
    Hello,

    On Thu, May 6, 2021 at 12:33 AM Sandro Tosi <morph@debian.org> wrote:
    that's correct, the package still needs to be on PyPI, as that's the
    place where py2dsp obtains most of the package metadata

    Can we change that or have a flag or something added so that it pulls
    from g/h directly?


    - u

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Utkarsh Gupta@21:1/5 to wrar@debian.org on Wed May 5 22:00:01 2021
    On Thu, May 6, 2021 at 1:20 AM Andrey Rahmatullin <wrar@debian.org> wrote:
    But Github doesn't provide the metadata. You would need to get a tarball
    and run sdist or something like that.

    Right and thus I am wondering if we could work through this, somehow?
    That is, $something fetches the tarball, runs sdist or whatever, and
    then the py2dsp magic.

    P.S. I know this sounds a little ambitious but I believe this would
    really help, too.


    - u

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Andrey Rahmatullin@21:1/5 to Utkarsh Gupta on Wed May 5 22:00:01 2021
    On Thu, May 06, 2021 at 01:16:40AM +0530, Utkarsh Gupta wrote:
    Hello,

    On Thu, May 6, 2021 at 12:33 AM Sandro Tosi <morph@debian.org> wrote:
    that's correct, the package still needs to be on PyPI, as that's the
    place where py2dsp obtains most of the package metadata

    Can we change that or have a flag or something added so that it pulls
    from g/h directly?
    But Github doesn't provide the metadata. You would need to get a tarball
    and run sdist or something like that.

    --
    WBR, wRAR

    -----BEGIN PGP SIGNATURE-----

    iQJhBAABCgBLFiEEolIP6gqGcKZh3YxVM2L3AxpJkuEFAmCS9vctFIAAAAAAFQAP cGthLWFkZHJlc3NAZ251cGcub3Jnd3JhckBkZWJpYW4ub3JnAAoJEDNi9wMaSZLh I/0P/2XbfeAFTkFhO99ns9yehTEc0WIO+PgqKKSTUo5dRFS8JCChM3Axq7cwCB71 fpKI5X2qHXw1CruePj6ds82yn6YN6w/5x9Y2G9SbFoN7pALxOVb0dGTPfSS1EbWI pmOeyFAvf1Fl7ik3gLl/5kfLPFgXaR2a4zTeKetbgCJXNFfVSVmyQ28jPUYsR/3M TF/VUr6kCKzi5pJK7k8dCFiw7iVvpF+BfS7Ng9gm6VlhzXh0cLMNlI8pg12msAJK kesCWVu2rnc7s5gN/TM11VAaqZHow/O1R8gzEINApkQsndUWxhZIi3gSR0COAj3v /PURVWN+JmknZ0v3uBLJVedHGbCnq19I7A89S8JmHMyWHEtwLxv6jIE3yEJ76R9/ ngf+zJ4baU/gpqgg+i4+CS5zllWyIMubAH7kK3tc4MlaR+hAbdClLf0tleFzMOkO r1d9YGMMla5l8ctquUn8+0s3q5ozd49Z+K5Aw/ksRakn/aZL4KwvBMHMiVjJoL0a MG+B67BKVz7prXWLNoat29OXIKJ7lL1+xPAD39KsWu1Ln1nz7EZyDFr9upwM67Eo Ct0DdKF5SiZRHWtBzH0i/Cv+AYWAU9CZ7vwUjVj53uFb5ooJTRJRxjG3ZFpLTX6L 66un/0qi6ynyIjlz0fw3gEoGbDeqeSxpr1xKPoGljvzhBENY
    =EVVW
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Paul Wise@21:1/5 to All on Thu May 6 00:50:01 2021
    On Wed, May 5, 2021 at 10:14 PM Sandro Tosi wrote:

    this solution also underestimates the in-progress migration towards
    poetry and pyproject.toml, where `python3 setup.py sdist` is not
    available.

    Where does the metadata come from for projects using these things?

    --
    bye,
    pabs

    https://wiki.debian.org/PaulWise

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Sandro Tosi@21:1/5 to All on Thu May 6 01:10:01 2021
    this solution also underestimates the in-progress migration towards
    poetry and pyproject.toml, where `python3 setup.py sdist` is not
    available.

    Where does the metadata come from for projects using these things?

    that'd be pyproject.toml AFAIUI

    the point i wanted to make is that it would require to implement a
    parser for setup.{py,cfg} and pyproject.toml, plus some way to decide
    which one to use in case a project supports both (it happens) and how
    to override the selection in case one system is available and outdated
    (it happens).

    All of this is done by PyPI already, and the fact a project exists on
    GH but not on PyPI it's either because it's such a niche project or
    it's still under heaving development, that working on a solution
    *right now* is not a good use of our time.

    --
    Sandro "morph" Tosi
    My website: http://sandrotosi.me/
    Me at Debian: http://wiki.debian.org/SandroTosi
    Twitter: https://twitter.com/sandrotosi

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Sandro Tosi@21:1/5 to All on Thu May 6 00:20:02 2021
    Right and thus I am wondering if we could work through this, somehow?
    That is, $something fetches the tarball, runs sdist or whatever, and
    then the py2dsp magic.

    P.S. I know this sounds a little ambitious but I believe this would
    really help, too.

    i do not plan to implement such a feature.

    this solution also underestimates the in-progress migration towards
    poetry and pyproject.toml, where `python3 setup.py sdist` is not
    available.

    --
    Sandro "morph" Tosi
    My website: http://sandrotosi.me/
    Me at Debian: http://wiki.debian.org/SandroTosi
    Twitter: https://twitter.com/sandrotosi

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Sandro Tosi@21:1/5 to morph@debian.org on Fri Oct 29 06:50:01 2021
    couple of features recently introduced that may interest people:

    - rudimental support for autopkgtests: autodep8 + a stub for
    developers to setup autopkgstests for unittests
    - salsa repo creation, if SALSA_TOKEN is available in ~/.devscripts;
    this also setup the KGB webhook to post in #d-p-changes

    I'm definitely no expert in autopkgtests, so if there's something to
    improve, lemme know.

    On Sun, Apr 25, 2021 at 12:22 AM Sandro Tosi <morph@debian.org> wrote:

    Hello,
    recently i've been making some enhancements to py2dsp (part of
    pypi2deb[1] ); for those who dont know what that is, py2dsp is a tool
    that, given a PyPI project, will create an (initial) Debian source
    package.

    [1] https://packages.qa.debian.org/p/pypi2deb.html

    I've just finished a patch that extend py2dsp to fetch the upstream
    tarball from GitHub instead; nowadays this is my preferred source for upstream tarballs, given it contains all the project files (not only
    the one published on pypi via sdist, often missing important files
    like tests, or doc sources, etc).

    it's currently available at the git branch at [2] (there's a PR open at [3]):

    [2] https://github.com/sandrotosi/pypi2deb/tree/morph
    [3] https://github.com/p1otr/pypi2deb/pull/27

    once you cloned/checkout that branch, you can run:

    $ ./py2dsp --profile dpt --distribution unstable --revision 1 --github https://github.com/USER/PROJECT

    alternatively, you can specify an additional argument `<source name>`,
    if PROJECT is not the source name you want to use in Debian:

    $ ./py2dsp --profile dpt --distribution unstable --revision 1 --github https://github.com/USER/PROJECT <source name>

    and it will create the source package in the `result/` directory.

    Let me know if you find this useful and if there are
    issues/enhancement you'd like to see added/fixed.

    Regards,
    --
    Sandro "morph" Tosi
    My website: http://sandrotosi.me/
    Me at Debian: http://wiki.debian.org/SandroTosi
    Twitter: https://twitter.com/sandrotosi



    --
    Sandro "morph" Tosi
    My website: http://sandrotosi.me/
    Me at Debian: http://wiki.debian.org/SandroTosi
    Twitter: https://twitter.com/sandrotosi

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)