• Making py2dsp/pypi2deb the default tool to create Python-based packages

    From Sandro Tosi@21:1/5 to All on Fri Jul 8 05:20:01 2022
    Hello,
    Piotr has kindly moved pypi2deb to salsa[1] and given me access to the
    project so i was able to merge my changes and release[2] a new version
    of this tool in Debian.

    [1] https://salsa.debian.org/python-team/tools/pypi2deb
    [2] https://tracker.debian.org/news/1343951/accepted-pypi2deb-320220707-source-into-unstable/

    My goal is to make py2dsp (contained in pypi2deb) the default tool
    used to create Python packages in Debian (like many other
    language-specific tools already do f.e. for go, rust, npm, etc). The
    new release contains several enhancements that should cover many of
    the packaging needs, in particular:

    * the ability to package directly from a github project url
    * create the salsa project in the DPT group

    Please let me know if you think something is missing, or should be expanded/fix, you can also open bugs against the package or directly
    MR to the salsa project.

    Should we start advertising this tool in other locations, like the
    python policy, guidelines, wiki, etc? what do y'all think?

    Thanks,
    --
    Sandro "morph" Tosi
    My website: http://sandrotosi.me/
    Me at Debian: http://wiki.debian.org/SandroTosi
    Twitter: https://twitter.com/sandrotosi

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Emmanuel Arias@21:1/5 to All on Fri Jul 8 13:20:01 2022
    Hi!

    El vie, 8 jul 2022 a la(s) 00:10, Sandro Tosi (morph@debian.org) escribió:

    Hello,
    Piotr has kindly moved pypi2deb to salsa[1] and given me access to the project so i was able to merge my changes and release[2] a new version
    of this tool in Debian.

    [1] https://salsa.debian.org/python-team/tools/pypi2deb
    [2] https://tracker.debian.org/news/1343951/accepted-pypi2deb-320220707-source-into-unstable/

    My goal is to make py2dsp (contained in pypi2deb) the default tool
    used to create Python packages in Debian (like many other
    language-specific tools already do f.e. for go, rust, npm, etc). The
    new release contains several enhancements that should cover many of
    the packaging needs, in particular:

    * the ability to package directly from a github project url
    * create the salsa project in the DPT group


    Great, this is a very useful tool. I used your fork and cover all that I
    need


    Please let me know if you think something is missing, or should be expanded/fix, you can also open bugs against the package or directly
    MR to the salsa project.

    Should we start advertising this tool in other locations, like the
    python policy, guidelines, wiki, etc? what do y'all think?


    I think that it should be mentioned at least here [0][1]. IMO they're the
    first
    resources that are checked when someone starts packaging Python modules in Debian.

    [0] https://www.debian.org/doc/packaging-manuals/python-policy/
    [1] https://wiki.debian.org/Python/LibraryStyleGuide

    Cheers,
    Emmanuel


    Thanks,
    --
    Sandro "morph" Tosi
    My website: http://sandrotosi.me/
    Me at Debian: http://wiki.debian.org/SandroTosi
    Twitter: https://twitter.com/sandrotosi



    <div dir="ltr"><div dir="ltr"><div style="font-family:monospace;font-size:small" class="gmail_default">Hi!</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">El vie, 8 jul 2022 a la(s) 00:10, Sandro Tosi (<a href="mailto:morph@
    debian.org">morph@debian.org</a>) escribió:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hello,<br>
    Piotr has kindly moved pypi2deb to salsa[1] and given me access to the<br> project so i was able to merge my changes and release[2] a new version<br>
    of this tool in Debian.<br>

    [1] <a href="https://salsa.debian.org/python-team/tools/pypi2deb" rel="noreferrer" target="_blank">https://salsa.debian.org/python-team/tools/pypi2deb</a><br>
    [2] <a href="https://tracker.debian.org/news/1343951/accepted-pypi2deb-320220707-source-into-unstable/" rel="noreferrer" target="_blank">https://tracker.debian.org/news/1343951/accepted-pypi2deb-320220707-source-into-unstable/</a><br>

    My goal is to make py2dsp (contained in pypi2deb) the default tool<br>
    used to create Python packages in Debian (like many other<br>
    language-specific tools already do f.e. for go, rust, npm, etc). The<br>
    new release contains several enhancements that should cover many of<br>
    the packaging needs, in particular:<br>

    * the ability to package directly from a github project url<br>
    * create the salsa project in the DPT group<br></blockquote><div><span class="gmail_default" style="font-family:monospace;font-size:small"><br></span></div><div><span class="gmail_default" style="font-family:monospace;font-size:small">Great, this is a
    very useful tool. I used your fork and cover all that I need</span> <br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">

    Please let me know if you think something is missing, or should be<br> expanded/fix, you can also open bugs against the package or directly<br>
    MR to the salsa project.<br>

    Should we start advertising this tool in other locations, like the<br>
    python policy, guidelines, wiki, etc? what do y&#39;all think?<br></blockquote><div><br></div><div style="font-family:monospace;font-size:small" class="gmail_default">I think that it should be mentioned at least here [0][1]. IMO they&#39;re the first</
    <div style="font-family:monospace;font-size:small" class="gmail_default">resources that are checked when someone starts packaging Python modules in Debian.</div><div style="font-family:monospace;font-size:small" class="gmail_default"></div><div><br></
    <div><div style="font-family:monospace;font-size:small" class="gmail_default">[0] <a href="https://www.debian.org/doc/packaging-manuals/python-policy/">https://www.debian.org/doc/packaging-manuals/python-policy/</a></div></div><div style="font-family:
    monospace;font-size:small" class="gmail_default">[1] <a href="https://wiki.debian.org/Python/LibraryStyleGuide">https://wiki.debian.org/Python/LibraryStyleGuide</a></div><div style="font-family:monospace;font-size:small" class="gmail_default"><br></div><
    div style="font-family:monospace;font-size:small" class="gmail_default">Cheers,</div><div style="font-family:monospace;font-size:small" class="gmail_default">Emmanuel<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:
    1px solid rgb(204,204,204);padding-left:1ex">

    Thanks,<br>
    -- <br>
    Sandro &quot;morph&quot; Tosi<br>
    My website: <a href="http://sandrotosi.me/" rel="noreferrer" target="_blank">http://sandrotosi.me/</a><br>
    Me at Debian: <a href="http://wiki.debian.org/SandroTosi" rel="noreferrer" target="_blank">http://wiki.debian.org/SandroTosi</a><br>
    Twitter: <a href="https://twitter.com/sandrotosi" rel="noreferrer" target="_blank">https://twitter.com/sandrotosi</a><br>

    </blockquote></div></div>

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Andreas Tille@21:1/5 to All on Thu Jul 21 17:30:01 2022
    Hi Sandro,

    Am Thu, Jul 07, 2022 at 11:10:01PM -0400 schrieb Sandro Tosi:
    My goal is to make py2dsp (contained in pypi2deb) the default tool
    used to create Python packages in Debian (like many other
    language-specific tools already do f.e. for go, rust, npm, etc). The
    new release contains several enhancements that should cover many of
    the packaging needs, in particular:

    Sounds really great!

    I just gave it a try:

    $ py2dsp -v pystow
    D: py2dsp py2dsp:156: version: 3.20220707
    D: py2dsp py2dsp:157: ['/usr/bin/py2dsp', '-v', 'pystow']
    /usr/bin/py2dsp:163: DeprecationWarning: There is no current event loop
    loop = asyncio.get_event_loop()
    D: py2dsp py2dsp:44: args: Namespace(verbose=True, quiet=False, root='/home/andreas/debian-maintain/salsa/python-team/packages/0_prospective/result', clean=False, build=False, application=False, profile=None, github=None, distribution='UNRELEASED',
    revision='0~py2deb', message='converte0~py2deb', name='pystow')
    E: py2dsp py2dsp:167: 'releases'
    Traceback (most recent call last):
    File "/usr/bin/py2dsp", line 165, in <module>
    loop.run_until_complete(main(args))
    File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
    File "/usr/bin/py2dsp", line 74, in main
    fname = yield from download(name, version=version, destdir=args.root)
    File "/usr/share/pypi2deb/pypi2deb/pypi.py", line 124, in download
    release = details['releases'].get(version, {})
    KeyError: 'releases'


    Am I missing something?

    Kind regards

    Andreas.

    --
    http://fam-tille.de

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Sandro Tosi@21:1/5 to All on Thu Jul 21 21:00:01 2022
    $ py2dsp -v pystow
    D: py2dsp py2dsp:156: version: 3.20220707
    D: py2dsp py2dsp:157: ['/usr/bin/py2dsp', '-v', 'pystow'] /usr/bin/py2dsp:163: DeprecationWarning: There is no current event loop
    loop = asyncio.get_event_loop()
    D: py2dsp py2dsp:44: args: Namespace(verbose=True, quiet=False, root='/home/andreas/debian-maintain/salsa/python-team/packages/0_prospective/result', clean=False, build=False, application=False, profile=None, github=None, distribution='UNRELEASED',
    revision='0~py2deb', message='converte0~py2deb', name='pystow')
    E: py2dsp py2dsp:167: 'releases'
    Traceback (most recent call last):
    File "/usr/bin/py2dsp", line 165, in <module>
    loop.run_until_complete(main(args))
    File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
    File "/usr/bin/py2dsp", line 74, in main
    fname = yield from download(name, version=version, destdir=args.root)
    File "/usr/share/pypi2deb/pypi2deb/pypi.py", line 124, in download
    release = details['releases'].get(version, {})
    KeyError: 'releases'

    a fix for this was available in the git repo but not released, i took
    care of that with version 3.20220721 that has just been ACCEPTED.

    --
    Sandro "morph" Tosi
    My website: http://sandrotosi.me/
    Me at Debian: http://wiki.debian.org/SandroTosi
    Twitter: https://twitter.com/sandrotosi

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Andreas Tille@21:1/5 to All on Thu Jul 21 23:00:01 2022
    Am Thu, Jul 21, 2022 at 02:52:38PM -0400 schrieb Sandro Tosi:

    a fix for this was available in the git repo but not released, i took
    care of that with version 3.20220721 that has just been ACCEPTED.

    Before I've sent my mail I also checked Git HEAD which was no change.
    Anyway, thanks for the quick response. However, there seems to be
    something wrong:


    $ apt policy pypi2deb | grep Install
    Installed: 3.20220721
    $ py2dsp --version
    py2dsp 3.20220707
    $ pypi2debian --version
    /usr/bin/pypi2debian:63: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
    def run(self):
    /usr/bin/pypi2debian:101: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
    def worker(self):
    /usr/bin/pypi2debian:169: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
    def build_src_worker(self):
    /usr/bin/pypi2debian:189: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
    def build_bin_worker(self):
    pypi2debian 3.20220707


    Are you sure thet the package version 3.20220721 contains the correct executables?

    Kind regards

    Andreas.

    --
    http://fam-tille.de

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Sandro Tosi@21:1/5 to All on Fri Jul 22 00:00:01 2022
    Before I've sent my mail I also checked Git HEAD which was no change.

    there was: https://salsa.debian.org/python-team/tools/pypi2deb/-/commit/f9eda106f1514a1fff83fb3a8324817a91489879

    Are you sure thet the package version 3.20220721 contains the correct executables?

    yes, i simply forgot to bump the internal version; what really matters
    is: does the last release fix the problem you were having?

    --
    Sandro "morph" Tosi
    My website: http://sandrotosi.me/
    Me at Debian: http://wiki.debian.org/SandroTosi
    Twitter: https://twitter.com/sandrotosi

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Andreas Tille@21:1/5 to All on Fri Jul 22 09:40:01 2022
    Am Thu, Jul 21, 2022 at 05:50:05PM -0400 schrieb Sandro Tosi:
    Are you sure thet the package version 3.20220721 contains the correct executables?

    yes, i simply forgot to bump the internal version; what really matters
    is: does the last release fix the problem you were having?

    No, the problem persists:

    $ py2dsp -v pystow
    D: py2dsp py2dsp:156: version: 3.20220707
    D: py2dsp py2dsp:157: ['/usr/bin/py2dsp', '-v', 'pystow']
    /usr/bin/py2dsp:163: DeprecationWarning: There is no current event loop
    loop = asyncio.get_event_loop()
    D: py2dsp py2dsp:44: args: Namespace(verbose=True, quiet=False, root='/tmp/result', clean=False, build=False, application=False, profile=None, github=None, distribution='UNRELEASED', revision='0~py2deb', message='converte0~py2deb', name='pystow')
    E: py2dsp py2dsp:167: 'releases'
    Traceback (most recent call last):
    File "/usr/bin/py2dsp", line 165, in <module>
    loop.run_until_complete(main(args))
    File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
    File "/usr/bin/py2dsp", line 74, in main
    fname = yield from download(name, version=version, destdir=args.root)
    File "/usr/share/pypi2deb/pypi2deb/pypi.py", line 124, in download
    release = details['releases'].get(version, {})
    KeyError: 'releases'


    Kind regards

    Andreas.

    --
    http://fam-tille.de

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Sandro Tosi@21:1/5 to All on Sat Jul 23 05:50:01 2022
    No, the problem persists:

    $ py2dsp -v pystow
    D: py2dsp py2dsp:156: version: 3.20220707
    D: py2dsp py2dsp:157: ['/usr/bin/py2dsp', '-v', 'pystow'] /usr/bin/py2dsp:163: DeprecationWarning: There is no current event loop
    loop = asyncio.get_event_loop()
    D: py2dsp py2dsp:44: args: Namespace(verbose=True, quiet=False, root='/tmp/result', clean=False, build=False, application=False, profile=None, github=None, distribution='UNRELEASED', revision='0~py2deb', message='converte0~py2deb', name='pystow')
    E: py2dsp py2dsp:167: 'releases'
    Traceback (most recent call last):
    File "/usr/bin/py2dsp", line 165, in <module>
    loop.run_until_complete(main(args))
    File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
    File "/usr/bin/py2dsp", line 74, in main
    fname = yield from download(name, version=version, destdir=args.root)
    File "/usr/share/pypi2deb/pypi2deb/pypi.py", line 124, in download
    release = details['releases'].get(version, {})
    KeyError: 'releases'

    this seems to be related to https://discuss.python.org/t/backwards-incompatible-change-to-pypi-json-api/17154
    , although they say /pypi/<project>/json (what py2dsp uses to gather
    the latest released verison) still contains the releases key, what i
    noticed is that endpoint now returns 2 concatenated jsons, and aiohttp
    json() (quite understandably) returns the latest one, which does not
    contain releases.

    appreciate if you can log a bug via reportbug

    --
    Sandro "morph" Tosi
    My website: http://sandrotosi.me/
    Me at Debian: http://wiki.debian.org/SandroTosi
    Twitter: https://twitter.com/sandrotosi

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Andreas Tille@21:1/5 to All on Sat Jul 23 08:40:01 2022
    Hi Sandro,

    Am Fri, Jul 22, 2022 at 11:39:50PM -0400 schrieb Sandro Tosi:

    this seems to be related to https://discuss.python.org/t/backwards-incompatible-change-to-pypi-json-api/17154
    , although they say /pypi/<project>/json (what py2dsp uses to gather
    the latest released verison) still contains the releases key, what i
    noticed is that endpoint now returns 2 concatenated jsons, and aiohttp
    json() (quite understandably) returns the latest one, which does not
    contain releases.

    appreciate if you can log a bug via reportbug

    Done (#1015888).

    I wonder whether you are able to reproduce the issue at your side since
    in one of your last mails you asked whether the new version might have
    fixed the issue. This might implicitly mean it works for you since I
    assume you fired up the command line at your side as well.

    Kind regards

    Andreas.

    --
    http://fam-tille.de

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Sandro Tosi@21:1/5 to All on Sat Jul 23 16:30:02 2022
    I wonder whether you are able to reproduce the issue at your side since
    in one of your last mails you asked whether the new version might have
    fixed the issue. This might implicitly mean it works for you since I
    assume you fired up the command line at your side as well.

    it never worked.

    --
    Sandro "morph" Tosi
    My website: http://sandrotosi.me/
    Me at Debian: http://wiki.debian.org/SandroTosi
    Twitter: https://twitter.com/sandrotosi

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Andreas Tille@21:1/5 to All on Sat Jul 23 18:00:02 2022
    Hi,

    Am Sat, Jul 23, 2022 at 10:21:40AM -0400 schrieb Sandro Tosi:
    I wonder whether you are able to reproduce the issue at your side since
    in one of your last mails you asked whether the new version might have fixed the issue. This might implicitly mean it works for you since I assume you fired up the command line at your side as well.

    it never worked.

    So is there anything I could do right now to automatically create pystow packaging or should I do it manually for the moment. Since you
    advertised this tool here I wonder whether I did something wrong when
    testing it.

    Kind regards

    Andreas.

    --
    http://fam-tille.de

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Andreas Tille@21:1/5 to All on Mon Jul 25 10:10:01 2022
    Hi Sandro,

    I was able to run

    py2dsp pystow

    with a sufficient result for my purposes with the following patch:

    diff --git a/pypi2deb/pypi.py b/pypi2deb/pypi.py
    index 3e342c0..0639de3 100644
    --- a/pypi2deb/pypi.py
    +++ b/pypi2deb/pypi.py
    @@ -38,8 +38,8 @@ log = logging.getLogger('pypi2deb')
    @asyncio.coroutine
    def get_pypi_info(name, version=None):
    url = PYPI_JSON_URL + '/' + name
    - if version:
    - url += '/' + version
    +# if version:
    +# url += '/' + version
    url += '/json'
    session = None
    try:

    Since I do not really understand pypi2deb and what role version might have I do not step further (MR or so) from this point but leave it to you to decide whether
    this is useful or not.

    Kind regards

    Andreas.

    --
    http://fam-tille.de

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Andreas Tille@21:1/5 to All on Mon Jul 25 11:20:01 2022
    Am Mon, Jul 25, 2022 at 10:09:05AM +0200 schrieb Andreas Tille:
    I was able to run

    py2dsp pystow

    with a sufficient result for my purposes...

    While I've read here the suggestion to fetch the Tarball rather vom Github than from pypi (and I agree with this) I would suggest to implement this suggestion in what is planed to become a default tool. With the command above the watch file points to pypi and thus I tried:

    $ py2dsp --github pystow
    /usr/bin/py2dsp:163: DeprecationWarning: There is no current event loop
    loop = asyncio.get_event_loop()
    E: py2dsp py2dsp:167: 404 {"message": "Not Found", "documentation_url": "https://docs.github.com/rest"}


    Since the original thread started with opinions to make the tool the default tool I'd suggest waiting a bit until it

    a) does not throw DeprecationWarnings
    b) works out of the box for simple usage

    IMHO the suggestions where it should be advertised are perfectly fine
    but its a bit bad timing when people stumble over it and run into
    trouble at first usage.

    Thanks for working on such a promising tools anyway

    Andreas.

    --
    http://fam-tille.de

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)