• WNPP/ITP/... for Amazon SDK for C++ ?

    From Dirk Eddelbuettel@21:1/5 to All on Mon Dec 27 01:20:01 2021
    Does anybody know where we are with respect to the WNPPs / ITPs / ... on the Amazon SDK for C++?

    I have a package that could take advantage of this if it were packaged, and I am sure a number of other packages are in a similar situation given how pervasive AWS use is. So does anybody know where this is at?

    FWIW I have packaged _subsets_ of the C++ SDK informally for my own use (also at Launchpad) but I don't think I have the time and energy to take this on as another package.

    Please CC me on replies as I am no longer subscribed to debian-devel.

    Cheers, Dirk

    --
    https://dirk.eddelbuettel.com | @eddelbuettel | edd@debian.org

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Noah Meyerhans@21:1/5 to Dirk Eddelbuettel on Mon Dec 27 05:20:01 2021
    On Sun, Dec 26, 2021 at 06:08:56PM -0600, Dirk Eddelbuettel wrote:
    I have a package that could take advantage of this if it were packaged, and I am sure a number of other packages are in a similar situation given how pervasive AWS use is. So does anybody know where this is at?

    FWIW I have packaged _subsets_ of the C++ SDK informally for my own use (also at Launchpad) but I don't think I have the time and energy to take this on as another package.

    The cloud-team could probably be a reasonable umbrella under which the
    package could be co-maintained. Would you be interested in
    co-maintaining it as part of that team? I'm not sure any of us have a
    lot of expertise or desire to work with C++, but we've got a lot of
    familiarity with cloud services, etc, and maintain other cloud service
    SDKs. If you can contribute on the C++ side, we could probably
    effectively maintain the package together.

    Of course, if others are already looking into packaging, they should by
    all means continue, with or without coordinating with the cloud-team.

    noah

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Dirk Eddelbuettel@21:1/5 to Noah Meyerhans on Mon Dec 27 15:20:01 2021
    Hi Noah,

    On 26 December 2021 at 20:18, Noah Meyerhans wrote:
    | On Sun, Dec 26, 2021 at 06:08:56PM -0600, Dirk Eddelbuettel wrote:
    | > I have a package that could take advantage of this if it were packaged, and I
    | > am sure a number of other packages are in a similar situation given how
    | > pervasive AWS use is. So does anybody know where this is at?
    | >
    | > FWIW I have packaged _subsets_ of the C++ SDK informally for my own use (also
    | > at Launchpad) but I don't think I have the time and energy to take this on as
    | > another package.
    |
    | The cloud-team could probably be a reasonable umbrella under which the
    | package could be co-maintained. Would you be interested in
    | co-maintaining it as part of that team? I'm not sure any of us have a
    | lot of expertise or desire to work with C++, but we've got a lot of
    | familiarity with cloud services, etc, and maintain other cloud service
    | SDKs. If you can contribute on the C++ side, we could probably
    | effectively maintain the package together.

    That could work. Would you be amenable to the subset packaging I have done
    (as opposed to grabbing all of the (rather large) SDK at GitHub) ?

    My (very informal) packaging has always been in the open (but on GitHub). A possible first step might be to review the added files in debian/ and in a first pass edit out all references to 'informal' or 'unofficial' packaging
    and making them more official in the cloud team repo -- which I presume is on salsa? If you or others want to look, I have this currently at github.com in

    eddelbuettel/pkg-aws-c-common common C layer
    eddelbuettel/pkg-aws-checksums checksum for transport
    eddelbuettel/pkg-aws-c-event-stream another C layer
    eddelbuettel/pkg-aws-sdk-cpp-only-s3 C++ SDK subset for S3

    The last repo is the one that is 'incomplete' as I skipped everything not
    need by my use case of accessing s3 programmatically from other C++ code.
    From a casual look at another C++ project I noticed that it had the same subset so this may make sense. It would allow us to proceed and start with
    the subset and add as needed (as opposed to be overwhelmed by 'all of it').

    Now, these are also not current as I packaged 'when I needed them' / 'when a change happened'. Upstream releases something every few days which we may or may not want to keep track of.

    I have however built them a few times over via launchpad for amd64. I do not know how they would fare on less common architectures. At least the checksum repo has IIRC some assembler code specific to the common platforms. Is that aspect something the cloud team had to deal with for other SDKs?

    | Of course, if others are already looking into packaging, they should by
    | all means continue, with or without coordinating with the cloud-team.

    Fully agreed. But if nobody steps forward this may be a viable addition.

    Cheers, Dirk

    --
    https://dirk.eddelbuettel.com | @eddelbuettel | edd@debian.org

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Wookey@21:1/5 to Dirk Eddelbuettel on Fri Dec 31 00:40:02 2021
    On 2021-12-26 18:08 -0600, Dirk Eddelbuettel wrote:

    Does anybody know where we are with respect to the WNPPs / ITPs / ... on the Amazon SDK for C++?

    I have a package that could take advantage of this if it were packaged, and I am sure a number of other packages are in a similar situation given how pervasive AWS use is.

    I don't know anything about the state of this, but am in the same
    place, in that I am packaging something that uses awsSDK pieces,
    specifically tensorflow which wants aws-cpp-sdk-core.

    FWIW I have packaged _subsets_ of the C++ SDK informally for my own use (also at Launchpad) but I don't think I have the time and energy to take this on as another package.

    Which bits have you done? They may or may not be sufficient to satisfy tensorflow.

    I too, am not bursting to add large packages that I don't use (or even
    properly know what they are) to my collection of responsibilities, but
    I can spend work time on helping get this going so I don't mind
    mucking in on some packaging work.

    Wookey
    --
    Principal hats: Linaro, Debian, Wookware, ARM
    http://wookware.org/

    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCgAdFiEER4nvI8Pe/wVWh5yq+4YyUahvnkcFAmHOQYgACgkQ+4YyUahv nkdnzQ/8D42nYpvlWldpYh64121rnUUSMXSXwbXMQcylBfy8VatyILk5rJZOMYls Se3+OeeC/Enqa5R6CcwiBVjl0Yez04WwtJ0QmXinA1M03zU6BFg9hckHRyvwSoRy aGm2VofwcN8ivJBIIj98qjPijShtV/SWEX63DgDUS7J2qfFRt82zzX/i5tMscMzC KBC4FjrVyVGp7YDbiuv2nEeHM80x8q3mp0Ux9LRbxDnNThWzW+22Ol9FSp1am1iA MG+5oQJDVv6qfvul8RJtmH0CwzusyWyei3zSHekd7sT11wUzkZCnSPy9vofCLytS 0n5M5dLxerFFGHq+QZoIo/6er3H26Zk0g8iJ29UYsLPWAQQvC3jFNHFMv9BSp1FQ ebfPpihcoEoqpqkRS9DDhKswaq5KCHyXve9uEoQfRghLxoWvhK+TjMY5Dn/IgmpI UijPVKFJAkPOXXPSj6dK6G6rBoZ6WyZHz9N2SqSjuw7LX6hzfKtExJTtxt6Ve8Hh TE/sR7oqyTuzle3BBjTJ/6/d4itWgDb2EmaL3n/z9RUwlF3P8tpnzCxUKRQCASJV 7d2SiARrU8ddnBJBrSbNkq5DrHRb6lnUqFMVdty5U/Qqh6WAgNy3Eq+xOUzqt/vk ygemZ+jpfXfamKpsItq1kutfQSdedWGGRxbGVyUdOxqEGUvmRDs=
    =amgw
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ross Vandegrift@21:1/5 to Dirk Eddelbuettel on Fri Dec 31 08:10:02 2021
    Hi Dirk,

    On Mon, Dec 27, 2021 at 08:10:50AM -0600, Dirk Eddelbuettel wrote:
    My (very informal) packaging has always been in the open (but on GitHub). A possible first step might be to review the added files in debian/ and in a first pass edit out all references to 'informal' or 'unofficial' packaging and making them more official in the cloud team repo -- which I presume is on salsa? If you or others want to look, I have this currently at github.com in

    eddelbuettel/pkg-aws-c-common common C layer
    eddelbuettel/pkg-aws-checksums checksum for transport
    eddelbuettel/pkg-aws-c-event-stream another C layer
    eddelbuettel/pkg-aws-sdk-cpp-only-s3 C++ SDK subset for S3

    I did some work on aws-c-common & aws-c-event-stream from a PoC of packaging for awscli v2. Details are in #966573, message 43 [1].

    So far, I haven't uploaded any of this. The AWS SDK now requires their own crypto & tls implementations. The cloud team discussed this a while back, no one found that responsibility very attractive. Still, we'll want awscli v2 eventually.

    The last repo is the one that is 'incomplete' as I skipped everything not need by my use case of accessing s3 programmatically from other C++ code. >From a casual look at another C++ project I noticed that it had the same subset so this may make sense. It would allow us to proceed and start with the subset and add as needed (as opposed to be overwhelmed by 'all of it').

    I'd be worried about confusing users who expect to have the whole SDK available. But as long as the packages only ship static libraries, I don't see how it would cause actual issues.

    Ross

    [1] - https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=966573#43

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Dirk Eddelbuettel@21:1/5 to Wookey on Fri Dec 31 15:00:01 2021
    (Dual reply to two emails)

    On 30 December 2021 at 23:32, Wookey wrote:
    | On 2021-12-26 18:08 -0600, Dirk Eddelbuettel wrote:
    | >
    | > Does anybody know where we are with respect to the WNPPs / ITPs / ... on the
    | > Amazon SDK for C++?
    | >
    | > I have a package that could take advantage of this if it were packaged, and I
    | > am sure a number of other packages are in a similar situation given how
    | > pervasive AWS use is.
    |
    | I don't know anything about the state of this, but am in the same
    | place, in that I am packaging something that uses awsSDK pieces,
    | specifically tensorflow which wants aws-cpp-sdk-core.
    |
    | > FWIW I have packaged _subsets_ of the C++ SDK informally for my own use (also
    | > at Launchpad) but I don't think I have the time and energy to take this on as
    | > another package.
    |
    | Which bits have you done? They may or may not be sufficient to satisfy tensorflow.

    In Debian testing (via informal 'PPA for testing' on GitHub) and for focal
    via ppa:edd/misc

    edd@rob:~/git/ppa/docs(master)$ ls -1 libaws*deb # omitting double entries libaws-c-common-dev_0.4.59-1.debian.1_amd64.deb libaws-c-event-stream-dev_0.1.6-1.debian.1_amd64.deb libaws-checksums-dev_0.1.9-1.debian.2_amd64.deb libaws-sdk-cpp-only-s3-dev_1.8.117-3.debian.1_amd64.deb edd@rob:~/git/ppa/docs(master)$

    edd@rob:~$ dpkg -l | grep libaws | cut -c-80
    ii libaws-c-common-dev 0.4.59-1.2004.1
    ii libaws-c-event-stream-dev 0.1.6-1.2004.1
    ii libaws-checksums-dev 0.1.9-1.2004.1
    ii libaws-sdk-cpp-only-s3-dev 1.8.117-1.2004.1
    edd@rob:~$

    | I too, am not bursting to add large packages that I don't use (or even
    | properly know what they are) to my collection of responsibilities, but
    | I can spend work time on helping get this going so I don't mind
    | mucking in on some packaging work.

    Great!


    On 30 December 2021 at 22:17, Ross Vandegrift wrote:
    | On Mon, Dec 27, 2021 at 08:10:50AM -0600, Dirk Eddelbuettel wrote:
    | > My (very informal) packaging has always been in the open (but on GitHub). A | > possible first step might be to review the added files in debian/ and in a | > first pass edit out all references to 'informal' or 'unofficial' packaging | > and making them more official in the cloud team repo -- which I presume is on
    | > salsa? If you or others want to look, I have this currently at github.com in
    | >
    | > eddelbuettel/pkg-aws-c-common common C layer
    | > eddelbuettel/pkg-aws-checksums checksum for transport
    | > eddelbuettel/pkg-aws-c-event-stream another C layer
    | > eddelbuettel/pkg-aws-sdk-cpp-only-s3 C++ SDK subset for S3
    |
    | I did some work on aws-c-common & aws-c-event-stream from a PoC of packaging | for awscli v2. Details are in #966573, message 43 [1].
    |
    | So far, I haven't uploaded any of this. The AWS SDK now requires their own
    | crypto & tls implementations. The cloud team discussed this a while back, no | one found that responsibility very attractive. Still, we'll want awscli v2
    | eventually.

    I think in what I need the crypto etc comes from curl and openssl as usaul.

    | > The last repo is the one that is 'incomplete' as I skipped everything not
    | > need by my use case of accessing s3 programmatically from other C++ code.
    | > >From a casual look at another C++ project I noticed that it had the same
    | > subset so this may make sense. It would allow us to proceed and start with | > the subset and add as needed (as opposed to be overwhelmed by 'all of it'). |
    | I'd be worried about confusing users who expect to have the whole SDK
    | available. But as long as the packages only ship static libraries, I don't see
    | how it would cause actual issues.

    Right on with the name and confusion so I picked 'libaws-sdk-cpp-only-s3-dev' to signal the subset nature, at least for now.

    Dirk


    --
    https://dirk.eddelbuettel.com | @eddelbuettel | edd@debian.org

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Dirk Eddelbuettel@21:1/5 to Dirk Eddelbuettel on Sat Jan 8 17:40:01 2022
    On 31 December 2021 at 07:51, Dirk Eddelbuettel wrote:
    | On 30 December 2021 at 22:17, Ross Vandegrift wrote:
    | | On Mon, Dec 27, 2021 at 08:10:50AM -0600, Dirk Eddelbuettel wrote:
    | | > My (very informal) packaging has always been in the open (but on GitHub). A
    | | > possible first step might be to review the added files in debian/ and in a
    | | > first pass edit out all references to 'informal' or 'unofficial' packaging
    | | > and making them more official in the cloud team repo -- which I presume is on
    | | > salsa? If you or others want to look, I have this currently at github.com in
    | | >
    | | > eddelbuettel/pkg-aws-c-common common C layer
    | | > eddelbuettel/pkg-aws-checksums checksum for transport
    | | > eddelbuettel/pkg-aws-c-event-stream another C layer
    | | > eddelbuettel/pkg-aws-sdk-cpp-only-s3 C++ SDK subset for S3
    | |
    | | I did some work on aws-c-common & aws-c-event-stream from a PoC of packaging
    | | for awscli v2. Details are in #966573, message 43 [1].

    You have a large set there which covers three of the ones I'd need. Could
    you upload these:

    https://salsa.debian.org/rvandegrift/aws-c-common
    https://salsa.debian.org/rvandegrift/aws-checksums
    https://salsa.debian.org/rvandegrift/aws-c-event-stream

    to allow partial progress, and a 'base camp' from which we can continue?
    (You have quite the load there too...) Maybe we can then build the AWS SDK (or parts thereof) we need on top?

    Dirk

    --
    https://dirk.eddelbuettel.com | @eddelbuettel | edd@debian.org

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ross Vandegrift@21:1/5 to Dirk Eddelbuettel on Fri Jan 14 04:40:02 2022
    On Sat, Jan 08, 2022 at 10:31:03AM -0600, Dirk Eddelbuettel wrote:
    You have a large set there which covers three of the ones I'd need. Could you upload these:

    https://salsa.debian.org/rvandegrift/aws-c-common
    https://salsa.debian.org/rvandegrift/aws-checksums
    https://salsa.debian.org/rvandegrift/aws-c-event-stream

    to allow partial progress, and a 'base camp' from which we can continue?
    (You have quite the load there too...) Maybe we can then build the AWS SDK (or parts thereof) we need on top?

    The cloud team met this week and we discussed this. We're happy to do
    this, but there's one caveat: as far as we know, upstream won't provide
    long term supported releases for this stack. They work in the
    "everyone runs the newest" model. Probably not a big deal for
    aws-c-common or aws-checksums. But aws-c-event-stream (transitively)
    depends on s2n and aws-lc where security support is critical. So it's
    probably not feasible to ship in a stable release.

    Still, it'd be useful to provide packages so folks can start to
    experiment - hopefully things will eventually settle down.

    I'll start making sure these are ready to go, and I'll ping you when
    they're ready. Would you mind uploading or sponsoring the initial
    round? I'm nonuploading, so I can't create new binary packages.

    Thanks,
    Ross

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)