• [gentoo-dev] [PATCH] cargo.eclass: use CARGO_CRATE_URIS if already avai

    From Florian Schmaus@21:1/5 to All on Mon Jul 31 12:50:02 2023
    With 59dbfb80f748 ("cargo.eclass: Add variable alternative to $(cargo_crate_uris)") the _cargo_set_crate_uris function was
    introduced. This function runs when the elcass is inherited and sets CARGO_CRATE_URIS.

    Ebuilds that use $(cargo_crate_uris) in SRC_URI will again invoke this function, even though CARGO_CRATE_URIS is already set. Avoiding this unnecessary computation reduces the ebuild source time of app-shells/nushell-0.83.0 from 21 ms to 14 ms.

    This is a significant reduction when compared to the variable-setting alternative that the commit 59dbfb80f748 ("cargo.eclass: Add variable alternative to $(cargo_crate_uris)") introduced. Using the
    variable-setting approach would reduce the ebuild source time only by a
    little bit more than one millisecond.

    Using

    pk pkg source --bench 10s '=app-shells/nushell-0.83.0'

    as benchmark, yields

    | | Cached CARGO_CRATE_URIS | Non-Cached CARGO_CRATE_URIS |
    |---------------------------+-------------------------+-----------------------------|
    | $(cargo_crate_uris) | mean: 14.189ms | mean: 21.445ms |
    | variable-setting approach | mean: 12.822ms | mean: 12.852ms |

    full benchmark output

    | | Cached CARGO_CRATE_URIS (this commit) | Non-Cached CARGO_CRATE_URIS |
    |---------------------------+------------------------------------------------------------------+------------------------------------------------------------------|
    | $(cargo_crate_uris) | mean: 14.189ms, min: 13.646ms, max: 15.103ms, σ = 149µs, N = 705 | mean: 21.445ms, min: 20.79ms, max: 22.832ms, σ = 228µs, N = 467 |
    | variable-setting approach | mean: 12.822ms, min: 12.41ms, max: 13.909ms, σ = 165µs, N = 780 | mean: 12.852ms, min: 12.367ms, max: 15.437ms, σ = 227µs, N = 779 |

    Signed-off-by: Florian Schmaus <flow@gentoo.org>
    ---
    eclass/cargo.eclass | 8 ++++++++
    1 file changed, 8 insertions(+)

    diff --git a/eclass/cargo.eclass b/eclass/cargo.eclass
    index 70b6008d9cd8..5d6911801097 100644
    --- a/eclass/cargo.eclass
    +++ b/eclass/cargo.eclass
    @@ -240,6 +240,14 @@ _cargo_set_crate_uris "${CRATES}"
    # Constructs a list of crates from its arguments.
    # If no arguments are provided, it uses the CRATES variable.
    cargo_crate_uris() {
    + # Use already existing value for CARGO_CRATE_URIS, computed by
    + # _cargo_set_crate_uris, when this function is invoked without
    + # arguments.
    + if [[ $# -eq 0 && -n "${CARGO_CRATE_URIS}" ]]; then
    + echo "${CARGO_CRATE_URIS}"
    + return
    + fi
    +
    local crates=${*-${CRATES}}
    if [[ -z ${crates} ]]; then
    eerror "CRATES variable is not defined and nothing passed as argument"
    --
    2.41.0

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?Micha=C5=82_G=C3=B3rny?=@21:1/5 to Florian Schmaus on Mon Jul 31 13:50:01 2023
    On Mon, 2023-07-31 at 12:41 +0200, Florian Schmaus wrote:
    With 59dbfb80f748 ("cargo.eclass: Add variable alternative to $(cargo_crate_uris)") the _cargo_set_crate_uris function was
    introduced. This function runs when the elcass is inherited and sets CARGO_CRATE_URIS.

    Ebuilds that use $(cargo_crate_uris) in SRC_URI will again invoke this function, even though CARGO_CRATE_URIS is already set. Avoiding this unnecessary computation reduces the ebuild source time of app-shells/nushell-0.83.0 from 21 ms to 14 ms.

    This is a significant reduction when compared to the variable-setting alternative that the commit 59dbfb80f748 ("cargo.eclass: Add variable alternative to $(cargo_crate_uris)") introduced. Using the
    variable-setting approach would reduce the ebuild source time only by a little bit more than one millisecond.

    Using

    pk pkg source --bench 10s '=app-shells/nushell-0.83.0'

    as benchmark, yields

    | Cached CARGO_CRATE_URIS | Non-Cached CARGO_CRATE_URIS |
    ---------------------------+-------------------------+-----------------------------|
    $(cargo_crate_uris) | mean: 14.189ms | mean: 21.445ms |
    variable-setting approach | mean: 12.822ms | mean: 12.852ms |

    full benchmark output

    | Cached CARGO_CRATE_URIS (this commit) | Non-Cached CARGO_CRATE_URIS |
    ---------------------------+------------------------------------------------------------------+------------------------------------------------------------------|
    $(cargo_crate_uris) | mean: 14.189ms, min: 13.646ms, max: 15.103ms, σ = 149µs, N = 705 | mean: 21.445ms, min: 20.79ms, max: 22.832ms, σ = 228µs, N = 467 |
    variable-setting approach | mean: 12.822ms, min: 12.41ms, max: 13.909ms, σ = 165µs, N = 780 | mean: 12.852ms, min: 12.367ms, max: 15.437ms, σ = 227µs, N = 779 |

    Signed-off-by: Florian Schmaus <flow@gentoo.org>
    ---
    eclass/cargo.eclass | 8 ++++++++
    1 file changed, 8 insertions(+)

    diff --git a/eclass/cargo.eclass b/eclass/cargo.eclass
    index 70b6008d9cd8..5d6911801097 100644
    --- a/eclass/cargo.eclass
    +++ b/eclass/cargo.eclass
    @@ -240,6 +240,14 @@ _cargo_set_crate_uris "${CRATES}"
    # Constructs a list of crates from its arguments.
    # If no arguments are provided, it uses the CRATES variable.
    cargo_crate_uris() {
    + # Use already existing value for CARGO_CRATE_URIS, computed by
    + # _cargo_set_crate_uris, when this function is invoked without
    + # arguments.
    + if [[ $# -eq 0 && -n "${CARGO_CRATE_URIS}" ]]; then
    + echo "${CARGO_CRATE_URIS}"
    + return
    + fi
    +
    local crates=${*-${CRATES}}
    if [[ -z ${crates} ]]; then
    eerror "CRATES variable is not defined and nothing passed as argument"

    This incorrectly assumes that the value of CRATES did not change which
    isn't guaranteed anywhere.

    --
    Best regards,
    Michał Górny

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Sam James@21:1/5 to Florian Schmaus on Mon Jul 31 13:50:01 2023
    Florian Schmaus <flow@gentoo.org> writes:

    With 59dbfb80f748 ("cargo.eclass: Add variable alternative to $(cargo_crate_uris)") the _cargo_set_crate_uris function was
    introduced. This function runs when the elcass is inherited and sets CARGO_CRATE_URIS.

    (I would've CC'd mgorny given presumably you want his input as the
    author.)

    Ebuilds that use $(cargo_crate_uris) in SRC_URI will again invoke this function, even though CARGO_CRATE_URIS is already set. Avoiding this unnecessary computation reduces the ebuild source time of app-shells/nushell-0.83.0 from 21 ms to 14 ms.

    This is a significant reduction when compared to the variable-setting alternative that the commit 59dbfb80f748 ("cargo.eclass: Add variable alternative to $(cargo_crate_uris)") introduced. Using the
    variable-setting approach would reduce the ebuild source time only by a little bit more than one millisecond.

    Using

    pk pkg source --bench 10s '=app-shells/nushell-0.83.0'

    as benchmark, yields

    | | Cached CARGO_CRATE_URIS | Non-Cached CARGO_CRATE_URIS |
    |---------------------------+-------------------------+-----------------------------|
    | $(cargo_crate_uris) | mean: 14.189ms | mean: 21.445ms |
    | variable-setting approach | mean: 12.822ms | mean: 12.852ms |

    full benchmark output

    | | Cached CARGO_CRATE_URIS (this commit) | Non-Cached CARGO_CRATE_URIS |
    |---------------------------+------------------------------------------------------------------+------------------------------------------------------------------|
    | $(cargo_crate_uris) | mean: 14.189ms, min: 13.646ms, max: 15.103ms, σ = 149µs, N = 705 | mean: 21.445ms, min: 20.79ms, max: 22.832ms, σ = 228µs, N = 467 |
    | variable-setting approach | mean: 12.822ms, min: 12.41ms, max: 13.909ms, σ = 165µs, N = 780 | mean: 12.852ms, min: 12.367ms, max: 15.437ms, σ = 227µs, N = 779 |


    Anyway, nice work, this seems reasonable, and it's consistent with the
    kind of thing I've done for Ruby. But let's see what mgorny says as
    well.

    Signed-off-by: Florian Schmaus <flow@gentoo.org>
    ---
    eclass/cargo.eclass | 8 ++++++++
    1 file changed, 8 insertions(+)

    diff --git a/eclass/cargo.eclass b/eclass/cargo.eclass
    index 70b6008d9cd8..5d6911801097 100644
    --- a/eclass/cargo.eclass
    +++ b/eclass/cargo.eclass
    @@ -240,6 +240,14 @@ _cargo_set_crate_uris "${CRATES}"
    # Constructs a list of crates from its arguments.
    # If no arguments are provided, it uses the CRATES variable.
    cargo_crate_uris() {
    + # Use already existing value for CARGO_CRATE_URIS, computed by
    + # _cargo_set_crate_uris, when this function is invoked without
    + # arguments.
    + if [[ $# -eq 0 && -n "${CARGO_CRATE_URIS}" ]]; then
    + echo "${CARGO_CRATE_URIS}"
    + return
    + fi
    +
    local crates=${*-${CRATES}}
    if [[ -z ${crates} ]]; then
    eerror "CRATES variable is not defined and nothing passed as argument"

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ionen Wolkens@21:1/5 to Florian Schmaus on Mon Jul 31 14:40:01 2023
    On Mon, Jul 31, 2023 at 12:41:19PM +0200, Florian Schmaus wrote:
    With 59dbfb80f748 ("cargo.eclass: Add variable alternative to $(cargo_crate_uris)") the _cargo_set_crate_uris function was
    introduced. This function runs when the elcass is inherited and sets CARGO_CRATE_URIS.

    Ebuilds that use $(cargo_crate_uris) in SRC_URI will again invoke this function, even though CARGO_CRATE_URIS is already set. Avoiding this unnecessary computation reduces the ebuild source time of app-shells/nushell-0.83.0 from 21 ms to 14 ms.

    Note that pkgcheck (just) gained a check for this, ebuilds should
    get fixed in time which would makes this change only an interim
    thing and imo not all that worth it.

    Aka:

    app-shells/nushell$ pkgcheck scan
    app-shells/nushell
    SuboptimalCratesSeparator: version 0.82.0: line: 9: using - as name-version separator in CRATES is suboptimal, use name@version instead
    SuboptimalCratesURICall: version 0.82.0: line: 601: calling '$(cargo_crate_uris)' is suboptimal, use '${CARGO_CRATE_URIS}' for global CRATES instead
    SuboptimalCratesURICall: version 0.83.0: line: 574: calling '$(cargo_crate_uris)' is suboptimal, use '${CARGO_CRATE_URIS}' for global CRATES instead

    --
    ionen

    -----BEGIN PGP SIGNATURE-----

    iQEzBAABCAAdFiEEx3SLh1HBoPy/yLVYskQGsLCsQzQFAmTHqVoACgkQskQGsLCs QzTKbQf/ZDms26lLi2JIoUjIHESMoTUu3mIXDvqW5IWYLVGEWarDy2dZUmdJ3yY7 MtVGQwCqGcdofMevzHbq96cqIW9ac1xrDw8Hu9M6s36b1N85uUMYRV1yb1k8YlGU tawtkBhhoS6JcnNdfWSxGG9vENrzxKKvU9HYifZwnhpLJk20syF2kA1HcpZxSK6+ b+YFRdIoXxjr2+/5yT2Rtzge7pQMDhXHPNIL7WWlGhyBEoPrWr3hL+24dYAZvzW+ satUvr+VPMEmxM0LJ3bOz7rp14UkN1dI9Kp1WfEDFPwatoGQ+lRsp6eUGD24rNCm TpGmjb4/OpMLp0ppzGA9x0380gfUZQ==
    =Puxv
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Sam James@21:1/5 to mgorny@gentoo.org on Mon Jul 31 14:50:01 2023
    Michał Górny <mgorny@gentoo.org> writes:

    On Mon, 2023-07-31 at 12:41 +0200, Florian Schmaus wrote:
    With 59dbfb80f748 ("cargo.eclass: Add variable alternative to
    $(cargo_crate_uris)") the _cargo_set_crate_uris function was
    introduced. This function runs when the elcass is inherited and sets
    CARGO_CRATE_URIS.

    Ebuilds that use $(cargo_crate_uris) in SRC_URI will again invoke this
    function, even though CARGO_CRATE_URIS is already set. Avoiding this
    unnecessary computation reduces the ebuild source time of
    app-shells/nushell-0.83.0 from 21 ms to 14 ms.

    This is a significant reduction when compared to the variable-setting
    alternative that the commit 59dbfb80f748 ("cargo.eclass: Add variable
    alternative to $(cargo_crate_uris)") introduced. Using the
    variable-setting approach would reduce the ebuild source time only by a
    little bit more than one millisecond.

    Using

    pk pkg source --bench 10s '=app-shells/nushell-0.83.0'

    as benchmark, yields

    | Cached CARGO_CRATE_URIS | Non-Cached CARGO_CRATE_URIS |
    ---------------------------+-------------------------+-----------------------------|
    $(cargo_crate_uris) | mean: 14.189ms | mean: 21.445ms |
    variable-setting approach | mean: 12.822ms | mean: 12.852ms |

    full benchmark output

    | Cached CARGO_CRATE_URIS (this commit) | Non-Cached CARGO_CRATE_URIS |
    ---------------------------+------------------------------------------------------------------+------------------------------------------------------------------|
    $(cargo_crate_uris) | mean: 14.189ms, min: 13.646ms, max: 15.103ms, σ = 149µs, N = 705 | mean: 21.445ms, min: 20.79ms, max: 22.832ms, σ = 228µs, N = 467 |
    variable-setting approach | mean: 12.822ms, min: 12.41ms, max: 13.909ms, σ = 165µs, N = 780 | mean: 12.852ms, min: 12.367ms, max: 15.437ms, σ = 227µs, N = 779 |

    Signed-off-by: Florian Schmaus <flow@gentoo.org>
    ---
    eclass/cargo.eclass | 8 ++++++++
    1 file changed, 8 insertions(+)

    diff --git a/eclass/cargo.eclass b/eclass/cargo.eclass
    index 70b6008d9cd8..5d6911801097 100644
    --- a/eclass/cargo.eclass
    +++ b/eclass/cargo.eclass
    @@ -240,6 +240,14 @@ _cargo_set_crate_uris "${CRATES}"
    # Constructs a list of crates from its arguments.
    # If no arguments are provided, it uses the CRATES variable.
    cargo_crate_uris() {
    + # Use already existing value for CARGO_CRATE_URIS, computed by
    + # _cargo_set_crate_uris, when this function is invoked without
    + # arguments.
    + if [[ $# -eq 0 && -n "${CARGO_CRATE_URIS}" ]]; then
    + echo "${CARGO_CRATE_URIS}"
    + return
    + fi
    +
    local crates=${*-${CRATES}}
    if [[ -z ${crates} ]]; then
    eerror "CRATES variable is not defined and nothing passed as argument"

    This incorrectly assumes that the value of CRATES did not change which
    isn't guaranteed anywhere.

    Ah, thanks! Now I remember - we discussed it before and rejected it for
    this reason.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)