• Re: Not running tests because tests miss source code is not useful

    From Marvin Renich@21:1/5 to All on Thu Oct 7 15:50:02 2021
    * Pirate Praveen <praveen@onenetbeyond.org> [211007 05:41]:
    On 7 October 2021 3:02:55 am IST, Thomas Goirand <zigo@debian.org> wrote:
    On 10/6/21 6:53 PM, Pirate Praveen wrote:
    On ബു, ഒക്ടോ 6 2021 at 12:16:07 വൈകു +0200 +0200, Jonas Smedegaard > <jonas@jones.dk> wrote:
    Quoting Yadd (2021-10-06 11:43:40)
     On Lu, 04 oct 21, 16:40:48, Bastien Roucari�s wrote:
    Source: src:node-lodash
    Version: 4.17.21+dfsg+~cs8.31.173-1
    Severity: serious
    Justification: do not compile from source
    ;
    Dear Maintainer,
    ;
    The vendor directory should be emptied
    ;
    The debug version is compiled without source (lintian warn)
    and moreover the rest of file are already packaged
    ;
    grep -R vendor * gives only a few hit that could be cured by
    symlinking
    ;
    Bastien
     Hi,

     this files are used for test only, maybe severity could be decreased. >>>
    I find the severity accurate: Relying on non-source code is a severe
    violation of Debian Policy, not matter the purpose of relying on it.

    I think we should change the policy here. Running tests helps improve
    the quality of the software we ship. Many times the vendored code is
    used to ensure the code does not break in a specific situation. I don't
    think reducing test coverage in such situations is really helpful.

    Right, running tests helps improve the quality of software we ship.
    Which is why you probably need to test using what's shipped in Debian >rather than using a vendored source-less code.

    We are not shipping the source less code. This is used only during
    tests. I don't think we are not gaining anything by removing tests
    here. Just making it harder for the package maintainer to run tests.

    If the original report is correct (I haven't looked at the package),
    then Debian is, indeed, shipping the source-less code. Debian ships
    both source packages and binary packages. The source-less code is in
    the source package, and is thus shipped by Debian. The DFSG applies to
    both source and binary packages.

    If we rely on non-free code for tests, that's really bad too, and that
    must be avoided just like we're avoiding source-less code everywhere
    else in Debian. The policy shall not change, please.

    +1

    The code is not non-free here, just a specific version of a Free
    Software code built outside Debian.

    I think you are confusing "Free Software" with "DFSG Free Software".
    Debian has more strict standards than some other parts of the Free
    Software community.

    I think tools required for tests should be considered separately from
    tools required to compile. I think it should be treated similar to
    test data.

    Actually, I agree with this, but not exactly in the way you intended. I
    think there should be a way for a source package to specifically
    identify a Build-Dep as a test (probably with a separate field, maybe Test-Depends), and to identify tests that must run for the build to be considered successful, and tests that are optional (perhaps
    Test-Suggests). This would allow Non-Free tests (and test data) to be
    packaged separately and included in non-free where they belong, with the requirement that a source package can Test-Suggests a non-free package,
    but it cannot Test-Depends such a package. This would also allow tests
    that require external networking resources, or tests that consumed an inordinate amount of CPU, or..., to be Test-Suggested, without
    interfering with a normal binary build.

    The buildds could be enhanced to build the binary packages without any
    of the test dependencies present, then install the desired set of test dependencies and run them. This would ensure that the binary packages
    were built without the test packages even present (a huge benefit, in my opinion), but that tests were still performed.

    I'm not sure how many existing tests require the built programs to still
    have the build environment present (i.e. vestiges of the build process
    are used by the test), but it would be nice if the binary package could
    be sent to another machine where the binary package (without the source package) was installed with the desired test packages, so the tests
    could be run on the properly installed binary package.

    This would, of course, require a moderate amount of implementation in
    the buildds and core tools such as dpkg, and I am not in a position to
    help with this, so the people who can do so would have to like my idea
    enough to spend their own time to implement it.

    I think blindly applying a rule without thinking of any consequences is bad too.

    I don't think anyone is blindly applying the DFSG to the source package
    here. It is important to separate "DFSG Free" (main) from "Free but not
    DFSG Free" (non-free) in all aspects of the source package, including
    tests. The problem is that there is currently no mechanism available to
    have the build process identify and select either free or non-free
    tests, so only DFSG Free tests are allowed.

    I think a nocheck build profile which excludes these files from build
    is sufficient...

    Only if the non-free parts of the tests are moved from the free source
    package to a separate non-free source package, which still requires
    something like my suggestion above to allow the non-free tests to be run
    on the free binaries.

    The current policy is not making Debian better.

    I disagree.

    ...Marvin

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to Richard Laager on Thu Oct 7 18:40:02 2021
    Richard Laager <rlaager@debian.org> writes:

    I haven't looked into the specifics of this situation, but in general,
    tests should be run against the same versions of dependencies that the
    actual code will use, for what should be obvious reasons. If Debian has
    the dependencies with different API versions, then it's all the more important that the tests run against the Debian versions of the
    dependencies.

    I believe the dependencies in question are test dependencies. In other
    words, they're dependencies required to drive the test suite machinery,
    not dependencies of the code under test.

    --
    Russ Allbery (rra@debian.org) <https://www.eyrie.org/~eagle/>

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Richard Laager@21:1/5 to Pirate Praveen on Thu Oct 7 18:20:01 2021
    This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --ExFfbT9WJr2NpJ61EHqQYhJt9BRzTHCyx
    Content-Type: text/plain; charset=utf-8; format=flowed
    Content-Language: en-US
    Content-Transfer-Encoding: quoted-printable

    On 10/7/21 4:40 AM, Pirate Praveen wrote:
    What you are proposing would require the package maintainer to adapt these tests to versions available (many times with different API versions) in Debian and the easier choice is disabling tests.

    I haven't looked into the specifics of this situation, but in general,
    tests should be run against the same versions of dependencies that the
    actual code will use, for what should be obvious reasons. If Debian has
    the dependencies with different API versions, then it's all the more
    important that the tests run against the Debian versions of the
    dependencies.

    Running tests against vendored dependencies one isn't going to use at
    run-time is of limited usefulness. Presumably upstream is already
    running those tests against the vendored dependencies, so the odds of
    Debian finding breakage in the _source_ at packaging time is low.

    --
    Richard


    --ExFfbT9WJr2NpJ61EHqQYhJt9BRzTHCyx--

    -----BEGIN PGP SIGNATURE-----

    iQIzBAEBCgAdFiEE1Ot9lOeOTujs4H+U+HlhmcBFhs4FAmFfHYMACgkQ+HlhmcBF hs6pbhAAiraAYgVHEkq6LcLP2R+FEJ5QXxJhqZeiLXdfLsppT0zeXfT4hak133e6 GovPO850nkAyg6hSfhErLlwgahynyF1rlAxEJ/Mxu432AVEpyxaVdLK/G5bOBGqi C3t7rC8l3sD3uMfH1wzlrr30Y3swomA4JjBugz5GKdQB8cvU4V3rwFOP9sCTp0iv NWZHXv7c5DbclP3pSLOIbXnjS7Ql66oyy8lnZ9CxnjkrccbFFC8ixy/BDuchq3zV DfEC28+J/oSCfvaz5W3VxVVOj3pU8thiPJyl8eDVUysY/YLtrPSztkfvTi0P0bDo kifkRSr2ZqhDbzgxNx9DDg83ynRyAJ/wY+t6BF6Y4o+zhA177r6FqVIea92xsv1y HGX5rJe742KPvtLiuegbc6tKV49ZBnrtFa8/dLvfIzozPF57X5Axe7TS+R7Jr6bQ 57bB8gOPrzuoGCP1RsKZndddgLffixg47qoS04Fv0EX32u/hxmUY7Qit0zd69PBq QmnQ16bC1f3VGEEUKgsryVEK9ggYicOsoZDrMYaaUz9IVzjYkDZrTsxd34osqvvW 5QSAAn8swngV9ine4l89VLfeBo9FYlFRKWbAnJFVRO8ngWG0hmnlIILbr/3fVoqi uwhQSi98H5JfHP146e2DzYfkSb0/a+N8YTeUJdOQTqXC8RD9Uxc=
    =tL0Q
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Richard Laager@21:1/5 to Russ Allbery on Thu Oct 7 20:00:01 2021
    This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --8MSPOKQsrduahs27FgylmYSKtjZkPpEgu
    Content-Type: text/plain; charset=utf-8; format=flowed
    Content-Language: en-US
    Content-Transfer-Encoding: quoted-printable

    On 10/7/21 11:35 AM, Russ Allbery wrote:
    Richard Laager <rlaager@debian.org> writes:

    I haven't looked into the specifics of this situation, but in general,
    tests should be run against the same versions of dependencies that the
    actual code will use, for what should be obvious reasons. If Debian has
    the dependencies with different API versions, then it's all the more
    important that the tests run against the Debian versions of the
    dependencies.

    I believe the dependencies in question are test dependencies. In other words, they're dependencies required to drive the test suite machinery,
    not dependencies of the code under test.

    Thanks for the clarification of the specifics!

    In that case, I personally don't see a big problem with them being
    vendored as opposed to using system copies.

    But AFAIK, they do still need to be DFSG-free to be in main if they are
    in the source tarball. And I personally would not consider minified JS
    (if that is indeed at issue here) to be "source code".

    --
    Richard


    --8MSPOKQsrduahs27FgylmYSKtjZkPpEgu--

    -----BEGIN PGP SIGNATURE-----

    iQIzBAEBCgAdFiEE1Ot9lOeOTujs4H+U+HlhmcBFhs4FAmFfM0QACgkQ+HlhmcBF hs4pbBAAoA3aOPrBavy4g98D2WQiyEp7XGxLaTJzIDhqVqCImAL36mdKMeeYtTmv wHwOQbpTV3c7ZVhiopJCLBtyZCU+D5oVfVRk4gXnqFi35hAohVBlhlsGNA62NdLY p7B8Gi1zLzXUgNY8FWhKNSWvuxM6BI+FG3UVh1Uatue9YpnhXnLfKUqF4wO9Dz8w MQz8qNLYx4CjhG4SjZpNUdUYhVxVr7rXj9osEuURFDeunPqom4dTY8VatMpKH7Iq dWvZZBonN97NvnJNZ3wJKrSe7ZLOTxMRigvQhV26cdnX6b+670EsgGXUXxUvKWdq BeCkhdxGpaRWE/PknDgNYDE3TwkSslRGXeBejQ3ax5/VHPUaeC0FquB5Q1JYp8jw NGUXcXVf5RwKThv0nJjqnpioJ9NoJa4e6ByiHmpVbQde7txYuJPXZfKFE2jfP7Zn meyQ2za7fVEdHg18FU2z3IOOqxth5P6yVqRA/7Vhl8G1xHmt5f3e/9YGBwXD0EcV hVQt5lZwj149efK8rgrvktiV0SXxfupLFC6JpKczYBlICz6UmYKSn6vF+cZwPPCI WyOoG4STkRODPcg/Q6q/YmKrZXCEFN27lrNG13WmjdsON/XPkO+Gz803hUA52UE3 wGKnrwUOhR6vL9HJqWAPUOKrmclQgdRJDZyhMF1GCVp+PNECruY=
    =KnPp
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to Jonas Smedegaard on Thu Oct 7 20:50:02 2021
    Jonas Smedegaard <jonas@jones.dk> writes:

    Right: It is ok to use upstream-provided pre-minified code, as long as
    that code is DFSG-free, which requires the source of that code must
    exist in Debian.

    ...and because that is often complicated to ensure (not because it
    violates DFSG in itself), it is easier to avoid upstream-provided pre-minified code.

    Test suites are often a licensing mess. Another common case that's not in
    play here but that I've seen before is that long-standing projects that
    have been used commercially often have test snippets with unclear
    licensing that check for regressions that were previously seen in
    proprietary environments.

    Debian historically has erred on the side of maintaining clear source availability and licensing status for everything in Debian (which includes everything in any source package) at the cost of not availing ourselves of
    test suites that would otherwise be useful. That's unfortunately probably
    the easy path here as well, until someone has time to find non-minified versions of the test dependencies and either package them or include them
    in the package. It's frustrating to remove the tests, but the DFSG source requirements as currently applied do not distinguish between code shipped
    only in source packages and code also shipped in binary packages.

    I can see an argument that we should not worry about minified files in
    main that are (a) only in the source package and not in any binary
    package, and (b) only used to run tests, not to build the binary packages.
    (I'm not saying I agree or disagree, just that I can see the argument.)
    But given the apparent consensus on this in past discussions, I suspect
    that changing that rule would be GR material rather than debian-devel
    thread material. Making that sort of change without a GR to be sure the project is behind it feels like the kind of thing that's likely to spawn endless arguments that will sap everyone's will to live.

    --
    Russ Allbery (rra@debian.org) <https://www.eyrie.org/~eagle/>

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jonas Smedegaard@21:1/5 to All on Thu Oct 7 20:40:01 2021
    Quoting Richard Laager (2021-10-07 19:49:56)
    On 10/7/21 11:35 AM, Russ Allbery wrote:
    Richard Laager <rlaager@debian.org> writes:

    I haven't looked into the specifics of this situation, but in
    general, tests should be run against the same versions of
    dependencies that the actual code will use, for what should be
    obvious reasons. If Debian has the dependencies with different API
    versions, then it's all the more important that the tests run
    against the Debian versions of the dependencies.

    I believe the dependencies in question are test dependencies. In
    other words, they're dependencies required to drive the test suite machinery, not dependencies of the code under test.

    Thanks for the clarification of the specifics!

    In that case, I personally don't see a big problem with them being
    vendored as opposed to using system copies.

    But AFAIK, they do still need to be DFSG-free to be in main if they
    are in the source tarball. And I personally would not consider
    minified JS (if that is indeed at issue here) to be "source code".

    Right: It is ok to use upstream-provided pre-minified code, as long as
    that code is DFSG-free, which requires the source of that code must
    exist in Debian.

    ...and because that is often complicated to ensure (not because it
    violates DFSG in itself), it is easier to avoid upstream-provided
    pre-minified code.


    - Jonas

    --
    * Jonas Smedegaard - idealist & Internet-arkitekt
    * Tlf.: +45 40843136 Website: http://dr.jones.dk/

    [x] quote me freely [ ] ask before reusing [ ] keep private --==============u62131492520692731=MIME-Version: 1.0
    Content-Transfer-Encoding: 7bit
    Content-Description: signature
    Content-Type: application/pgp-signature; name="signature.asc"; charset="us-ascii"

    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCgAdFiEEn+Ppw2aRpp/1PMaELHwxRsGgASEFAmFfPLMACgkQLHwxRsGg ASFA2Q/+IVvZAGnF2N8ieM+hlETg+IWJU1QrXr2su5RHIABHalylcbG8dNYbtv54 nL9s9bKOGMogZ+ujp8Rxl61xo4wp4a02bB2C5o9fwkluNkaxDcV3D7RcrRco2XJk 1HnCZOuYx1DT+Lyy8fJvhiOhLy1Od8Y2NeufBkIo3mmEMBOf7PfM66tG8VUiTP2Y YNUwFCvE6h8Yoiw/5x3LlkMySbN3CIStgk9oPW1W86VWl0OCyH66J6bLbV+wHxik 382T3QgPFVNa/I4jkeVf+8IWGsmtfsxDvGL3n2Uq0wlBO63eYenzcl6v9J4FawHg I+yDhoCcfTMD/HfazjPF8tk2HF+/WCg8czCvnauZb9iKhmiVKuV7WB4g3znxYDVZ 3CKJb8nXg0gF50WZtHSwibwYCwBp+MO3g5FaYlOILyhHcXylB1Id+01ATbDkFlXe D+vcY2SXyPRVAwHEsYvxFB3r8Co/X8l+dUxQrCtzFrstrWgIbqNlS0Blr05GC9Qp Isyz1Ie65CtPxfDXb
  • From Pirate Praveen@21:1/5 to Russ Allbery on Fri Oct 8 19:10:02 2021
    On വ്യാ, ഒക്ടോ 7 2021 at 11:43:52 രാവിലെ
    -0700 -0700, Russ Allbery <rra@debian.org> wrote:
    Jonas Smedegaard <jonas@jones.dk> writes:

    Right: It is ok to use upstream-provided pre-minified code, as long
    as
    that code is DFSG-free, which requires the source of that code must
    exist in Debian.

    ...and because that is often complicated to ensure (not because it
    violates DFSG in itself), it is easier to avoid upstream-provided
    pre-minified code.

    Test suites are often a licensing mess. Another common case that's
    not in
    play here but that I've seen before is that long-standing projects
    that
    have been used commercially often have test snippets with unclear
    licensing that check for regressions that were previously seen in
    proprietary environments.

    Debian historically has erred on the side of maintaining clear source availability and licensing status for everything in Debian (which
    includes
    everything in any source package) at the cost of not availing
    ourselves of
    test suites that would otherwise be useful. That's unfortunately
    probably
    the easy path here as well, until someone has time to find
    non-minified
    versions of the test dependencies and either package them or include
    them
    in the package. It's frustrating to remove the tests, but the DFSG
    source
    requirements as currently applied do not distinguish between code
    shipped
    only in source packages and code also shipped in binary packages.

    I can see an argument that we should not worry about minified files in
    main that are (a) only in the source package and not in any binary
    package, and (b) only used to run tests, not to build the binary
    packages.
    (I'm not saying I agree or disagree, just that I can see the
    argument.)
    But given the apparent consensus on this in past discussions, I
    suspect
    that changing that rule would be GR material rather than debian-devel
    thread material. Making that sort of change without a GR to be sure
    the
    project is behind it feels like the kind of thing that's likely to
    spawn
    endless arguments that will sap everyone's will to live.

    Thanks for this summary. I wanted to first see what everyone thinks
    here first. I will propose a GR.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Pirate Praveen@21:1/5 to All on Fri Oct 8 20:00:01 2021
    Draft text for the GR (this is my first time proposing a GR text, so
    help is welcome to make the text clearer).

    We should not worry about minified files in main that are (a) only in
    the source package and not in any binary package, and (b) only used to
    run tests, not to build the binary packages.

    To ensure these are not used during generation of binary packages, a
    nocheck build profile should be provided which will remove these files
    from the build environment.

    Rationale: Most of the time these minified files create a specific
    instance of a test case and these tests are not the only way to ensure functionality of the code we ship. Even though upstream runs these
    tests, during transitions we deviate from upstream for runtime
    dependencies and a way to ensure the functionality is not broken by a dependency change is essential. Insisting these test cases are built
    using only packages in main just makes running tests harder and
    disabing tests is choosen as an easier option. Treating the code used
    to run only tests is the same as generated code we ship is not useful.
    In case of the code we ship missing sources, our users can't modify the software. But if these sourceless code is used during tests, only
    ability to modify the test cases is lost. A test case is an arbitrary combination of the code and we have many different ways of testing a
    code.

    Option 1: Agree, we should allow these files in main
    Option 2: Disagree, we should not allow these files in main
    Option 3: Further discussion

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jonas Smedegaard@21:1/5 to All on Fri Oct 8 21:10:01 2021
    Quoting Pirate Praveen (2021-10-08 19:51:44)
    Draft text for the GR (this is my first time proposing a GR text, so
    help is welcome to make the text clearer).

    We should not worry about minified files in main that are (a) only in
    the source package and not in any binary package, and (b) only used to
    run tests, not to build the binary packages.

    To ensure these are not used during generation of binary packages, a
    nocheck build profile should be provided which will remove these files
    from the build environment.

    Rationale: Most of the time these minified files create a specific
    instance of a test case and these tests are not the only way to ensure functionality of the code we ship. Even though upstream runs these
    tests, during transitions we deviate from upstream for runtime
    dependencies and a way to ensure the functionality is not broken by a dependency change is essential. Insisting these test cases are built
    using only packages in main just makes running tests harder and
    disabing tests is choosen as an easier option. Treating the code used
    to run only tests is the same as generated code we ship is not useful.
    In case of the code we ship missing sources, our users can't modify the software. But if these sourceless code is used during tests, only
    ability to modify the test cases is lost. A test case is an arbitrary combination of the code and we have many different ways of testing a
    code.

    Option 1: Agree, we should allow these files in main
    Option 2: Disagree, we should not allow these files in main
    Option 3: Further discussion

    I think above text fails to adequately narrow the scope of concern.

    E.g. it could be interpreted to mean ignore licensing altogether, and it
    could cover any files used for tests that is obfuscated somehow -
    neither of which is your intent, I suspect.

    I guess what you mean by "allow" is to ignore [DFSG #2].

    I guess what you mean by "these files" is (in addition to the
    limitations described about them being used only for tests) limited to
    only JavaScript code which compared to its source code only has had unnecessary characters removed without changing its functionality (based
    on Wikipedia article on Minification).

    I think composing the ballot is better discussed at the mailinglist debian-vote@lists.debian.org


    - Jonas


    [DFSG #2]: "The program must include source code, and must allow
    distribution in source code as well as compiled form." at https://www.debian.org/social_contract#guidelines

    --
    * Jonas Smedegaard - idealist & Internet-arkitekt
    * Tlf.: +45 40843136 Website: http://dr.jones.dk/

    [x] quote me freely [ ] ask before reusing [ ] keep private --==============F41464866340627536=MIME-Version: 1.0
    Content-Transfer-Encoding: 7bit
    Content-Description: signature
    Content-Type: application/pgp-signature; name="signature.asc"; charset="us-ascii"

    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCgAdFiEEn+Ppw2aRpp/1PMaELHwxRsGgASEFAmFglmcACgkQLHwxRsGg ASFQrQ//YZjX5YnPjcTg7aLFU/m83vcjOjv3EThK6EYjiF0JV82g5GLRf/k9Buf9 L3qSja2VxjRydZFLXfC4KMgOn9aquxG3aPHqOH0SdSZkMaOtw5U19SUGzYC8WfTx YBdJ7ZobOtxIad+zqEaqjCnEdV/3EYBij4ma7ybQ9PgDF0YHcY85M/W/Z1rX3e0a kF5ndg/dLyrFEsbz6lT0IZRiw4yymkIhZ56mUnYIwYRHjY8ZW4Zg3njBHFzpyJ6K seZJCKa00mkCpQfwhPTrLcHqTncNp4uLnTKvmnzLrjN3MiA3zqvEBgQ9EUJpegD+ Ga3ZsyRDrhsbf7nICP64KGWwYvqPcVxwKR0hDrkMJbjh6ucX6aNG6d++bRbf731X mKF1nFtEljrkmK9L1gPFDN0yX1EIdu0ZkSfTaX6kzNBKeH5PXScYROBY0BCiGy4R MjTJQcyC9KgL9dvhVdcJxLKxuK21Pi42+DxBQIFC+aXq2eyDAMmRen3LguePPhXX 3l0rjFtOJi6sDd63k
  • From Thomas Goirand@21:1/5 to Pirate Praveen on Fri Oct 8 22:50:02 2021
    On 10/8/21 7:51 PM, Pirate Praveen wrote:
    Draft text for the GR (this is my first time proposing a GR text, so
    help is welcome to make the text clearer).

    We should not worry about minified files in main that are (a) only in
    the source package and not in any binary package, and (b) only used to
    run tests, not to build the binary packages.

    To ensure these are not used during generation of binary packages, a
    nocheck build profile should be provided which will remove these files
    from the build environment.

    Rationale: Most of the time these minified files create a specific
    instance of a test case and these tests are not the only way to ensure functionality of the code we ship. Even though upstream runs these
    tests, during transitions we deviate from upstream for runtime
    dependencies and a way to ensure the functionality is not broken by a dependency change is essential. Insisting these test cases are built
    using only packages in main just makes running tests harder and disabing tests is choosen as an easier option. Treating the code used to run only tests is the same as generated code we ship is not useful. In case of
    the code we ship missing sources, our users can't modify the software.
    But if these sourceless code is used during tests, only ability to
    modify the test cases is lost. A test case is an arbitrary combination
    of the code and we have many different ways of testing a code.

    Option 1: Agree, we should allow these files in main
    Option 2: Disagree, we should not allow these files in main
    Option 3: Further discussion



    A few problems with what you wrote above...

    * Please don't use the word "minified" which is too much in the
    Javascript world. Please use "binary object without source code"
    instead, so it's more generic to any language.

    * The "rationale" has nothing to do in there, if you're just attempting
    to make a policy change.

    * What you're proposing is clearly in strong opposition to the DFSG and
    social contract, so it's IMO invalid if you're not proposing a change
    over there as well to include your exception. Please propose such change
    in your GR.

    But if these sourceless code is used during tests, only ability to
    modify the test cases is lost.

    I strongly do not agree with the sentence above: in the way it's
    constructed, in how it's formulated, and with the consequences.

    I am convince that, without running a test case, it's not reasonable to
    even attempt to modify a piece source code. Tests are written because
    it's easy to break some code. Not having access to the test case makes
    it very hard to understand how it works, and how the software is
    actually built, and potentially hacked/changed.

    A test case is an arbitrary combination of the code and we have many different ways of testing a code.

    A test case is not "an arbitrary combination of the code", in a TDD way,
    it's often written before the rest of the code, and it is necessary in
    order to avoid breakage. If you're arguing that tests could be rewritten
    by anyone, then yeah: do it and ship the rewritten fully-free-software
    test then, if it's that easy... This is rhetorical: both you and me know
    it's not the case that it is easy to rewrite a full set of tests. It
    takes a lot of time too.

    Cheers,

    Thomas Goirand (zigo)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)