• Bug#872944: www.debian.org: Debian Policy Manual not fully published

    From Russ Allbery@21:1/5 to Sean Whitton on Wed Aug 23 07:30:02 2017
    XPost: linux.debian.bugs.dist

    Sean Whitton <spwhitton@spwhitton.name> writes:

    The Debian Policy Manual just changed its HTML output and while the HTML
    has published, the CSS and included images have not.

    Looking at [1], the CSS and included images should have been published because they're still installed to
    /usr/share/doc/debian-policy/policy.html/. So I think this is a problem
    on your end rather than ours. Please do reassign this bug if I'm wrong
    about that, and thanks in advance for your help.

    [1] https://anonscm.debian.org/cgit/debwww/cron.git/tree/parts/7doc

    Hi debian-www folks,

    Just a heads-up that we're fixing some more stuff and Policy 4.1.1.0 will
    be a bit different again (it's not yet uploaded). Here's what I currently
    have staged for the next version:

    * usr/share/doc/debian-policy/policy.html/ in the package will have the
    multi-file HTML version. This is a directory with a couple of
    subdirectories, and all the internal HTML links should be relative.
    However, please note that some of the Javascript is symlinks to files
    that are shipped in the libjs-sphinxdoc package, so you may need a bit
    of machinery to turn those symlinks into real files.

    * usr/share/doc/debian-policy/policy-1.html will be the single-file HTML
    version. However, the Sphinix single-file version still requires static
    assets in usr/share/doc/debian-policy/{_images,_static} that will need
    to be in the right position relative to policy-1.html. These have the
    same symlinks to libjs-sphinxdoc.

    The current state in the archive right now only has the single-file
    version, and it is in the policy.html subdirectory. But we got feedback
    after this release that people really wanted both versions available.

    Unfortunately things might be just a touch chaotic for a bit as we work
    through finishing this reStructuredText conversion. Apologies for the
    extra work! We think the long-term result will be worth it both in terms
    of better output and in terms of a more maintainable Policy document.

    --
    Russ Allbery (rra@debian.org) <http://www.eyrie.org/~eagle/>

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Laura Arjona Reina@21:1/5 to All on Thu Aug 24 10:10:03 2017
    XPost: linux.debian.bugs.dist

    Hello Russ, Sean and website team

    Thanks for your mail.

    El 23/08/17 a las 06:01, Russ Allbery escribió:
    Sean Whitton <spwhitton@spwhitton.name> writes:

    The Debian Policy Manual just changed its HTML output and while the HTML
    has published, the CSS and included images have not.

    Looking at [1], the CSS and included images should have been published
    because they're still installed to
    /usr/share/doc/debian-policy/policy.html/. So I think this is a problem
    on your end rather than ours. Please do reassign this bug if I'm wrong
    about that, and thanks in advance for your help.

    [1] https://anonscm.debian.org/cgit/debwww/cron.git/tree/parts/7doc

    The CSS and images issue is solved now.

    If you publish a future version of Debian Policy including a customized
    theme, if that theme is in the _static folders, it will be shown too. If
    the theme is in another folder, ping us, because the function
    mvhtml_sphinx() in our https://anonscm.debian.org/cgit/debwww/cron.git/tree/parts/7doc script
    should be modified to include the new folder.


    Hi debian-www folks,

    Just a heads-up that we're fixing some more stuff and Policy 4.1.1.0 will
    be a bit different again (it's not yet uploaded). Here's what I currently have staged for the next version:

    * usr/share/doc/debian-policy/policy.html/ in the package will have the
    multi-file HTML version. This is a directory with a couple of
    subdirectories, and all the internal HTML links should be relative.> However, please note that some of the Javascript is symlinks to files
    that are shipped in the libjs-sphinxdoc package, so you may need a bit
    of machinery to turn those symlinks into real files.

    These Javascript symlinks are present in the current version too, and
    I'm thinking about how to solve this issue.

    I've found that for the .js files to be "available" we need to unpack
    these packages (in a similar fashion as we do with debian-policy package):

    libjs-sphinxdoc
    libjs-jquery
    libjs-underscore

    Then, the /usr tree that we create in the crondir/tmp for unpacking and
    copying the manuals to the website, will be consistent, but until now we
    were copying only documents or images under the /usr/share/doc folder in
    that tree, and now we should copy *code* files under the
    /usr/share/javascript folder too.
    I'm not sure about the points below:

    * If we should copy those javascript files to "live" under www.debian.org/doc/policy-manual or in another place in the
    www.debian.org site (or in cgi.debian.org?). For now, policy-manual is
    the only manual using them, but maybe in the future, more manuals are
    moved to use sphinx too. I'm not aware of any other piece of the website
    using Javascript, so I have no references of a canonical place to copy
    the files (we would also need to change the symlinks to point to that
    place, but that's another topic).

    * If we should use the packages from "sid" as we use for the
    documentation packages, or stable, or backports...

    * If the www-master.debian.org machine (and all the mirrors) should have
    the libjs-sphinxdoc, libjs-jquery and libjs-underscore packages
    installed, and we should find the way to use those from the website
    (change the symlinks, I guess, to... which path?)

    * Other solution?

    I hope other website team members or DSA can shed a light on the best
    way to solve this.

    Cheers

    --
    Laura Arjona
    https://wiki.debian.org/LauraArjona


    * usr/share/doc/debian-policy/policy-1.html will be the single-file HTML
    version. However, the Sphinix single-file version still requires static
    assets in usr/share/doc/debian-policy/{_images,_static} that will need
    to be in the right position relative to policy-1.html. These have the
    same symlinks to libjs-sphinxdoc.

    The current state in the archive right now only has the single-file
    version, and it is in the policy.html subdirectory. But we got feedback after this release that people really wanted both versions available.

    Unfortunately things might be just a touch chaotic for a bit as we work through finishing this reStructuredText conversion. Apologies for the
    extra work! We think the long-term result will be worth it both in terms
    of better output and in terms of a more maintainable Policy document.


    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Laura Arjona Reina@21:1/5 to All on Wed Aug 23 16:20:02 2017
    XPost: linux.debian.bugs.dist

    Hello again

    El 23/08/17 a las 09:43, Laura Arjona Reina escribió:


    See commit:

    https://anonscm.debian.org/cgit/debwww/cron.git/commit/?id=64faa5504f980c377ed6e44d0a61f1943336b102


    Well, that didn't work but it was because the fix was incomplete (I was
    not creating the correct _images and _static subfolders).
    The full patch would be this one:

    https://anonscm.debian.org/cgit/debwww/cron.git/diff/?id=b15d0a058f78b27a2588e6f020f10409f594e590&id2=eef2ed535138e12afd3878c8cd9a486cae489e58

    Cheers
    --
    Laura Arjona Reina
    https://wiki.debian.org/LauraArjona

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Laura Arjona Reina@21:1/5 to All on Wed Aug 23 11:00:02 2017
    XPost: linux.debian.bugs.dist

    Hello

    I've been reading the 7doc script and I think the problem is that
    currently the sphinx-based policy manual stores the css files in a
    subfolder called _static and our cron script does not contemplate that.

    The called function is mvhtml() that uses pagepattern="$basedir/*.css"
    for moving the css files, but that does not match the structure of
    sphinx-based documentation.

    We have another function mvhtml2() that uses pagepattern="$basedir/Common_Content/css/*.css" (for the Debian
    Handbook), but that structure of folders does not match the sphinx-based documentation either.

    I have created a new mvhtml_sphinx() function, adapting mvhtml() to copy
    _static/* (instead of /*.css) as well as _image/* (instead of image/*);
    and called that function for debian-policy manual, instead of calling
    mvhtml().

    See commit:

    https://anonscm.debian.org/cgit/debwww/cron.git/commit/?id=64faa5504f980c377ed6e44d0a61f1943336b102

    Let's see if after that the debian-policy manual appears in the website incuding images and styiling.

    The _static folder includes some symlinks to javascript. Those will
    probably not work yet, we'll need to tweak more the mvhtml_sphinx()
    function, but let's go step by step.

    Cheers

    --
    Laura Arjona Reina
    https://wiki.debian.org/LauraArjona


    El 22/08/17 a las 21:21, Sean Whitton escribió:
    Package: www.debian.org
    Severity: important

    Hello webmasters,

    The Debian Policy Manual just changed its HTML output and while the
    HTML has published, the CSS and included images have not.

    Looking at [1], the CSS and included images should have been published because they're still installed to
    /usr/share/doc/debian-policy/policy.html/. So I think this is a problem
    on your end rather than ours. Please do reassign this bug if I'm wrong
    about that, and thanks in advance for your help.

    [1] https://anonscm.debian.org/cgit/debwww/cron.git/tree/parts/7doc

    -- System Information:
    Debian Release: buster/sid
    APT prefers testing
    APT policy: (900, 'testing'), (100, 'unstable'), (1, 'experimental') Architecture: amd64 (x86_64)

    Kernel: Linux 4.12.0-1-amd64 (SMP w/4 CPU cores)
    Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8), LANGUAGE=en_GB.UTF-8 (charmap=UTF-8)
    Shell: /bin/sh linked to /bin/dash
    Init: systemd (via /run/systemd/system)


    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Julien Cristau@21:1/5 to Laura Arjona Reina on Thu Aug 24 11:10:03 2017
    XPost: linux.debian.bugs.dist

    On 08/24/2017 09:01 AM, Laura Arjona Reina wrote:
    I'm not sure about the points below:

    * If we should copy those javascript files to "live" under www.debian.org/doc/policy-manual or in another place in the
    www.debian.org site (or in cgi.debian.org?). For now, policy-manual is
    the only manual using them, but maybe in the future, more manuals are
    moved to use sphinx too. I'm not aware of any other piece of the website using Javascript, so I have no references of a canonical place to copy
    the files (we would also need to change the symlinks to point to that
    place, but that's another topic).

    * If we should use the packages from "sid" as we use for the
    documentation packages, or stable, or backports...

    * If the www-master.debian.org machine (and all the mirrors) should have
    the libjs-sphinxdoc, libjs-jquery and libjs-underscore packages
    installed, and we should find the way to use those from the website
    (change the symlinks, I guess, to... which path?)

    That won't work. The mirroring will not like symlinks that point
    outside of the source /srv/www.debian.org/www directory on www-master, IIRC.

    * Other solution?

    We could add aliases in the apache config on www mirrors to point at /usr/share/javascript/foo, I guess, but that assumes things are happy
    with the stable version of the libjs-* packages, which I'm not sure we
    can rely on.

    Cheers,
    Julien

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Sean Whitton@21:1/5 to Laura Arjona Reina on Fri Aug 25 00:20:04 2017
    XPost: linux.debian.bugs.dist

    Hello Laura,

    On Thu, Aug 24 2017, Laura Arjona Reina wrote:

    The CSS and images issue is solved now.

    Thank you for this!

    If you publish a future version of Debian Policy including a
    customized theme, if that theme is in the _static folders, it will be
    shown too. If the theme is in another folder, ping us, because the
    function mvhtml_sphinx() in our https://anonscm.debian.org/cgit/debwww/cron.git/tree/parts/7doc script
    should be modified to include the new folder.

    Okay. I don't know enough about Sphinx themes to say in advance.

    * If we should copy those javascript files to "live" under www.debian.org/doc/policy-manual or in another place in the
    www.debian.org site (or in cgi.debian.org?). For now, policy-manual is
    the only manual using them, but maybe in the future, more manuals are
    moved to use sphinx too. I'm not aware of any other piece of the website using Javascript, so I have no references of a canonical place to copy
    the files (we would also need to change the symlinks to point to that
    place, but that's another topic).

    * If we should use the packages from "sid" as we use for the
    documentation packages, or stable, or backports...

    If it's easy, we'd prefer you to use them from sid, because that's what
    we develop the package against.

    --
    Sean Whitton

    --=-=-Content-Type: application/pgp-signature; name="signature.asc"

    -----BEGIN PGP SIGNATURE-----

    iQIzBAEBCgAdFiEEm5FwB64DDjbk/CSLaVt65L8GYkAFAlmfP5oACgkQaVt65L8G YkALZg//cBYj2+TcDRl/XdLYALA0M1nW3DhBKj+J5vmgjS7eK8JzgNn3dkgZvbwB 3Zljd2ZDHKzQ6kJvsGgu263nZS7u838WwxaCn7qkorAV5nA9nlI1JS8qUNJN7oYn HKFmKDkdQE7meZgKdmWgiCMmd+w+JHNVs++2HggaWSiI4Nq/vimh/jWOZdG/AXGt qEFiLvcfTMQK/BuvsgqOhkgELmrkT+TKJY00guQlkAy729lQ0ytRZYjlHzptzPXf syxiw5TElTVuWOtcqzZrzaj8xQqyVxt23uyM/UaA0fzWqA+hm+RclR0UMmLwPqby 47q5SqV/M7tslZry14CNFjFmnqF/V18pqU7TOVI9jmgCpHRSLUxiFxwM4bxXXZDZ i3w7pG3m6e9IVwtId9SFlXimPZE/XCml1wNNnQyOYN9ptJuJfwDyBnaLZMPRzrTT sq1W5lu9zuPWg9zP0l+xayeYAJh/quuRbhxCjSO7IISUb88jNsQJ5Uq9bQQCtbJr cu+qdlIU9sFY9aFBPDqPx7E6JqX62fEUZNxTTgQdWGD92QhwHMDUL5GM8Fx7c76a coTnFgJgZ64imQmTUrLhA4lA7luES5D2VZKuRUCIZiMWB5CoZ8qlqhvc8Mzxz1cW Etc3KSSiJiMHWcDKmlHedAqlFnoIA/BrdhDZ0ykSQWFWRjfqZDI=Aq7q
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Laura Arjona Reina@21:1/5 to All on Fri Aug 25 01:20:01 2017
    XPost: linux.debian.bugs.dist

    El 24/08/17 a las 23:05, Sean Whitton escribió:
    Hello Laura,

    On Thu, Aug 24 2017, Laura Arjona Reina wrote:


    * If we should use the packages from "sid" as we use for the
    documentation packages, or stable, or backports...

    If it's easy, we'd prefer you to use them from sid, because that's what
    we develop the package against.


    I guess it's the same easy to use one of another. My concern is about doing the right thing... making our web visitors run javascript code from sid in their browsers does not sound right for me.

    Would you (Debian Policy Team) consider acceptable to leave the website version of the manual as it is now, without any javascript? I guess we are loosing some search function and maybe some text animations, but for what I've seen, the page
    looks ok... and in the manual it's said that the manual is provided by the package debian-policy, and that is available in different formats. The website version could be considered a "plain HTML" version, then...

    Best regards

    --
    Laura Arjona Reina
    https://wiki.debian.org/LauraArjona

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Sean Whitton@21:1/5 to Laura Arjona Reina on Fri Aug 25 22:40:01 2017
    XPost: linux.debian.bugs.dist

    On Fri, Aug 25 2017, Laura Arjona Reina wrote:

    My concern is about doing the right thing... making our web visitors
    run javascript code from sid in their browsers does not sound right
    for me.

    Would you (Debian Policy Team) consider acceptable to leave the website version
    of the manual as it is now, without any javascript?

    I am sympathetic to your concern.

    I'd want us to generate output that doesn't try to load any JavaScript,
    though, rather than publishing something which we expect to be buggy.
    [1] looks like a good starting point.

    Russ: do you agree? If so, we can file a bug against policy to produce
    output without javascript, and block this bug by that one.

    [1] https://stackoverflow.com/questions/31951553/customizing-sphinx-to-avoid-generating-search-box

    --
    Sean Whitton

    --=-=-Content-Type: application/pgp-signature; name="signature.asc"

    -----BEGIN PGP SIGNATURE-----

    iQIzBAEBCgAdFiEEm5FwB64DDjbk/CSLaVt65L8GYkAFAlmge9oACgkQaVt65L8G YkCWIxAArW48AMq6sU9a2EWz0o2+EzRpdZK4zeBNrINkfAM6fc5uYUzIyW0Zfh+7 LEqudH/YjD5fGAEzMyCKYQWvLK14wwif1EeKED2Z06fL0EjxPj2FAogPSjfUkNLo Al4i0Gu+cpvdYKph7Sk2t6M5wGNHoc2Rar8QUrxCxDRrVkxB4faYnAOSWkZOhVpA xeSyZcAxlyiqw3FTvFZapsDE/17LuIhKqXj/GDmJLU8boPeGxEcplYhuyjCcT+rU MHleI0JbHNvlVfIm/bAnd2eiJIa3xpqSBvjwDNgblkuxJ3X6PWPPP91g00zvqBdB JPHIobDdaFCyKebWXG6X5RYez+gI0kc7GmBo8LRuYHAqUgprGF+QDLi9lW02ufg8 d5pP4//OuR3siCsEsM+GbZkI6QqBV0Cect55xB7pssXhMtFoXtiu2Okrn89cwy5j 3ZVTuTSjeaJHsa5XVTC1peyHx7Bp1P6CIUCkZ8KgZvM20BwlOQHYlZhI+OFgv9M4 c64RlwrEDVZQT/rP+GygKHyU0quniDyDtZig2QOTysXzlq2vZEDtiv4RJBiNC7mz JBbrLHj3VCFg/iArp4OG3tBRZoZzYLOHb2/R/4qR3N+aFg5oGCLKmC7SxwiuK+XI mKlElktYsX2uo6KnKmFYnyP7xXbBcHgmHmHK3UM6GJwrVJ0+Ae0=NjE3
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to Sean Whitton on Mon Sep 18 01:00:01 2017
    XPost: linux.debian.bugs.dist

    Sean Whitton <spwhitton@spwhitton.name> writes:
    On Fri, Aug 25 2017, Laura Arjona Reina wrote:

    My concern is about doing the right thing... making our web visitors
    run javascript code from sid in their browsers does not sound right for
    me.

    It's fairly unlikely that this would cause a problem in practice given how
    the Javascript is used in this case and given the other contents of www.debian.org. The primary concern with Javascript is that it could
    expose the site to XSS or other web vulnerabilities, but I believe the
    content of www.debian.org is entirely public, so there's no meaningful XSS
    or CSRF or related vulnerability that I can think of.

    The remaining issues seem fairly obscure.

    That said, introducing Javascript for the first time does feel like a
    large-ish step, and the reluctance also makes sense. I'm not sure the
    search functionality really adds much. (I haven't checked to confirm that
    is the only thing in the Sphinx output that uses Javascript, and that it's
    not used for something more useful like responsive design on mobile
    browsers, but maybe Sean has.)

    Would you (Debian Policy Team) consider acceptable to leave the website
    version of the manual as it is now, without any javascript?

    I have no objections! I'm happy to have the web team make the call for
    what makes the most sense for the web site.

    I'd want us to generate output that doesn't try to load any JavaScript, though, rather than publishing something which we expect to be buggy.
    [1] looks like a good starting point.

    Russ: do you agree? If so, we can file a bug against policy to produce output without javascript, and block this bug by that one.

    I suppose that also works, although it assumes that the only use of
    Javascript is just the search box. I don't really want to do a lot of
    meddling with the Sphinx output (since part of the goal is to let Sphinx
    take care of the details of output), but this doesn't look like a ton of
    work and looks likely to continue to be supported.

    --
    Russ Allbery (rra@debian.org) <http://www.eyrie.org/~eagle/>

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)