• Maybe not a Tcl question

    From Alexandru@21:1/5 to All on Thu Dec 16 14:41:12 2021
    I' trying to process a redirect from this URL https://www.scilab.org/download using Tcl:

    # Compose the URL of the Software executable
    set url "https://www.scilab.org/download"
    # Get URL content (the server automatically redirects to a link that contains the actual release number in it)
    set code [catch {set token [::http::geturl $url]} err]
    # If no communication error is returned
    if {!$code} {
    # Get server response
    upvar #0 $token state
    set ncode [lindex $state(http) 1]
    # If server redirects
    if {[string index $ncode 0]==3} {
    array set meta $state(meta)
    if {![info exists meta(Location)]} {
    http::cleanup $token
    return
    }
    set url $meta(Location)
    } else {
    return
    }
    } else {
    return
    }
    return $url

    The redirect os of type "moved permanently" and my browser can process the redirect correctly. But the obove code in Tcl leads to same URL as the source URL. So the Location is the same. That would theoretically lead to an infinite loop.

    Any ideas how to deal with this? Are there any hidden informations in the response header that gives the true target URL?

    Thanks
    Alexandru

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Schelte@21:1/5 to Alexandru on Fri Dec 17 00:10:54 2021
    On 16/12/2021 23:41, Alexandru wrote:
    I' trying to process a redirect from this URL https://www.scilab.org/download using Tcl:

    [snip]

    The redirect os of type "moved permanently" and my browser can process the redirect correctly. But the obove code in Tcl leads to same URL as the source URL. So the Location is the same. That would theoretically lead to an infinite loop.

    Using my www package
    (https://chiselapp.com/user/schelte/repository/www), the following code:
    package require www 2.0
    www log puts
    puts [www get https://www.scilab.org/download]

    Produces:
    GET /download HTTP/1.1
    Host: www.scilab.org
    User-Agent: Tcl-www/2.0 (linux-gnu)
    Accept: */*
    Accept-Encoding: identity, gzip, deflate

    HTTP/1.1 301 Moved Permanently
    Date: Thu, 16 Dec 2021 23:03:44 GMT
    Server: Apache
    Strict-Transport-Security: max-age=63072000; includeSubdomains; Permissions-Policy: fullscreen=*
    X-Content-Type-Options: nosniff
    Location: https://www.scilab.org/download/
    Cache-Control: max-age=1209600
    Expires: Thu, 30 Dec 2021 23:03:44 GMT
    Content-Length: 240
    Content-Type: text/html; charset=iso-8859-1

    Received 240/240
    Redirected to: https://www.scilab.org/download/
    GET /download/ HTTP/1.1
    Host: www.scilab.org
    User-Agent: Tcl-www/2.0 (linux-gnu)
    Accept: */*
    Accept-Encoding: identity, gzip, deflate

    HTTP/1.1 200 OK
    Date: Thu, 16 Dec 2021 23:03:44 GMT
    Server: Apache
    Strict-Transport-Security: max-age=63072000; includeSubdomains; Permissions-Policy: fullscreen=*
    X-Content-Type-Options: nosniff
    Last-Modified: Mon, 01 Nov 2021 13:17:29 GMT
    ETag: "83-5cfb9ffad4c40-gzip"
    Accept-Ranges: bytes
    Cache-Control: max-age=1209600
    Expires: Thu, 30 Dec 2021 23:03:44 GMT
    Vary: Accept-Encoding
    Content-Encoding: gzip
    X-XSS-Protection: 1; mode=block
    Referrer-Policy: no-referrer-when-downgrade
    Content-Length: 121
    Content-Type: text/html

    Received 121/121
    <!DOCTYPE html>
    <html>
    <head>
    <meta http-equiv="refresh" content="0; url='/latest'" />
    </head>
    <body>
    </body>
    </html>

    So you end up with an HTTP meta element that points to https://www.scilab.org/latest

    Any ideas how to deal with this? Are there any hidden informations in
    the response header that gives the true target URL?

    The Location header gives the true target URL. "/download/" is not the
    same as "/download". The trailing slash makes a difference.


    Schelte.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Alexandru@21:1/5 to Schelte on Thu Dec 16 15:21:27 2021
    Schelte schrieb am Freitag, 17. Dezember 2021 um 00:10:57 UTC+1:
    On 16/12/2021 23:41, Alexandru wrote:
    I' trying to process a redirect from this URL https://www.scilab.org/download using Tcl:

    [snip]

    The redirect os of type "moved permanently" and my browser can process the redirect correctly. But the obove code in Tcl leads to same URL as the source URL. So the Location is the same. That would theoretically lead to an infinite loop.

    Using my www package
    (https://chiselapp.com/user/schelte/repository/www), the following code: package require www 2.0
    www log puts
    puts [www get https://www.scilab.org/download]

    Produces:
    GET /download HTTP/1.1
    Host: www.scilab.org
    User-Agent: Tcl-www/2.0 (linux-gnu)
    Accept: */*
    Accept-Encoding: identity, gzip, deflate

    HTTP/1.1 301 Moved Permanently
    Date: Thu, 16 Dec 2021 23:03:44 GMT
    Server: Apache
    Strict-Transport-Security: max-age=63072000; includeSubdomains; Permissions-Policy: fullscreen=*
    X-Content-Type-Options: nosniff
    Location: https://www.scilab.org/download/
    Cache-Control: max-age=1209600
    Expires: Thu, 30 Dec 2021 23:03:44 GMT
    Content-Length: 240
    Content-Type: text/html; charset=iso-8859-1

    Received 240/240
    Redirected to: https://www.scilab.org/download/
    GET /download/ HTTP/1.1
    Host: www.scilab.org
    User-Agent: Tcl-www/2.0 (linux-gnu)
    Accept: */*
    Accept-Encoding: identity, gzip, deflate

    HTTP/1.1 200 OK
    Date: Thu, 16 Dec 2021 23:03:44 GMT
    Server: Apache
    Strict-Transport-Security: max-age=63072000; includeSubdomains; Permissions-Policy: fullscreen=*
    X-Content-Type-Options: nosniff
    Last-Modified: Mon, 01 Nov 2021 13:17:29 GMT
    ETag: "83-5cfb9ffad4c40-gzip"
    Accept-Ranges: bytes
    Cache-Control: max-age=1209600
    Expires: Thu, 30 Dec 2021 23:03:44 GMT
    Vary: Accept-Encoding
    Content-Encoding: gzip
    X-XSS-Protection: 1; mode=block
    Referrer-Policy: no-referrer-when-downgrade
    Content-Length: 121
    Content-Type: text/html

    Received 121/121
    <!DOCTYPE html>
    <html>
    <head>
    <meta http-equiv="refresh" content="0; url='/latest'" />
    </head>
    <body>
    </body>
    </html>

    So you end up with an HTTP meta element that points to https://www.scilab.org/latest
    Any ideas how to deal with this? Are there any hidden informations in
    the response header that gives the true target URL?

    The Location header gives the true target URL. "/download/" is not the
    same as "/download". The trailing slash makes a difference.


    Schelte.
    Oh, indeed. I missed that trailing slash.
    Thank you very much!
    Regards
    Alexandru

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)