1. Forum
  2. Usenet
  3. COMP.TEXT.XML

  • Parsing XML pages

    From paolopiace@gmail.com@21:1/5 to All on Sat Jul 23 19:59:25 2016
    This url

    http://finance.yahoo.com/quote/GE/history?period1=0&period2=1469170800&interval=div|split&filter=split&frequency=1d

    outputs a page which at its bottom has this content:

    https://1drv.ms/i/s!AhvJcZiY8TTdhWx_35S5R2hZ99BX

    I save the source page html and search some strings in it.
    I search "3/1", "Stock Split", "May 16, 1994" and so on.

    Well, nothing like this is in the source page!

    Where the hell are those info? If I see them on the browser, they must be stored somewhere. If not in the html source page, where are they?

    May I have some directions, please?

    Thanks.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Peter Flynn@21:1/5 to paolopiace@gmail.com on Mon Aug 1 13:30:22 2016
    On 24/07/16 03:59, paolopiace@gmail.com wrote:
    This url

    http://finance.yahoo.com/quote/GE/history?period1=0&period2=1469170800&interval=div|split&filter=split&frequency=1d

    outputs a page which at its bottom has this content:

    https://1drv.ms/i/s!AhvJcZiY8TTdhWx_35S5R2hZ99BX

    I save the source page html and search some strings in it.
    I search "3/1", "Stock Split", "May 16, 1994" and so on.

    Well, nothing like this is in the source page!

    Unsurprising, given that it's financial.

    Where the hell are those info?

    They are being inserted in real time from an external source, probably
    via Javascript.

    If I see them on the browser, they must be stored somewhere.

    Or they might be being calculated on-the-fly, from *data* stored elsewhere.

    If not in the html source page, where are they?

    They have been deliberately obfuscated so that you can't steal them.

    May I have some directions, please?

    Use a browser which has an Inspection mode. Right-click one of the
    values and look at the pseudo-HTML:

    <td class="Ta(c) Py(10px)" colspan="5" data-reactid=".1kvth1ckyua.1.$0.0.0.3.1.$main-0-Quote-Proxy.$main-0-Quote.0.2.0.2.$history-table.1.$0.1"><strong
    data-reactid=".1kvth1ckyua.1.$0.0.0.3.1.$main-0-Quote-Proxy.$main-0-Quote.0.2.0.2.$history-table.1.$0.1.0">3/1</strong><span
    data-reactid=".1kvth1ckyua.1.$0.0.0.3.1.$main-0-Quote-Proxy.$main-0-Quote.0.2.0.2.$history-table.1.$0.1.1">
    </span><span data-reactid=".1kvth1ckyua.1.$0.0.0.3.1.$main-0-Quote-Proxy.$main-0-Quote.0.2.0.2.$history-table.1.$0.1.2">Stock
    Split</span></td>

    etc. Now go find the code which recognises this, unscramble it, and find
    out what machine it's coming from. Then break into the machine to get at
    the source data (just kidding, NSA :-)

    Good luck...

    ///Peter

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Luuk@21:1/5 to paolopiace@gmail.com on Sat Aug 6 11:43:51 2016
    On 24-07-16 04:59, paolopiace@gmail.com wrote:
    This url

    http://finance.yahoo.com/quote/GE/history?period1=0&period2=1469170800&interval=div|split&filter=split&frequency=1d

    outputs a page which at its bottom has this content:

    https://1drv.ms/i/s!AhvJcZiY8TTdhWx_35S5R2hZ99BX

    I save the source page html and search some strings in it.
    I search "3/1", "Stock Split", "May 16, 1994" and so on.

    Well, nothing like this is in the source page!

    Where the hell are those info? If I see them on the browser, they must be stored somewhere. If not in the html source page, where are they?

    May I have some directions, please?

    Thanks.


    If you want to do something with the output, you probably should have a
    look at this page: http://meumobi.github.io/stocks%20apis/2016/03/13/get-realtime-stock-quotes-yahoo-finance-api.html

    an (slighly modified) example from the documention: https://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20yahoo.finance.quotes%20where%20symbol%20in%20(%22YHOO%22)&format=xml&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys&callback=


    or, above the table on the page you linked, i see alink to 'Download
    Data', which give a csv with the data in the table

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)