• FireFox Search provider XML description sought

    From R.Wieser@21:1/5 to All on Wed Jul 14 09:51:37 2021
    JJ,

    Though I'm now curious to why you thought I ment otherwise ...

    Because you think deleting "search.json.mozlz4" file is enough.

    ???? I'm missing the part where you explain why it wouldn't be. Mind you,
    I already wrote that deleting that file seems to work fine for me.

    It seems more than likely to me that you are thinking/talking about a
    different version of FF than I specified in my initial message (v52). It certainly would explain the confusion.

    Also, somehow I can't wrap my head around how you could have jumped to me wanting to create a new, second "search engine list" from that 'delete the
    file to reinitialize' remark. You stil absolutily lost me there.

    Regards,
    Rudy Wieser

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From JJ@21:1/5 to R.Wieser on Wed Jul 14 18:43:13 2021
    On Wed, 14 Jul 2021 09:51:37 +0200, R.Wieser wrote:

    It seems more than likely to me that you are thinking/talking about a different version of FF than I specified in my initial message (v52). It certainly would explain the confusion.

    Ah, you're right. I wasn't paying attention to the version number.

    For older version, use the `window.external.AddSearchProvider(url)` where
    `url` is the URL pointing to a HTTP/S served OpenSearch XML resource with
    the correct OpenSearch MIME type.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From R.Wieser@21:1/5 to All on Wed Jul 14 15:09:34 2021
    JJ,

    Ah, you're right. I wasn't paying attention to the version number.

    For further reference (though I think you already know) : FF v52 is the very last one that will run on XP (the newsgroup we are in).

    For older version, use the `window.external.AddSearchProvider(url)`

    Two things: First that would need some coding, likely in the form of yet another plugin. I think that for "one off" stuff like this the crude
    method works as well and takes a lot less work. Second is that it doesn't allow me to load my own, local (dummy) "search engine" XML file.

    You know, it /almost/ feels like the FF guys really didn't want me to be
    able to add my own. :-D

    Regards,
    Rudy Wieser

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From JJ@21:1/5 to R.Wieser on Thu Jul 15 13:22:53 2021
    On Wed, 14 Jul 2021 15:09:34 +0200, R.Wieser wrote:

    You know, it /almost/ feels like the FF guys really didn't want me to be
    able to add my own. :-D

    You're not wrong. They never want us to be able to do so.

    First of all, they never adds the ability to specifically add a new, or edit existing search engine (and not just delete).

    Then they make ridiculously strict OpenSearch requirements.

    And finally, they "lock" the search engine list file with a compression
    which no way for a user to specifically uncompress/recompress it. They might argue that it's for protection against malwares, but the fact that users
    can't still able to specifically add a new, or edit existing search engine, only gives hints that they simply don't want us to be able to do those.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From R.Wieser@21:1/5 to All on Thu Jul 15 10:49:30 2021
    JJ,

    You're not wrong. They never want us to be able to do so.

    The question is: why ? The "search engine" XML file effectivily just contains a three references to the same website, only differing in usage.
    Whats there to be denied access to - or dangerous if a user edits it himself
    ?

    And finally, they "lock" the search engine list file with a
    compression which no way for a user to specifically
    uncompress/recompress it.

    :-) I've been looking at that too - if only to see whats actually in that "search.json.mozlz4" file. Presumably the same as in the "search.json"
    file, but you never know. The compression seems to be LZ4, and I did manage
    to find a Windows DLL for it (on GitHub). I also found some references to programs (in sourcefile format) to at least unpack it.

    [quote]
    When adding the search engine, Firefox sneakily sends a non loggable
    network request using the URL from the OpenSearch LINK element
    where it will be validated then added into Firefox's Search Engine list
    which users can't directly modify existing or create a new Search Engine. [/quote]

    A convoluted approach no doubt build upon legacy requirements.

    If the targetted, supposed to be a search engine, website must be asked to verify the XML file it could as well just return the (microscopic in size)
    file itself (which is than always up-to-date). Which means that the
    initial, to-be-verified copy of the XML file could be replaced by a simple
    URL.

    Regards,
    Rudy Wieser

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Paul@21:1/5 to R.Wieser on Thu Jul 15 15:19:19 2021
    R.Wieser wrote:

    :-) I've been looking at that too - if only to see whats actually in that "search.json.mozlz4" file. Presumably the same as in the "search.json"
    file, but you never know. The compression seems to be LZ4, and I did manage to find a Windows DLL for it (on GitHub). I also found some references to programs (in sourcefile format) to at least unpack it.

    I wrote such a program myself. A short piece of C code to call "lz4.dll",
    so not exactly rocket science :-) I never try to work too hard when
    I write code - I am a copy/pasta programmer by nature.

    You should open "search.json.mozlz4" and "search.json" in
    a text editor and compare them. The LZ4 compressor is little
    better than RLE. You can read a lot of the text strings
    under mozlz4, or guess at their formulation. It's not
    a compressor, and not even a good obfuscater. It's
    a nuisance-ifier.

    It does make you wonder what they were thinking when they
    applied LZ4 :-)

    Writing the compressor call, would be good exercise for you :-)

    And why wouldn't LZ4.c be in the Firefox tarball ? You should
    not necessarily have to go to another site to get it.

    gcc -c lz4.c
    gcc -c unmoz.c
    gcc -o unmoz.exe lz4.o unmoz.o

    See, it's hard hard work. You have to check the label on
    the thing, to see whether they've changed it or not. This
    is a small snippet from the program.

    /* mozLz40\0 */
    /* 109, 111, 122, 76, 122, 52, 48, 0 */

    if ( bufin[0] != 109 ||
    bufin[1] != 111 ||
    bufin[2] != 122 ||
    bufin[3] != 76 ||
    bufin[4] != 122 ||
    bufin[5] != 52 ||
    bufin[6] != 48 ||
    bufin[7] != 0 ) {

    perror("\n Version magic does not match mozLz40\n");
    return 0;
    }

    I wrote this because I wanted my damn output, not because
    the source code is "pretty". If you want the compression
    function, copy it from the tarball. There would have to
    be a pre-format call before lz4() is called, to put the
    version string on it. Since they write out mozlz4 files,
    using Firefox, the compressor code has to be in there. And
    with the correct header, a portion of which is demoed above.

    You could try a text search on "mozLz40" and see what pops up
    in the unpacked tarball.

    361MB (unpack with 7ZIP or whatever)

    http://releases.mozilla.org/pub/firefox/releases/90.0/source/firefox-90.0.source.tar.xz

    Paul

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From R.Wieser@21:1/5 to you already on Thu Jul 15 22:21:20 2021
    Paul,

    I wrote such a program myself.

    As will I. If I would have been lazy I would have just grabbed me a pre-compiled extractor from somewhere, and be done with it. But no, I want
    to do it myself. :-)

    A short piece of C code to call "lz4.dll",

    In my case it will be Assembly, but same thing.

    You should open "search.json.mozlz4" and "search.json" in a text editor
    and compare them.

    That was pretty-much my idea. I'm currently not sure about the value of
    that comparision anymore though, as deleting the "search.json" file does not seem to bother FF, nor does it get re-generated.

    The LZ4 compressor is little better than RLE. You can read a lot of the
    text strings under mozlz4

    I noticed (the first thing I do is throwing it it in a hex viewer).

    It does make you wonder what they were thinking when they applied LZ4 :-)

    Well, you already said it yourself : to make anuissance of themselves. :-p

    Writing the compressor call, would be good exercise for you :-)

    Most of the work will go into figuring out (by looking at the example code) which functions need to be called, and which arguments they need.

    .. wait. The "example" directory /doesn't/ contain example pack and unpack progams, just a "fullbench" and "xxhash". Oh well, its going to take a bit more.

    gcc -c lz4.c
    gcc -c unmoz.c
    gcc -o unmoz.exe lz4.o unmoz.o

    Two problems: I'm not using C, nor am I using Linux ...

    Although I was able to find the "lz4.c" file in a sourcefile pack that I
    have backed up (you never know), it did not contain anything by the name "unmoz"

    You could try a text search on "mozLz40" and see what pops up
    in the unpacked tarball.

    361MB (unpack with 7ZIP or whatever)

    http://releases.mozilla.org/pub/firefox/releases/90.0/source/firefox-90.0.source.tar.xz

    :-) You're in a XP newsgroup. The highest version of FF that XP can run
    is 52.

    Thanks for the suggestions though, I'll see what I can do with them.

    Regards,
    Rudy Wieser

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From R.Wieser@21:1/5 to All on Fri Jul 16 13:25:57 2021
    Paul,

    You should open "search.json.mozlz4" and "search.json" in a text editor
    and compare them.

    I just wrote the code to unpack the file (which takes one call with four, rather obvious arguments), and was ready to compare the files - until I realized that "search.json.mozlz4" was about twice as large as
    "search.json". IOW, no chance in hell that the uncompressed (and thus
    larger) file would even come near to having the same contents as
    "search.json". :-)

    But I died find something interresting : references to the build-in "search engine" XML files. If I ever want to make sure that a profile has my
    preferred engine on top than I know where to look.

    Regards,
    Rudy Wieser

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)