• oxp: UTF-8

    From August Abolins@2:221/1.58 to All on Thu Apr 4 00:52:00 2019

    I have Config/Options/MessageReader configured with UTF-8 enabled (it's actually the default.

    But UTF-8 chars don't seem to render properly.

    It that a bug?

    Regards,
    ../|ug

    --- OpenXP 5.0.36
    * Origin: /|ug's Point, ONT, CANADA (2:221/1.58)
  • From Richard Menedetter@2:310/31 to August Abolins on Thu Apr 4 09:43:28 2019
    Hi August!

    04 Apr 2019 00:52, from August Abolins -> All:

    I have Config/Options/MessageReader configured with UTF-8 enabled
    (it's actually the default.
    But UTF-8 chars don't seem to render properly.

    Interesting ... I was not aware that OpenXP has UTF-8 support.
    Need to install it once more and check that out ... later ;)

    CU, Ricsi

    ... The law says you can have only one spouse. This is called monotony.
    --- GoldED+/LNX
    * Origin: Pleasant thoughts make pleasant lives. (2:310/31)
  • From Martin Foster@2:310/31.3 to August Abolins on Thu Apr 4 14:19:00 2019
    Hello August!

    On 04.04.19 at 00:52, August Abolins wrote to All:

    I have Config/Options/MessageReader configured with UTF-8 enabled (it's actually the default.

    But UTF-8 chars don't seem to render properly.

    It that a bug?

    Not sure about that but the developer did a bugfix relating to UTF-8 recently, perhaps that may fix it.

    Regards,
    Martin

    --- OpenXP 5.0.36
    * Origin: Bitz-Box - Bradford - UK (2:310/31.3)
  • From mark lewis@1:3634/12.73 to August Abolins on Thu Apr 4 12:20:28 2019

    On 2019 Apr 04 00:52:00, you wrote to All:

    I have Config/Options/MessageReader configured with UTF-8 enabled
    (it's actually the default.

    But UTF-8 chars don't seem to render properly.

    It that a bug?

    what is your OS using? that's the key... if you are on winwhatever, you may be using CP1252 but in modern winwhatever, it could be UTF-16... the thing is that the old (single byte) codepages are basically out but there's still a lot of software that tries to use them when mapping characters and rendering glyphs... if the character doesn't exist in the SBCP (single byte codepage), then it is not rendered at all or maybe it is rendered as a box with an 'x' in it or possibly as a question mark... if there's a character in that position in the 256 byte codepage, you may see the wrong glyph...

    it gets really deep when entering the world of MBCS (multi byte character sets)...

    FWIW: the message i'm replying to carries a "CHRS: ASCII 1" control line... this is perfectly fine since your post contains only 7bit ASCII characters... if you are interested in looking further, http://ftsc.org/docs/fts-5003.001

    )\/(ark

    Always Mount a Scratch Monkey
    Do you manage your own servers? If you are not running an IDS/IPS yer doin' it wrong...
    ... Frog philosophy: Time's fun when you're having flies.
    ---
    * Origin: (1:3634/12.73)
  • From August Abolins@2:221/1.58 to mark lewis on Thu Apr 4 21:05:00 2019
    Hello mark!

    ** 04.04.19 - 12:20, mark lewis wrote to August Abolins:

    I have Config/Options/MessageReader configured with UTF-8 enabled
    (it's actually the default.

    But UTF-8 chars don't seem to render properly.

    It that a bug?

    what is your OS using? that's the key... if you are on winwhatever, you
    may be using CP1252 but in modern winwhatever, it could be UTF-16...

    On XP here with my T60 Thinkpad. I've never played/changed codepage settings. I'm not even sure how to do that.


    ...then it is not rendered at all or maybe it is rendered as a box
    with an 'x' in it or possibly as a question mark... if there's a
    character in that position in the 256 byte codepage, you may see the
    wrong glyph...

    oxp displays the ? char where I expect the utf-8. The same text looks perfect in Thunderbird/nntp. So, the boss node is retaining the
    integrity of utf-8. But oxp is breaking it. (I'm not sure what WinPoint would do.)


    FWIW: the message i'm replying to carries a "CHRS: ASCII 1" control
    line... this is perfectly fine since your post contains only 7bit ASCII characters...

    I don't know how to pre-view the CHRS with oxp. I can request various
    header info, but CHRS is not one of them. Thanks anyway.


    if you are interested in looking further,
    http://ftsc.org/docs/fts-5003.001

    I may check it out some time in the future. BTW.. I skimmed through some
    of the other fts docs a few weeks ago. The one about the BEER tag had me ROTFL. Are there any more like that?

    Regards,
    ../|ug

    --- OpenXP 5.0.36
    * Origin: /|ug's Point, ONT, CANADA (2:221/1.58)
  • From Martin Foster@2:240/2188.31 to August Abolins on Fri Apr 5 14:47:00 2019
    Hello August!

    On 04.04.19 at 14:19, Martin Foster wrote to August Abolins:

    I have Config/Options/MessageReader configured with UTF-8 enabled (it's
    actually the default.

    But UTF-8 chars don't seem to render properly.

    It that a bug?

    Not sure about that but the developer did a bugfix relating to UTF-8 recently, perhaps that may fix it.

    Later .....

    I've contacted the developer about this and here's what he had to say:

    ---------- 8< ----------
    "Depends on the message content and the specified message charset :-)

    And he also has to configure an appropriate font in the windows
    console, e.g. "Lucida Console".

    Here on the Continent we probably have more UTF-8 encoded messages
    than in GB or in USA, and I think I've pulled the maximum of UTF-8 functionality out of OpenXP (which internally still work with MS-DOS
    Codepage 437 in many areas, as it's roots were in the MS-DOS era of
    the early nineties).

    Maybe he can forward the problematic message to me."
    ---------- 8< ----------

    Over to you :)

    Regards,
    Martin

    --- OpenXP 5.0.36
    * Origin: Bitz-Box - Bradford - UK (2:240/2188.31)
  • From August Abolins@2:221/1.58 to Martin Foster on Fri Apr 5 21:41:00 2019
    Hello Martin!

    ** 05.04.19 - 14:47, Martin Foster wrote to August Abolins:


    I have Config/Options/MessageReader configured with UTF-8 enabled (it's AA>>> actually the default.

    But UTF-8 chars don't seem to render properly.


    I've contacted the developer about this and here's what he had to say:

    ---------- 8< ----------
    "Depends on the message content and the specified message charset :-)

    And he also has to configure an appropriate font in the windows
    console, e.g. "Lucida Console".

    Here on the Continent we probably have more UTF-8 encoded messages
    than in GB or in USA, and I think I've pulled the maximum of UTF-8 functionality out of OpenXP (which internally still work with MS-DOS Codepage 437 in many areas, as it's roots were in the MS-DOS era of
    the early nineties).

    Maybe he can forward the problematic message to me."
    ---------- 8< ----------

    Over to you :)


    Sounds good. I had *not* fiddled with the font offerings in the console before all this. I only played with the screen size.

    My console setting only offers two fonts: True-type Lucinda, or Raster.

    Changed it to Lucinda.

    This was probably the issue! Maybe this is another matter that would be useful in the guide. For proper Unicode/UTF-8 support, set your Console
    to use a suitable TrueType font." ?

    Some swedish/finnish chars:

    ,,,,,

    Looks ok here.

    Some danish/norwegian chars:

    ,,,

    Looks good.

    Let's see if they make it intact up the chain...

    Regards,
    ../|ug

    --- OpenXP 5.0.36
    * Origin: /|ug's Point, ONT, CANADA (2:221/1.58)
  • From Martin Foster@2:310/31.3 to August Abolins on Sat Apr 6 15:02:00 2019
    Hello August!

    On 05.04.19 at 21:41, August Abolins wrote to Martin Foster:

    I have Config/Options/MessageReader configured with UTF-8 enabled
    (it's actually the default.

    But UTF-8 chars don't seem to render properly.

    I've contacted the developer about this and here's what he had to say:

    ---------- 8< ----------
    "Depends on the message content and the specified message charset :-)

    And he also has to configure an appropriate font in the windows
    console, e.g. "Lucida Console".

    Here on the Continent we probably have more UTF-8 encoded messages
    than in GB or in USA, and I think I've pulled the maximum of UTF-8
    functionality out of OpenXP (which internally still work with MS-DOS
    Codepage 437 in many areas, as it's roots were in the MS-DOS era of
    the early nineties).

    Maybe he can forward the problematic message to me."
    ---------- 8< ----------

    Over to you :)


    Sounds good. I had *not* fiddled with the font offerings in the console before all this. I only played with the screen size.

    My console setting only offers two fonts: True-type Lucinda, or Raster.

    Changed it to Lucinda.

    This was probably the issue! Maybe this is another matter that would be useful in the guide. For proper Unicode/UTF-8 support, set your Console to use a suitable TrueType font." ?

    Yes it would but it may be even better in the online help?

    Some swedish/finnish chars:

    ,,,,,

    Looks ok here.

    Looks OK here(I think) in OpenXP(Linux).

    Some danish/norwegian chars:

    ,,,

    Looks good.

    Hmmmmm, dunno if these are OK?

    Regards,
    Martin

    --- OpenXP 5.0.37
    * Origin: Bitz-Box - Bradford - UK (2:310/31.3)
  • From August Abolins@2:221/1.58 to Martin Foster on Sat Apr 6 14:05:39 2019
    Hello Martin,

    On 06.04.19, you wrote to August Abolins:

    Some swedish/finnish chars:
    ,,Ae,ae,Oe,oe

    Looks ok here.

    Looks OK here(I think) in OpenXP(Linux).

    Some danish/norwegian chars:
    ,,,

    Looks good.

    Hmmmmm, dunno if these are OK?
    Regards, Martin
    -+- OpenXP 5.0.37


    YES!... This time looks very good in WinPoint.

    BUT.. you are using the same program to read it, and I *think* I sent that one from TB, same as another one?


    ../|ug


    --- WinPoint Beta 5 (359.1)
    * Origin: Reluctantly Revisiting Fidonet (2:221/1.58)
  • From Richard Menedetter@2:310/31 to August Abolins on Sun Apr 7 09:59:44 2019
    Hi August!

    05 Apr 2019 21:41, from August Abolins -> Martin Foster:

    @CHRS: IBMPC 2

    You have sent it as IBMPC ... definitely not UTF-8.
    This is the reason why Goated (UTF-8 capable reader) shows it as garbage.

    It should read CHRS: UTF-8 4

    Some swedish/finnish chars:
    ,,,,,

    Shown as garbage because wrong charset sent.

    CU, Ricsi

    ... Everyone wants to change the world, no one wants to change himself.
    --- GoldED+/LNX
    * Origin: The perfect woman: A nymphomaniac owning a liquor store (2:310/31)
  • From Richard Menedetter@2:310/31 to Martin Foster on Sun Apr 7 14:35:08 2019
    Hi Martin!

    06 Apr 2019 15:02, from Martin Foster -> August Abolins:

    Some swedish/finnish chars:
    ,,,,,
    Looks OK here(I think) in OpenXP(Linux).

    Because that time it was marked correctly as using UTF-8.

    CU, Ricsi

    ... If you do a favor, forget it. If you receive a favor, remember it.
    --- GoldED+/LNX
    * Origin: Fat persons are nutritional overachievers. (2:310/31)