• issue with popen and reading from stdout

    From [via djgpp@delorie.com]" @21:1/5 to All on Sat Mar 27 16:39:08 2021
    The code is basically:

    p = popen(cmdline, "rb");
    while ((n = fread(buf, 1, BUFLEN, p)) > 0) {
    fwrite(buf, 1, n, t);
    }
    pclose(p);

    .. where cmdline is like "unrar p -inul ponylips.rar". The "p" makes
    unrar to extract to stdout, -inul makes unrar to emit no messages.

    The issue is: Whatever I do, including inserting calls like setmode(fileno(stdout),O_BINARY), extracted file has 0x0a -> 0x0d 0x0a
    changes and corrupted. For what it's worth, manually running that cmd
    on the console itself, like "unrar p -inul ponylips.rar > ponylips.mod"
    gives the same result.

    Is this a thing with djgpp, or possibly something to do with unrar for
    dos? (The unrar version in question is v3.80 and is downloadable from https://www.rarlab.com/rar_add.htm which seems to be built using djppp.)

    --
    O.S.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From [via djgpp@delorie.com]" @21:1/5 to All on Sat Mar 27 17:02:13 2021
    From: "Ozkan Sezer (sezeroz@gmail.com) [via djgpp@delorie.com]" <djgpp@delorie.com>
    Date: Sat, 27 Mar 2021 16:39:08 +0300

    The code is basically:

    p = popen(cmdline, "rb");
    while ((n = fread(buf, 1, BUFLEN, p)) > 0) {
    fwrite(buf, 1, n, t);
    }
    pclose(p);

    ... where cmdline is like "unrar p -inul ponylips.rar". The "p" makes
    unrar to extract to stdout, -inul makes unrar to emit no messages.

    The issue is: Whatever I do, including inserting calls like setmode(fileno(stdout),O_BINARY), extracted file has 0x0a -> 0x0d 0x0a changes and corrupted. For what it's worth, manually running that cmd
    on the console itself, like "unrar p -inul ponylips.rar > ponylips.mod"
    gives the same result.

    I don't think I understand what you mean by "extracted file has 0x0a
    0x0d 0x0a changes and corrupted".

    The above program's expected effect is to read what "unrar -p" emits
    without any EOL conversions. Programs that run on MS-DOS/Windows
    usually output CRLF end-of-line sequences (i.e. 0x0d + 0x0a), so the
    program you show should:

    . read these CRLF EOLs without any conversion
    . write each line to stdout

    Whether 'fwrite' add another 0x0d to each line depends on the mode of
    the stream 't'; if you open it in binary mode, I would expect to see
    the same CRLF EOLs as what "unrar -p" emits. If 't' is in text mode,
    you will see each line end with 0x0d 0x0d 0x0a, i.e. 2 CR characters
    followed by a single newline.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From [via djgpp@delorie.com]" @21:1/5 to djgpp@delorie.com on Sat Mar 27 17:18:20 2021
    On 3/27/21, Eli Zaretskii (eliz@gnu.org) [via djgpp@delorie.com] <djgpp@delorie.com> wrote:
    I don't think I understand what you mean by "extracted file has 0x0a
    0x0d 0x0a changes and corrupted".

    The above program's expected effect is to read what "unrar -p" emits
    without any EOL conversions. Programs that run on MS-DOS/Windows
    usually output CRLF end-of-line sequences (i.e. 0x0d + 0x0a), so the
    program you show should:

    . read these CRLF EOLs without any conversion
    . write each line to stdout

    OK, so it's possibly an issue with unrar itself

    Whether 'fwrite' add another 0x0d to each line depends on the mode of
    the stream 't'; if you open it in binary mode, I would expect to see
    the same CRLF EOLs as what "unrar -p" emits. If 't' is in text mode,
    you will see each line end with 0x0d 0x0d 0x0a, i.e. 2 CR characters
    followed by a single newline.

    It's opened with "w+b" mode, like:
    fd = mkstemp(*filename);
    t = fdopen(fd, "w+b");

    --
    O.S.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From [via djgpp@delorie.com]" @21:1/5 to djgpp@delorie.com on Sat Mar 27 17:48:25 2021
    On 3/27/21, Eli Zaretskii (eliz@gnu.org) [via djgpp@delorie.com] <djgpp@delorie.com> wrote:
    From: "Ozkan Sezer (sezeroz@gmail.com) [via djgpp@delorie.com]"
    <djgpp@delorie.com>
    Date: Sat, 27 Mar 2021 17:18:20 +0300

    The above program's expected effect is to read what "unrar -p" emits
    without any EOL conversions. Programs that run on MS-DOS/Windows
    usually output CRLF end-of-line sequences (i.e. 0x0d + 0x0a), so the
    program you show should:

    . read these CRLF EOLs without any conversion
    . write each line to stdout

    OK, so it's possibly an issue with unrar itself

    You expected unrar to produce lines with a single 0x0a at the end of
    each line? That's extremely unusual with DOS/Windows programs.

    The archived file is binary not a text, although unrar might have screwed up.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From [via djgpp@delorie.com]" @21:1/5 to djgpp@delorie.com on Sat Mar 27 17:51:51 2021
    On 3/27/21, Eli Zaretskii (eliz@gnu.org) [via djgpp@delorie.com] <djgpp@delorie.com> wrote:
    From: "Ozkan Sezer (sezeroz@gmail.com) [via djgpp@delorie.com]"
    <djgpp@delorie.com>
    Date: Sat, 27 Mar 2021 17:18:20 +0300

    The above program's expected effect is to read what "unrar -p" emits
    without any EOL conversions. Programs that run on MS-DOS/Windows
    usually output CRLF end-of-line sequences (i.e. 0x0d + 0x0a), so the
    program you show should:

    . read these CRLF EOLs without any conversion
    . write each line to stdout

    OK, so it's possibly an issue with unrar itself

    You expected unrar to produce lines with a single 0x0a at the end of
    each line? That's extremely unusual with DOS/Windows programs.

    The archived file is binary not a text, although unrar might have screwed up.

    EDIT: For what it's worth, windows version doesn't have the issue.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From [via djgpp@delorie.com]" @21:1/5 to All on Sat Mar 27 18:16:26 2021
    From: "Ozkan Sezer (sezeroz@gmail.com) [via djgpp@delorie.com]" <djgpp@delorie.com>
    Date: Sat, 27 Mar 2021 17:48:25 +0300

    OK, so it's possibly an issue with unrar itself

    You expected unrar to produce lines with a single 0x0a at the end of
    each line? That's extremely unusual with DOS/Windows programs.

    The archived file is binary not a text, although unrar might have screwed up.

    If "unrar -p" is supposed to output the archived file to stdout, then
    unrar should do that in binary mode; if it doesn't, that's a bug
    (possibly in the DOS port).

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From [via djgpp@delorie.com]" @21:1/5 to djgpp@delorie.com on Sat Mar 27 18:32:06 2021
    On 3/27/21, Eli Zaretskii (eliz@gnu.org) [via djgpp@delorie.com] <djgpp@delorie.com> wrote:
    The archived file is binary not a text, although unrar might have screwed
    up.

    If "unrar -p" is supposed to output the archived file to stdout, then
    unrar should do that in binary mode; if it doesn't, that's a bug
    (possibly in the DOS port).

    My thoughts exactly.

    Thanks for taking time to respond though all this.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From RayeR@21:1/5 to All on Fri Apr 2 13:28:51 2021
    Yes I met some apps originally made for linux that used fopen ("r" instead of "rb" / or "wb" that caused various issues and needed to patch them for windows/dos...

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)