On 2023-01-09, Muttley@dastardlyhq.com <Muttley@dastardlyhq.com> wrote:
Has anyone ever used popen() with the r+ option on MacOS? Writing to the file
Coprocesses are tricky; it's easy to get into a deadlock if you just
read and write with one thread, without multiplexing the I/O using
poll or select.
descriptor works fine (the output goes to the printer queue) but any attempt >to
read the response from CUPS from the file descriptor simply hangs. eg:
if ((pfile = popen("lp","r+")))
{
... write data to printer ...
Did you fflush(pfile) here? Maybe all the data didn't go.
You would think that a simple half-duplex situation like this would
avoid deadlock, since lp should read your entire request until the end,
and only then produce a response.
However, if some trailing part of your request is sitting in a buffer,
such that lp is waiting for that last bit before replying, then you have
a deadlock.
(Note that ISO C itself requires a flush operation when switching from
output to input on a bidirectional stream, as a matter of well-defined >behavior. When switching from input to output, a file positioning
operation is required (regardless of whether the stream supports
positioning; seeking zero bytes from the current position with
fseek should do the trick). I think the main reason for this is that
Has anyone ever used popen() with the r+ option on MacOS? Writing to the file
descriptor works fine (the output goes to the printer queue) but any attempt to
read the response from CUPS from the file descriptor simply hangs. eg:
if ((pfile = popen("lp","r+")))
{
... write data to printer ...
fgets(str,sizeof(str),pfile); <- hangs here
}
However, if some trailing part of your request is sitting in a buffer,
such that lp is waiting for that last bit before replying, then you have
a deadlock.
I would guess it is not enough: lp is probably waiting for EOF, and there is >>no way to close only half of a popen stream.Surely if you're going to implement r+ functionality you'd make sure you flushed the write buffer before the close so this issue doesn't occur?
Kaz Kylheku , dans le message <20230109030226.641@kylheku.com>, a
écrit :
However, if some trailing part of your request is sitting in a buffer,
such that lp is waiting for that last bit before replying, then you have
a deadlock.
I would guess it is not enough: lp is probably waiting for EOF, and there is >no way to close only half of a popen stream.
Muttley@dastardlyhq.com, dans le message <tphdfa$1016$1@gioia.aioe.org>,
a écrit :
I would guess it is not enough: lp is probably waiting for EOF, and there is >>>no way to close only half of a popen stream.Surely if you're going to implement r+ functionality you'd make sure you
flushed the write buffer before the close so this issue doesn't occur?
The issue is not about an unflushed buffer, it is about a stream that is >still open and therefore could receive more data. Consider this:
f = popen("some_command", "r+");
fprintf(f, "Hello.\n");
fflush(f);
fgets(line, f);
How does some_command know you are done talking after "Hello."?
Has anyone ever used popen() with the r+ option on MacOS? Writing to the file >descriptor works fine (the output goes to the printer queue) but any attempt to
read the response from CUPS from the file descriptor simply hangs. eg:
if ((pfile = popen("lp","r+")))
{
... write data to printer ...
fgets(str,sizeof(str),pfile); <- hangs here
}
It also hangs if I try while(!feof(pfile)) or read(fileno(pfile),....)
lp is producing output and I can redirect to a file , eg popen("lp > myfile",..)
I assume this is some kind of buffer flush issue but I can't find anything on >google about it.
Muttley@dastardlyhq.com writes:
Has anyone ever used popen() with the r+ option on MacOS? Writing to the file >>descriptor works fine (the output goes to the printer queue) but any attempt >to
read the response from CUPS from the file descriptor simply hangs. eg:
if ((pfile = popen("lp","r+")))
{
... write data to printer ...
fgets(str,sizeof(str),pfile); <- hangs here
}
It also hangs if I try while(!feof(pfile)) or read(fileno(pfile),....)
lp is producing output and I can redirect to a file , eg popen("lp > >myfile",..)
I assume this is some kind of buffer flush issue but I can't find anything on >>google about it.
The 'p' in "popen" stands for "pipe". Pipes in unix are unidirectional.
From https://pubs.opengroup.org/onlinepubs/9699919799/functions/popen.html
"Since open files are shared, a mode r command can be used as
an input filter and a mode w command as an output filter."
and
"The behavior of popen() is specified for values of mode of r and w."
You can't do what you want to do with popen.
Muttley@dastardlyhq.com, dans le message <tphdfa$1016$1@gioia.aioe.org>,
a crit:
I would guess it is not enough: lp is probably waiting for EOF, and there is >>>no way to close only half of a popen stream.Surely if you're going to implement r+ functionality you'd make sure you
flushed the write buffer before the close so this issue doesn't occur?
The issue is not about an unflushed buffer, it is about a stream that is >still open and therefore could receive more data. Consider this:
f = popen("some_command", "r+");
fprintf(f, "Hello.\n");
fflush(f);
fgets(line, f);
How does some_command know you are done talking after "Hello."?
In article <63bc3c2e$0$3184$426a74cc@news.free.fr>,
Nicolas George <nicolas$george@salle-s.org> wrote:
Muttley@dastardlyhq.com, dans le message <tphdfa$1016$1@gioia.aioe.org>,
a crit:
I would guess it is not enough: lp is probably waiting for EOF, and there isSurely if you're going to implement r+ functionality you'd make sure you >>> flushed the write buffer before the close so this issue doesn't occur?
no way to close only half of a popen stream.
The issue is not about an unflushed buffer, it is about a stream that is >>still open and therefore could receive more data. Consider this:
f = popen("some_command", "r+");
fprintf(f, "Hello.\n");
fflush(f);
fgets(line, f);
How does some_command know you are done talking after "Hello."?
Right. The classic example of this is "sort". sort cannot possibly
generate any output until it is certain that no more input is coming.
I assume "lp" behaves likewise. It looks to me that the best solution is
as given by OP: redirect output to a file (in the shell command passed to >popen()), then parse that file after doing pclose().
I assume "lp" behaves likewise. It looks to me that the best solution is >>as given by OP: redirect output to a file (in the shell command passed to >>popen()), then parse that file after doing pclose().
Or bite the bullet and use fork/exec and set up stdin and stdout
for the child appropriately with two stdio streams, one for read
and one for write.
^^^^^^Has anyone ever used popen() with the r+ option on MacOS?
The 'p' in "popen" stands for "pipe". Pipes in unix are unidirectional.
From https://pubs.opengroup.org/onlinepubs/9699919799/functions/popen.html
"Since open files are shared, a mode r command can be used as
an input filter and a mode w command as an output filter."
and
"The behavior of popen() is specified for values of mode of r and w."
You can't do what you want to do with popen.
"The behavior of popen() is specified for values of mode of r and w."
You can't do what you want to do with popen.
From the MacOS popen() man page:
reading or writing, not both. Because popen() is now implemented using a
bidirectional pipe, the mode argument may request a bidirectional data
flow. The mode argument is a pointer to a null-terminated string which
must be `r' for reading, `w' for writing, or `r+' for reading and
writing.
HTH.
gazelle@shell.xmission.com (Kenny McCormack) writes:
In article <63bc3c2e$0$3184$426a74cc@news.free.fr>,
Nicolas George <nicolas$george@salle-s.org> wrote: >>>Muttley@dastardlyhq.com, dans le message <tphdfa$1016$1@gioia.aioe.org>,
a crit:
I would guess it is not enough: lp is probably waiting for EOF, and there >isSurely if you're going to implement r+ functionality you'd make sure you >>>> flushed the write buffer before the close so this issue doesn't occur?
no way to close only half of a popen stream.
The issue is not about an unflushed buffer, it is about a stream that is >>>still open and therefore could receive more data. Consider this:
f = popen("some_command", "r+");
fprintf(f, "Hello.\n");
fflush(f);
fgets(line, f);
How does some_command know you are done talking after "Hello."?
Right. The classic example of this is "sort". sort cannot possibly >>generate any output until it is certain that no more input is coming.
I assume "lp" behaves likewise. It looks to me that the best solution is >>as given by OP: redirect output to a file (in the shell command passed to >>popen()), then parse that file after doing pclose().
Or bite the bullet and use fork/exec and set up stdin and stdout
for the child appropriately with two stdio streams, one for read
and one for write.
Scott Lurndal, dans le message <a8XuL.575788$GNG9.424370@fx18.iad>, a
écrit :
^^^^^^Has anyone ever used popen() with the r+ option on MacOS?
The 'p' in "popen" stands for "pipe". Pipes in unix are unidirectional.
From https://pubs.opengroup.org/onlinepubs/9699919799/functions/popen.html >>
"Since open files are shared, a mode r command can be used as
an input filter and a mode w command as an output filter."
and
"The behavior of popen() is specified for values of mode of r and w."
You can't do what you want to do with popen.
Unfortunately, they can: >https://developer.apple.com/library/archive/documentation/System/Conceptual/Man
Pages_iPhoneOS/man3/popen.3.html
“or `r+' for reading and writ-ing.â€
Blame on Apple for adding this extension without documenting how to >half-close the stream. They could have returned two FILE*, or documented the >use of shutdown(fileno(p)).
Lucky guess: try write(fileno(p), "", 0).
In article <tphh1a$q5j$1@gioia.aioe.org>, <Muttley@dastardlyhq.com> wrote: >....
"The behavior of popen() is specified for values of mode of r and w."
You can't do what you want to do with popen.
From the MacOS popen() man page:
reading or writing, not both. Because popen() is now implemented using a
bidirectional pipe, the mode argument may request a bidirectional data >> flow. The mode argument is a pointer to a null-terminated string which >> must be `r' for reading, `w' for writing, or `r+' for reading and
writing.
HTH.
When answering questions like this, it is important to be clear as to
whether you are answering from a "POSIX standards" POV vs. from the OP's >actual use case situation. The POSIX standards answer is, as stated above, >you can't do that (and now go home and bother us no more). However, OP >stated that he was using a Mac and that Mac documentation explicitly allows >this (with, of course, the caveat that it is an "extension").
Note, BTW, that I don't currently have access to a Mac to check this, but I >am assuming that the reports posted so far in this thread are accurate.
Interestingly, the Linux documentation that just checked makes it clear
that only "r" and "w" are valid. Even though, Linux certainly has
bidirectional pipes available (and, in fact, I think it is common knowledge >that pipe is implemented with socketpair), so you'd think Linux would have
a bidirectional version of popen() available, but apparently such is not
the case.
It does look like Apples implementation is
broken, or more politely - incomplete. There seems to be no way to close one half of the pipe to force the child process to complete its stdout.
On Mon, 09 Jan 2023 18:44:08 GMT
scott@slp53.sl.home (Scott Lurndal) wrote:
gazelle@shell.xmission.com (Kenny McCormack) writes:
In article <63bc3c2e$0$3184$426a74cc@news.free.fr>,
Nicolas George <nicolas$george@salle-s.org> wrote: >>>>Muttley@dastardlyhq.com, dans le message <tphdfa$1016$1@gioia.aioe.org>, >>>> a crit:
The issue is not about an unflushed buffer, it is about a stream that is >>>>still open and therefore could receive more data. Consider this:I would guess it is not enough: lp is probably waiting for EOF, and there >>isSurely if you're going to implement r+ functionality you'd make sure you >>>>> flushed the write buffer before the close so this issue doesn't occur? >>>>
no way to close only half of a popen stream.
f = popen("some_command", "r+");
fprintf(f, "Hello.\n");
fflush(f);
fgets(line, f);
How does some_command know you are done talking after "Hello."?
Right. The classic example of this is "sort". sort cannot possibly >>>generate any output until it is certain that no more input is coming.
I assume "lp" behaves likewise. It looks to me that the best solution is >>>as given by OP: redirect output to a file (in the shell command passed to >>>popen()), then parse that file after doing pclose().
Or bite the bullet and use fork/exec and set up stdin and stdout
for the child appropriately with two stdio streams, one for read
and one for write.
Urgh. Yes I could but thats what one wants to avoid by using popen(). Its such >a faff to setup plus all the error checking whereas popen() is 1 line of code.
Muttley@dastardlyhq.com writes:
Or bite the bullet and use fork/exec and set up stdin and stdout
for the child appropriately with two stdio streams, one for read
and one for write.
Urgh. Yes I could but thats what one wants to avoid by using popen(). Its >such
a faff to setup plus all the error checking whereas popen() is 1 line of code.
Does OSX have a printer API that allows one to submit a job (rather than >using system(3) or popen(3))?
Muttley wrote:
It does look like Apples implementation is
broken, or more politely - incomplete. There seems to be no way to close one >> half of the pipe to force the child process to complete its stdout.
If popen(..., "r+") is only usable with commands that don't wait for
EOF, then execute a command that doesn't wait for EOF.
You were trying to execute "lp". You could instead execute something
like:
sed '/^=EnD=oF=InPuT=$/{d;q;}' | lp
and write that sentinel string (plus a newline) to the stream after
the data to be printed. To reduce the risk of the sentinel string
occurring in the data to be printed, you could include some rarely-used >control characters.
On Mon, 09 Jan 2023 16:19:18 GMT
scott@slp53.sl.home (Scott Lurndal) wrote:
You can't do what you want to do with popen.
From the MacOS popen() man page:
reading or writing, not both. Because popen() is now implemented using a
bidirectional pipe, the mode argument may request a bidirectional data
flow. The mode argument is a pointer to a null-terminated string which
must be `r' for reading, `w' for writing, or `r+' for reading and
writing.
Kaz Kylheku , dans le message <20230109030226.641@kylheku.com>, a
écrit :
However, if some trailing part of your request is sitting in a buffer,
such that lp is waiting for that last bit before replying, then you have
a deadlock.
I would guess it is not enough: lp is probably waiting for EOF, and there is no way to close only half of a popen stream.
And, so, continuing in the same vein as my previous posting: we have had bidirectional pipes in Unix for decades. The socketpair function creates
two descriptors and places them into an array, just like pipe. But
they are a pair of connected sockets: writing to one makes data appear
in the other and vice versa.
So there are two possibilities:
1. MacOS invented a new kind of bidirectional pipe that is other than a
socket pair.
2. The bidirectional pipe is actually a socket pair.
I'm fairly sure it's a true bidirectional pipe, not a socket pair.
On 2023-01-09, Nicolas George <nicolas$george@salle-s.org> wrote:
Kaz Kylheku , dans le message <20230109030226.641@kylheku.com>, a
écrit :
However, if some trailing part of your request is sitting in a buffer,
such that lp is waiting for that last bit before replying, then you have >>> a deadlock.
I would guess it is not enough: lp is probably waiting for EOF, and there is >> no way to close only half of a popen stream.
That's a great observation.
Now the "r+" mode is not portable to begin with.
The POSIX model of standard I/O streams is such that a stream has one >underlying descriptor, accessible via fileno.
Suppose an implementation implements a bidirectional popen using only
one descriptor, such that the descriptor returned by fileno is all there
is; that object handles both data transfer directions.
Such an implementation is quite possibly using the socketpair function
to creae the bidirectional descriptor.
And so if socketpair is being used, then it should be possible to use >shutdown on the descriptor to do a half-close.
As in:
// write data to lp ...
// send everything to socket
fflush(pipe_stream);
// close in writing direction
if (shutdown(fileno(pipe_stream), SHUT_WR) != 0)
{
// didn't work: not a socket?
}
It's worth a try.
Kaz Kylheku wrote:
2. The bidirectional pipe is actually a socket pair.
This would not conform to POSIX, as fstat() would show S_ISSOCK(st_mode)
true instead of S_ISFIFO(st_mode).
| Sysop: | Keyop |
|---|---|
| Location: | Huddersfield, West Yorkshire, UK |
| Users: | 307 |
| Nodes: | 16 (2 / 14) |
| Uptime: | 83:46:44 |
| Calls: | 6,921 |
| Calls today: | 6 |
| Files: | 12,382 |
| Messages: | 5,433,379 |