I'm trying to write a client for an existing server (an Asterisk[snip]
"Call Manager" server) that converses in text over a bog-standard bi-directional TCP connection. I haven't done much sockets programming
and need some advice.
For the line read, I decided to try stdio, so I fdopen(,"r") using
the acquired socket. The next steps are the ones that are currently
causing me issues.
For the line read, I decided to try stdio ...
I'm trying to write a client for an existing server (an Asterisk
"Call Manager" server) that converses in text over a bog-standard bi-directional TCP connection. I haven't done much sockets
programming and need some advice. [...]
char vrm[16];
fscanf(fd,"Asterisk Call Manager/%s\r\n",vrm);
I'm trying to write a client for an existing server (an Asterisk
"Call Manager" server) that converses in text over a bog-standard bi-directional TCP connection. I haven't done much sockets programming
and need some advice.
I successfully establish my socket connection to the server using getaddrinfo() (to build the IPV4 address, socket() (to establish
an AF_INET,SOCK_STREAM socket) and connect(). The protocol requires
me to read a text line from the socket before commencing my client
requests.
For the line read, I decided to try stdio, so I fdopen(,"r") using
the acquired socket. The next steps are the ones that are currently
causing me issues.
Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:
I'm trying to write a client for an existing server (an Asterisk
"Call Manager" server) that converses in text over a bog-standard
bi-directional TCP connection. I haven't done much sockets
programming and need some advice. [...]
I've done a fair amount of simple sockets programming. I wouldn't
call myself an expert but I'm certainly not a novice. I have
several bits of advice to offer.
Use read() and write() on any sockets. Avoid stdio.
Do the appropriate calls to make socket I/O non-blocking.
At some
point you may want to get fancy and use select() or something
similar, but starting out it's probably good enough to do the
read()s and write()s in a loop, with a sleep for a hundredth of a
second in the loop so the program doesn't burn up cpu cycles.
Put all the calls to esoteric functions like getaddrinfo() in a
separate .c file, so the main body of code can be compiled with
options -std=c?? -pedantic, and only the one special .c file
needs all the special enabling to get non-standard functions.
I'm trying to write a client for an existing server (an Asterisk
"Call Manager" server) that converses in text over a bog-standard bi-directional TCP connection. I haven't done much sockets programming
and need some advice.
I successfully establish my socket connection to the server using getaddrinfo() (to build the IPV4 address, socket() (to establish
an AF_INET,SOCK_STREAM socket) and connect(). The protocol requires
me to read a text line from the socket before commencing my client
requests.
For the line read, I decided to try stdio, so I fdopen(,"r") using
the acquired socket. The next steps are the ones that are currently
causing me issues.
IF I
char vrm[16];
fscanf(fd,"Asterisk Call Manager/%s\r\n",vrm);
to extract the AMI version number that follows the text slash, my
fscanf() hangs.
BUT, if I
char buffer[1024], vrm[16];
fgets(buffer,sizeof buffer,fd);
sscanf(buffer,"Asterisk Call Manager/%s\r\n",vrm);
I manage to obtain the appropriate data. A dump of the
buffer array shows that I did indeed capture the introduction line
from the server.
So, why the difference in behaviour?
Obviously, in the fscanf() version,
I've not set something up right. Any hints as to what I've done wrong
would be greatly appreciated.
Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:
I'm trying to write a client for an existing server (an Asterisk
"Call Manager" server) that converses in text over a bog-standard
bi-directional TCP connection. I haven't done much sockets
programming and need some advice. [...]
I've done a fair amount of simple sockets programming. I wouldn't
call myself an expert but I'm certainly not a novice. I have
several bits of advice to offer.
Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:
I'm trying to write a client for an existing server (an Asterisk
"Call Manager" server) that converses in text over a bog-standard
bi-directional TCP connection. I haven't done much sockets programming
and need some advice.
I successfully establish my socket connection to the server using
getaddrinfo() (to build the IPV4 address, socket() (to establish
an AF_INET,SOCK_STREAM socket) and connect(). The protocol requires
me to read a text line from the socket before commencing my client
requests.
For the line read, I decided to try stdio, so I fdopen(,"r") using
the acquired socket. The next steps are the ones that are currently
causing me issues.
I did some tests with this with the built-in inetd echo server and the fscanf, regardless of buffering mode the stream has been set to, simply
does another read immediately after the first which returned the
data. As no more data will be received, this other read just blocks
forever. fgets happens to work because the call returns after a complete
line has been read. But AFAIK, that's not mandated behaviour and a
different implementation may well hang in fgets, too.
The basic problem is that you're doing real-time I/O using an
I/O-facility not intended to operate in real-time. If real-time
operation is desired, which is generally the case for all kinds of request-response protocol operations, you'll need to use read/write or send/recv.
Am 13.02.24 um 02:44 schrieb Lew Pitcher:[snip]
For the line read, I decided to try stdio, so I fdopen(,"r") using
the acquired socket. The next steps are the ones that are currently
causing me issues.
You do not want to use the buffered FILE API for bidirectional sockets
with message based protocols.
Typically the best is to use an appropriate library for the protocol you need.
On Tue, 13 Feb 2024 01:44:53 -0000 (UTC), Lew Pitcher wrote:
For the line read, I decided to try stdio ...
Don’t. Your code is probably reading past the end of the response and trying to get more.
Also, remember that Asterisk AMI terminates lines with CR/LF.
Sample Python code, from <https://gitlab.com/ldo/seaskirt/>, that[snip python code]
decodes responses properly:
Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:[snip]
I'm trying to write a client for an existing server (an Asterisk
"Call Manager" server) that converses in text over a bog-standard
bi-directional TCP connection. I haven't done much sockets programming
and need some advice.
I successfully establish my socket connection to the server using
getaddrinfo() (to build the IPV4 address, socket() (to establish
an AF_INET,SOCK_STREAM socket) and connect(). The protocol requires
me to read a text line from the socket before commencing my client
requests.
For the line read, I decided to try stdio, so I fdopen(,"r") using
the acquired socket. The next steps are the ones that are currently
causing me issues.
IF I
char vrm[16];
fscanf(fd,"Asterisk Call Manager/%s\r\n",vrm);
to extract the AMI version number that follows the text slash, my
fscanf() hangs.
So, why the difference in behaviour?
You've had some good advice, but no one has answered the question.
The scanf functions are not intuitive -- you really have to read the
manual carefully. The trouble you are having is that space characters
are directives and are not treated literally. The \r directs fscanf to
read one or more space characters, and it won't stop trying until it
reads a non-space character. That's more reading than you want in a
protocol stream.
Though you might be able to fix it (you could read exactly two single characters and check that they are \r and \n) it's generally better to
read a response and use sscanf if that suits the application
(The only advice I'd question is that of using non-block I/O by default.
I would be surprised if that helps you in this project, and it might
make things more complicated.)
Lew Pitcher , dans le message <uqehik$1mdue$4@dont-email.me>, a écrit :
char vrm[16];
fscanf(fd,"Asterisk Call Manager/%s\r\n",vrm);
And wham the security flaw.
On Tue, 13 Feb 2024 12:54:23 +0000, Ben Bacarisse wrote:
Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:[snip]
I'm trying to write a client for an existing server (an Asterisk
"Call Manager" server) that converses in text over a bog-standard
bi-directional TCP connection. I haven't done much sockets programming
and need some advice.
I successfully establish my socket connection to the server using
getaddrinfo() (to build the IPV4 address, socket() (to establish
an AF_INET,SOCK_STREAM socket) and connect(). The protocol requires
me to read a text line from the socket before commencing my client
requests.
For the line read, I decided to try stdio, so I fdopen(,"r") using
the acquired socket. The next steps are the ones that are currently
causing me issues.
IF I
char vrm[16];
fscanf(fd,"Asterisk Call Manager/%s\r\n",vrm);
to extract the AMI version number that follows the text slash, my
fscanf() hangs.
So, why the difference in behaviour?
You've had some good advice, but no one has answered the question.
The scanf functions are not intuitive -- you really have to read the
manual carefully. The trouble you are having is that space characters
are directives and are not treated literally. The \r directs fscanf to
read one or more space characters, and it won't stop trying until it
reads a non-space character. That's more reading than you want in a
protocol stream.
AHA! That makes sense. Thanks, Ben, for the clarification. I had forgotten that fscanf() treats \r and \n as whitespace. And, that explains why the sscanf() doen't fail; it encounters the end-of-string and treats it as
the equivalent of an EOF or non-space character. And so, doesn't hang.
Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:
IF I
char vrm[16];
fscanf(fd,"Asterisk Call Manager/%s\r\n",vrm);
to extract the AMI version number that follows the text slash, my
fscanf() hangs.
The scanf functions are not intuitive -- you really have to read the
manual carefully. The trouble you are having is that space characters
are directives and are not treated literally. The \r directs fscanf to
read one or more space characters, and it won't stop trying until it
reads a non-space character.
On Tue, 13 Feb 2024 12:54:23 +0000, Ben Bacarisse wrote:
Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:
(The only advice I'd question is that of using non-block I/O by default.
I would be surprised if that helps you in this project, and it might
make things more complicated.)
I've done less programming with non-blocking I/O than I've done with
sockets. I have heard the advice to use non-blocking I/O, and will
consider it, but I don't want to take on too much in one shot.
But, this is an experiment (acknowledged as an experiment) under
laboratory conditions
On Tue, 13 Feb 2024 16:12:01 +0000, Rainer Weikusat wrote:
Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:[snip]
On Tue, 13 Feb 2024 12:54:23 +0000, Ben Bacarisse wrote:
Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:
[...]
(The only advice I'd question is that of using non-block I/O by default. >>>> I would be surprised if that helps you in this project, and it might
make things more complicated.)
I've done less programming with non-blocking I/O than I've done with
sockets. I have heard the advice to use non-blocking I/O, and will
consider it, but I don't want to take on too much in one shot.
The idea behind non-blocking I/O is usually that of a single-threaded
program (or a single thread of a program) which needs to handle
real-time inputÂą on more than one file descriptor and thus, cannot block
waiting for input on one of them as data might arrive at one of the
others first.
A somewhat simpler use case is I/O on only one file descriptor but with[snip]
a need to impose timeouts to cope with unreliable communication
partners
For this mini-project, both client and server live on the same machine,
and communicate through the loopback ("localhost"). There won't be
any significant latency issues or other communications interference.
I'm only concerned with writing the client, and it will only interact
over one channel in a simple request/reply type protocol. There doesn't
seem to be a pressing need for nonblocking I/O.
But, as it has been suggested here, I will look into nonblocking I/O.
Perhaps I'm missing something that others have seen.
Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:[snip]
On Tue, 13 Feb 2024 12:54:23 +0000, Ben Bacarisse wrote:
Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:
[...]
(The only advice I'd question is that of using non-block I/O by default. >>> I would be surprised if that helps you in this project, and it might
make things more complicated.)
I've done less programming with non-blocking I/O than I've done with
sockets. I have heard the advice to use non-blocking I/O, and will
consider it, but I don't want to take on too much in one shot.
The idea behind non-blocking I/O is usually that of a single-threaded
program (or a single thread of a program) which needs to handle
real-time inputÂą on more than one file descriptor and thus, cannot block waiting for input on one of them as data might arrive at one of the
others first.
A somewhat simpler use case is I/O on only one file descriptor but with[snip]
a need to impose timeouts to cope with unreliable communication
partners
Lew Pitcher , dans le message <uqfsgu$23ni8$6@dont-email.me>, a écrit :
But, this is an experiment (acknowledged as an experiment) under
laboratory conditions
You posted it on Usenet.
However, outside of a couple of out of maintenance, 3rd-party projects
from Asterisk users trying to do the same thing as I am, there's /no/ official "library for the protocol" available, that I can find.
On Tue, 13 Feb 2024 13:45:36 -0000 (UTC), Lew Pitcher wrote:
However, outside of a couple of out of maintenance, 3rd-party projects
from Asterisk users trying to do the same thing as I am, there's /no/
official "library for the protocol" available, that I can find.
My Python library, Seaskirt (link posted elsewhere) is the only one I know
of that covers AMI, AGI, Async-AGI, ARI and even the console interface. It offers both synchronous and asynchronous (async/await) versions of all the main API classes, and even allows for SSL/TLS connections where Asterisk
will accept these.
On Tue, 13 Feb 2024 12:54:23 +0000, Ben Bacarisse wrote:
(The only advice I'd question is that of using non-block I/O by default.
I would be surprised if that helps you in this project, and it might
make things more complicated.)
This is why we have the “async” paradigm, a.k.a. the resurgence of coroutines.
(The only advice I'd question is that of using non-block I/O by default.
I would be surprised if that helps you in this project, and it might
make things more complicated.)
Lawrence D'Oliveiro <ldo@nz.invalid> writes:
This is why we have the “async” paradigm, a.k.a. the resurgence of
coroutines.
From coroutines, it's usually one step until somebody again reimplements cooperative userspace threading.
At the base of this is an I/O completion object I've called a 'want' (as
it describes something a program wants). This object contains a file
handle/ file descriptor, an event the program wants (input available or output possible) and a continuation closure which should be invoked
after the even has occurred.
At the beginning of each iteration of the main event handling loop,
there's a set of current wants and a poll call is made waiting for any
of the I/O events indicated by these.
On Tue, 13 Feb 2024 21:09:45 +0000, Rainer Weikusat wrote:
Lawrence D'Oliveiro <ldo@nz.invalid> writes:
This is why we have the “async” paradigm, a.k.a. the resurgence of
coroutines.
From coroutines, it's usually one step until somebody again reimplements
cooperative userspace threading.
That’s what “async” is all about. In Python asyncio, they use the term “tasks” for these cooperative schedulable entities. They are wrappers around Python coroutine objects.
At the base of this is an I/O completion object I've called a 'want' (as
it describes something a program wants). This object contains a file
handle/ file descriptor, an event the program wants (input available or
output possible) and a continuation closure which should be invoked
after the even has occurred.
In the Python asyncio library, there is this concept of a “future”. In JavaScript, they call it a “promise”. This is not tied in any way to particular open files or anything else: think of it as a box, initially empty, with a sign above it saying “watch this space”. At some point, something should happen, and tasks can block on waiting for it. When something appears in the box, they are woken up and can retrieve a copy of the contents.
At the beginning of each iteration of the main event handling loop,
there's a set of current wants and a poll call is made waiting for any
of the I/O events indicated by these.
Not just I/O events, you also want timers as well.
That's an overgeneralisation of a useful concept people
keep reinventing ...
On Tue, 13 Feb 2024 21:57:54 +0000, Rainer Weikusat wrote:
That's an overgeneralisation of a useful concept people
keep reinventing ...
If it really was a generalization, then there would be nothing to “reinvent”. You only need to “reinvent” something if a prior concept was
not general enough.
TL;DR: “overgeneralization ... you keep using that word ... I do not think it means what you think it means”.
On Tue, 13 Feb 2024 21:57:54 +0000, Rainer Weikusat wrote:
That's an overgeneralisation of a useful concept people
keep reinventing ...
If it really was a generalization, then there would be nothing to “reinvent”. You only need to “reinvent” something if a prior concept was
not general enough.
The idea behind non-blocking I/O is usually that of a single-threaded
program (or a single thread of a program) which needs to handle
real-time inputą on more than one file descriptor and thus, cannot block waiting for input on one of them as data might arrive at one of the
others first.
POSIX threads are useful for performance, but it is a mistake to think
they help for I/O. And that mistake is pervasive enough that we must be careful not to amplify it.
Indeed, I did. And your point is?
[..][...]
Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:
On Tue, 13 Feb 2024 12:54:23 +0000, Ben Bacarisse wrote:
[...](The only advice I'd question is that of using non-block I/O by
default. I would be surprised if that helps you in this project,
and it might make things more complicated.)
I've done less programming with non-blocking I/O than I've done
with sockets. I have heard the advice to use non-blocking I/O,
and will consider it, but I don't want to take on too much in one
shot.
For this mini-project, both client and server live on the same
machine, and communicate through the loopback ("localhost").
There won't be any significant latency issues or other
communications interference.
I'm only concerned with writing the client, and it will only
interact over one channel in a simple request/reply type protocol.
There doesn't seem to be a pressing need for nonblocking I/O.
But, as it has been suggested here, I will look into nonblocking
I/O. Perhaps I'm missing something that others have seen.
I guess I should expand on the idea that using non-blocking I/O
is a good idea.
First, my comment was offered as pragmatic advice following many
years of experience both with networking and with socket-level
programming. It is generic advice, given without taking into
account any of the specifics of your project.
Assuming you do go ahead with using blocking I/O, and get things
working, there is still a fair chance that at some point down the
road (probably after the application has been deployed) the program
will be the victim of a network hiccup and go catatonic.
If and when that happens: one, it will be frustrating; two, the problem will be non-reproducible; and three, you won't have any tools to
help diagnose what's going on to cause the problem, because all the significant events are happening inside one of the blocking system
calls. Conversely, if all the socket I/O is non-blocking, the code
can be instrumented (whether before the fact or after) with various
sorts of logging, and the log information can be examined to see
what's going on.
Final point: probably the most important lesson here is that we
may expect that network system calls are going to fail, and should
program accordingly. When working with files it's easy to get
into the habit of thinking the calls will never fail, because they
almost never do. Network system calls are different. They might
not fail all the time, but they *are* going to fail, and it's
better to anticipate that, and program accordingly. Using
non-blocking I/O makes that easier, partly because "failures"
happen more often in the form of "operation would block", so one
gets into the habit of writing code that handles different kinds
of error returns. This means a little more work at the outset,
but ultimately less work (we hope!) overall.
Sysop: | Keyop |
---|---|
Location: | Huddersfield, West Yorkshire, UK |
Users: | 299 |
Nodes: | 16 (2 / 14) |
Uptime: | 76:52:13 |
Calls: | 6,695 |
Calls today: | 5 |
Files: | 12,228 |
Messages: | 5,347,344 |
Posted today: | 2 |