• PPP Flag Sequence

    From Federico Frigo@21:1/5 to All on Tue Feb 2 03:13:48 2021
    Hi there, I'm trying to connect my RaspberryPi to an FTDI chip. They communicate over serial, and they use PPP as underlying protocol. I'm able to make them communicate, but the FTDI implementation is quite buggy, so I'm having some trubles with the Flag
    Sequence, since the FTDI chip wants it at the beginning of every package that it receives.

    I'm using the pppd package, i tried to modify it to achive my needs but it was a failure, and now I'm wondering if I have to modify the Linux kernel to do so or if I have to simply modify the pppd package.

    Is there someone who can point me to the right direction?

    Thanks!

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Rob van der Putten@21:1/5 to Federico Frigo on Tue Feb 2 13:05:27 2021
    Hi there


    On 02/02/2021 12:13, Federico Frigo wrote:

    Hi there, I'm trying to connect my RaspberryPi to an FTDI chip. They communicate over serial, and they use PPP as underlying protocol. I'm able
    to make them communicate, but the FTDI implementation is quite buggy, so I'm having some trubles with the Flag Sequence, since the FTDI chip wants it at the beginning of every package that it receives.

    I'm using the pppd package, i tried to modify it to achive my needs but it was a failure, and now I'm wondering if I have to modify the Linux kernel
    to do so or if I have to simply modify the pppd package.

    Is there someone who can point me to the right direction?

    This way to fuzzy.
    Which FTDI chip do you mean?
    And as far as I know the Flag Sequence (01111110 / 0x7e / '~') is always
    there. You can disable or enable Van Jacobson TCP/IP Header Compression
    though.
    AFAIK the Raspberry Pi serial port doesn't have RTS and CTS. Without
    those you may lose data at high speeds.
    And USB to serial converters can be quite buggy. Some may drop an odd
    byte here and there. That will mess up your PPP.


    Regards,
    Rob

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Rob van der Putten@21:1/5 to Federico Frigo on Tue Feb 2 15:11:37 2021
    Hi there


    On 02/02/2021 14:29, Federico Frigo wrote:

    Il giorno martedì 2 febbraio 2021 alle 13:20:05 UTC+1 Rob van der Putten ha scritto:
    Hi there
    On 02/02/2021 12:13, Federico Frigo wrote:

    Hi there, I'm trying to connect my RaspberryPi to an FTDI chip. They
    communicate over serial, and they use PPP as underlying protocol. I'm able >>> to make them communicate, but the FTDI implementation is quite buggy, so I'm
    having some trubles with the Flag Sequence, since the FTDI chip wants it at >>> the beginning of every package that it receives.

    I'm using the pppd package, i tried to modify it to achive my needs but it >>> was a failure, and now I'm wondering if I have to modify the Linux kernel >>> to do so or if I have to simply modify the pppd package.

    Is there someone who can point me to the right direction?
    This way to fuzzy.
    Which FTDI chip do you mean?
    And as far as I know the Flag Sequence (01111110 / 0x7e / '~') is always
    there. You can disable or enable Van Jacobson TCP/IP Header Compression
    though.
    AFAIK the Raspberry Pi serial port doesn't have RTS and CTS. Without
    those you may lose data at high speeds.
    And USB to serial converters can be quite buggy. Some may drop an odd
    byte here and there. That will mess up your PPP.


    Regards,
    Rob
    I know for sure that not all ppp packets are sent with the leading '~',
    in fact I found this in the linux kernel on RaspberryPi GitHub repo: https://github.com/raspberrypi/linux/blob/rpi-5.10.y/drivers/net/ppp/ppp_async.c#L560

    As you can see, on timing basis, the leading flag sequence is not
    inserted. So maybe I have to patch this file to make it put the '~'
    on each packet no matter what, ain't it?

    Without wading through the entire source code, I expect this to be ~Packet~Packet~, vs ~Packet~~Packet~. Both are OK. ~PacketPacket~ is
    not! Synchronous PPP will send ~~~~~ when idle, asynchronous PPP will not. Linux has a freaky optimization in the LCP ping. It will only send a LCP
    ping when there hasn't been any other data for some time.
    Furthermore, the number of config requests in one LCP packet may vary.
    A PPP implementation without '~' at packet boundaries seems broken to me
    (See RFC). Are you this is what is happening? Make a raw data dump.

    Again, RasPi serial ports are broken. USB-Serial converters are broken.
    They do drop bytes (including '~'). Use decent serial hardware or use
    Ethernet for TCP/IP.

    And you still haven't explained what you are trying to do. Draw some
    ASCII art.


    Regards,
    Rob

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Federico Frigo@21:1/5 to All on Tue Feb 2 05:29:33 2021
    Il giorno martedì 2 febbraio 2021 alle 13:20:05 UTC+1 Rob van der Putten ha scritto:
    Hi there
    On 02/02/2021 12:13, Federico Frigo wrote:

    Hi there, I'm trying to connect my RaspberryPi to an FTDI chip. They communicate over serial, and they use PPP as underlying protocol. I'm able to make them communicate, but the FTDI implementation is quite buggy, so I'm
    having some trubles with the Flag Sequence, since the FTDI chip wants it at
    the beginning of every package that it receives.

    I'm using the pppd package, i tried to modify it to achive my needs but it was a failure, and now I'm wondering if I have to modify the Linux kernel to do so or if I have to simply modify the pppd package.

    Is there someone who can point me to the right direction?
    This way to fuzzy.
    Which FTDI chip do you mean?
    And as far as I know the Flag Sequence (01111110 / 0x7e / '~') is always there. You can disable or enable Van Jacobson TCP/IP Header Compression though.
    AFAIK the Raspberry Pi serial port doesn't have RTS and CTS. Without
    those you may lose data at high speeds.
    And USB to serial converters can be quite buggy. Some may drop an odd
    byte here and there. That will mess up your PPP.


    Regards,
    Rob
    I know for sure that not all ppp packets are sent with the leading '~', in fact I found this in the linux kernel on RaspberryPi GitHub repo: https://github.com/raspberrypi/linux/blob/rpi-5.10.y/drivers/net/ppp/ppp_async.c#L560

    As you can see, on timing basis, the leading flag sequence is not inserted. So maybe I have to patch this file to make it put the '~' on each packet no matter what, ain't it?

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Federico Frigo@21:1/5 to All on Tue Feb 2 06:36:58 2021
    Il giorno martedì 2 febbraio 2021 alle 15:11:41 UTC+1 Rob van der Putten ha scritto:
    Hi there
    On 02/02/2021 14:29, Federico Frigo wrote:

    Il giorno martedì 2 febbraio 2021 alle 13:20:05 UTC+1 Rob van der Putten ha scritto:
    Hi there
    On 02/02/2021 12:13, Federico Frigo wrote:

    Hi there, I'm trying to connect my RaspberryPi to an FTDI chip. They
    communicate over serial, and they use PPP as underlying protocol. I'm able
    to make them communicate, but the FTDI implementation is quite buggy, so I'm
    having some trubles with the Flag Sequence, since the FTDI chip wants it at
    the beginning of every package that it receives.

    I'm using the pppd package, i tried to modify it to achive my needs but it
    was a failure, and now I'm wondering if I have to modify the Linux kernel
    to do so or if I have to simply modify the pppd package.

    Is there someone who can point me to the right direction?
    This way to fuzzy.
    Which FTDI chip do you mean?
    And as far as I know the Flag Sequence (01111110 / 0x7e / '~') is always >> there. You can disable or enable Van Jacobson TCP/IP Header Compression >> though.
    AFAIK the Raspberry Pi serial port doesn't have RTS and CTS. Without
    those you may lose data at high speeds.
    And USB to serial converters can be quite buggy. Some may drop an odd
    byte here and there. That will mess up your PPP.


    Regards,
    Rob
    I know for sure that not all ppp packets are sent with the leading '~',
    in fact I found this in the linux kernel on RaspberryPi GitHub repo: https://github.com/raspberrypi/linux/blob/rpi-5.10.y/drivers/net/ppp/ppp_async.c#L560

    As you can see, on timing basis, the leading flag sequence is not inserted. So maybe I have to patch this file to make it put the '~'
    on each packet no matter what, ain't it?
    Without wading through the entire source code, I expect this to be ~Packet~Packet~, vs ~Packet~~Packet~. Both are OK. ~PacketPacket~ is
    not! Synchronous PPP will send ~~~~~ when idle, asynchronous PPP will not. Linux has a freaky optimization in the LCP ping. It will only send a LCP ping when there hasn't been any other data for some time.
    Furthermore, the number of config requests in one LCP packet may vary.
    A PPP implementation without '~' at packet boundaries seems broken to me (See RFC). Are you this is what is happening? Make a raw data dump.

    Again, RasPi serial ports are broken. USB-Serial converters are broken.
    They do drop bytes (including '~'). Use decent serial hardware or use Ethernet for TCP/IP.

    And you still haven't explained what you are trying to do. Draw some
    ASCII art.


    Regards,
    Rob
    Oh ok, now I got this clear in my mind.

    Without taking too much time on diagrams or long explanation, I just want to force the PPP to always send packets like this:
    ~Packet~~Packet~~Packet~
    So that all packets always have their own leading and trailing '~'.

    This is because PPP stack on the other end is buggy (it's a custom implementation that unfortunately I cannot fix), and if it gets packets in the form of:
    ~Packet~Packet~Packet~
    it crashes and shut down the connection.

    I know this way is absolutely not optimized but I unfortunately don't have other options.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Rob van der Putten@21:1/5 to Federico Frigo on Tue Feb 2 16:05:03 2021
    Hi there


    On 02/02/2021 15:36, Federico Frigo wrote:

    <Cut>

    Oh ok, now I got this clear in my mind.

    Without taking too much time on diagrams or long explanation, I just
    want to force the PPP to always send packets like this: ~Packet~~Packet~~Packet~
    So that all packets always have their own leading and trailing '~'.

    This is because PPP stack on the other end is buggy (it's a custom implementation that unfortunately I cannot fix), and if it gets
    packets in the form of:
    ~Packet~Packet~Packet~
    it crashes and shut down the connection.

    I know this way is absolutely not optimized but I unfortunately
    don't have other options.

    Patch the source or pipe PPP through some process that adds '~'.


    Regards,
    Rob

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Federico Frigo@21:1/5 to All on Tue Feb 2 07:25:49 2021
    Il giorno martedì 2 febbraio 2021 alle 16:20:05 UTC+1 Rob van der Putten ha scritto:
    Hi there


    On 02/02/2021 15:36, Federico Frigo wrote:

    <Cut>
    Oh ok, now I got this clear in my mind.

    Without taking too much time on diagrams or long explanation, I just
    want to force the PPP to always send packets like this: ~Packet~~Packet~~Packet~
    So that all packets always have their own leading and trailing '~'.

    This is because PPP stack on the other end is buggy (it's a custom implementation that unfortunately I cannot fix), and if it gets
    packets in the form of:
    ~Packet~Packet~Packet~
    it crashes and shut down the connection.

    I know this way is absolutely not optimized but I unfortunately
    don't have other options.
    Patch the source or pipe PPP through some process that adds '~'.


    Regards,
    Rob
    I think i'll patch the kernel, so it'll be easier to keep under control this feature. Thanks a lot Rob, for helping me and for your quick answer! I really appreciated that.

    Best regards,
    Federico

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Tauno Voipio@21:1/5 to Federico Frigo on Tue Feb 2 19:39:07 2021
    On 2.2.21 13.13, Federico Frigo wrote:
    Hi there, I'm trying to connect my RaspberryPi to an FTDI chip. They communicate over serial, and they use PPP as underlying protocol. I'm able to make them communicate, but the FTDI implementation is quite buggy, so I'm having some trubles with the
    Flag Sequence, since the FTDI chip wants it at the beginning of every package that it receives.

    I'm using the pppd package, i tried to modify it to achive my needs but it was a failure, and now I'm wondering if I have to modify the Linux kernel to do so or if I have to simply modify the pppd package.

    Is there someone who can point me to the right direction?

    Thanks!


    The HDLC flag is a part of the PPP encapsulation for serial line,
    see RFC1662.

    It is not a property of the FTDI chip, unless you can prove
    that the flag is spoiled by the chip in some way.
    The flag (0x7e) is needed to delimit the PPP packets.

    --

    -TV

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Federico Frigo@21:1/5 to All on Thu Feb 4 03:40:56 2021
    Il giorno martedì 2 febbraio 2021 alle 18:39:14 UTC+1 Tauno Voipio ha scritto:
    On 2.2.21 13.13, Federico Frigo wrote:
    Hi there, I'm trying to connect my RaspberryPi to an FTDI chip. They communicate over serial, and they use PPP as underlying protocol. I'm able to make them communicate, but the FTDI implementation is quite buggy, so I'm having some trubles with the
    Flag Sequence, since the FTDI chip wants it at the beginning of every package that it receives.

    I'm using the pppd package, i tried to modify it to achive my needs but it was a failure, and now I'm wondering if I have to modify the Linux kernel to do so or if I have to simply modify the pppd package.

    Is there someone who can point me to the right direction?

    Thanks!
    The HDLC flag is a part of the PPP encapsulation for serial line,
    see RFC1662.

    It is not a property of the FTDI chip, unless you can prove
    that the flag is spoiled by the chip in some way.
    The flag (0x7e) is needed to delimit the PPP packets.

    --

    -TV
    Just to update you guys that helped me out and for everyone looking for answers about this problem, this is how I worked this out:
    - pppd package leverage on ppp module inside linux kernel in case of Debian distros for most of its operations (like, in my case, ppp packet encapsulation)
    - in case of solaris os, most of the code (don't know if all of it) is contained in pppd package itself

    Depending on your case, you need to modify either linux kernel or pppd package. In my case I had to modify Raspberry Pi OS kernel (https://github.com/raspberrypi/linux), then I had to build it and mount it on Raspberry Pi SD following their guide (https:/
    /www.raspberrypi.org/documentation/linux/kernel/building.md). Now everything is working like a charm.
    Thanks again guys!
    Sincerely,

    Federico

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Rob van der Putten@21:1/5 to Federico Frigo on Fri Feb 5 09:29:44 2021
    Hi there


    On 04/02/2021 12:40, Federico Frigo wrote:

    Il giorno martedì 2 febbraio 2021 alle 18:39:14 UTC+1 Tauno Voipio ha scritto:
    On 2.2.21 13.13, Federico Frigo wrote:
    Hi there, I'm trying to connect my RaspberryPi to an FTDI chip. They communicate over serial, and they use PPP as underlying protocol. I'm able to make them communicate, but the FTDI implementation is quite buggy, so I'm having some trubles with the
    Flag Sequence, since the FTDI chip wants it at the beginning of every package that it receives.

    I'm using the pppd package, i tried to modify it to achive my needs but it was a failure, and now I'm wondering if I have to modify the Linux kernel to do so or if I have to simply modify the pppd package.

    Is there someone who can point me to the right direction?

    Thanks!
    The HDLC flag is a part of the PPP encapsulation for serial line,
    see RFC1662.

    It is not a property of the FTDI chip, unless you can prove
    that the flag is spoiled by the chip in some way.
    The flag (0x7e) is needed to delimit the PPP packets.

    --

    -TV
    Just to update you guys that helped me out and for everyone looking for answers about this problem, this is how I worked this out:
    - pppd package leverage on ppp module inside linux kernel in case of Debian distros for most of its operations (like, in my case, ppp packet encapsulation)
    - in case of solaris os, most of the code (don't know if all of it) is contained in pppd package itself

    Depending on your case, you need to modify either linux kernel or pppd package. In my case I had to modify Raspberry Pi OS kernel (https://github.com/raspberrypi/linux), then I had to build it and mount it on Raspberry Pi SD following their guide (
    https://www.raspberrypi.org/documentation/linux/kernel/building.md). Now everything is working like a charm.

    Perhaps you can turn it into a config option and supply a patch.


    Regards,
    Rob

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Federico Frigo@21:1/5 to All on Tue Feb 16 02:21:12 2021
    Il giorno venerdì 5 febbraio 2021 alle 09:30:06 UTC+1 Rob van der Putten ha scritto:
    Hi there
    On 04/02/2021 12:40, Federico Frigo wrote:

    Il giorno martedì 2 febbraio 2021 alle 18:39:14 UTC+1 Tauno Voipio ha scritto:
    On 2.2.21 13.13, Federico Frigo wrote:
    Hi there, I'm trying to connect my RaspberryPi to an FTDI chip. They communicate over serial, and they use PPP as underlying protocol. I'm able to make them communicate, but the FTDI implementation is quite buggy, so I'm having some trubles with
    the Flag Sequence, since the FTDI chip wants it at the beginning of every package that it receives.

    I'm using the pppd package, i tried to modify it to achive my needs but it was a failure, and now I'm wondering if I have to modify the Linux kernel to do so or if I have to simply modify the pppd package.

    Is there someone who can point me to the right direction?

    Thanks!
    The HDLC flag is a part of the PPP encapsulation for serial line,
    see RFC1662.

    It is not a property of the FTDI chip, unless you can prove
    that the flag is spoiled by the chip in some way.
    The flag (0x7e) is needed to delimit the PPP packets.

    --

    -TV
    Just to update you guys that helped me out and for everyone looking for answers about this problem, this is how I worked this out:
    - pppd package leverage on ppp module inside linux kernel in case of Debian distros for most of its operations (like, in my case, ppp packet encapsulation)
    - in case of solaris os, most of the code (don't know if all of it) is contained in pppd package itself

    Depending on your case, you need to modify either linux kernel or pppd package. In my case I had to modify Raspberry Pi OS kernel (https://github.com/raspberrypi/linux), then I had to build it and mount it on Raspberry Pi SD following their guide (
    https://www.raspberrypi.org/documentation/linux/kernel/building.md). Now everything is working like a charm.
    Perhaps you can turn it into a config option and supply a patch.


    Regards,
    Rob

    Hi again, I found out that flag_time is a module parameter in linux kernel (see: https://github.com/torvalds/linux/blob/master/drivers/net/ppp/ppp_async.c#L88), so my guess is that is possible to set its value from the outside somehow (i read this guide
    about that: https://tldp.org/LDP/lkmpg/2.6/html/x323.html).

    If this is the case, then it should be possible to add this feature directly to pppd module instead of editing linux kernel; the problem is that I don't know how to do this... Could you maybe point me to the right direction?

    Thanks!

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Federico Frigo@21:1/5 to All on Tue Feb 16 03:56:26 2021
    Il giorno martedì 16 febbraio 2021 alle 11:21:15 UTC+1 Federico Frigo ha scritto:
    Il giorno venerdì 5 febbraio 2021 alle 09:30:06 UTC+1 Rob van der Putten ha scritto:
    Hi there
    On 04/02/2021 12:40, Federico Frigo wrote:

    Il giorno martedì 2 febbraio 2021 alle 18:39:14 UTC+1 Tauno Voipio ha scritto:
    On 2.2.21 13.13, Federico Frigo wrote:
    Hi there, I'm trying to connect my RaspberryPi to an FTDI chip. They communicate over serial, and they use PPP as underlying protocol. I'm able to make them communicate, but the FTDI implementation is quite buggy, so I'm having some trubles with
    the Flag Sequence, since the FTDI chip wants it at the beginning of every package that it receives.

    I'm using the pppd package, i tried to modify it to achive my needs but it was a failure, and now I'm wondering if I have to modify the Linux kernel to do so or if I have to simply modify the pppd package.

    Is there someone who can point me to the right direction?

    Thanks!
    The HDLC flag is a part of the PPP encapsulation for serial line,
    see RFC1662.

    It is not a property of the FTDI chip, unless you can prove
    that the flag is spoiled by the chip in some way.
    The flag (0x7e) is needed to delimit the PPP packets.

    --

    -TV
    Just to update you guys that helped me out and for everyone looking for answers about this problem, this is how I worked this out:
    - pppd package leverage on ppp module inside linux kernel in case of Debian distros for most of its operations (like, in my case, ppp packet encapsulation)
    - in case of solaris os, most of the code (don't know if all of it) is contained in pppd package itself

    Depending on your case, you need to modify either linux kernel or pppd package. In my case I had to modify Raspberry Pi OS kernel (https://github.com/raspberrypi/linux), then I had to build it and mount it on Raspberry Pi SD following their guide (
    https://www.raspberrypi.org/documentation/linux/kernel/building.md). Now everything is working like a charm.
    Perhaps you can turn it into a config option and supply a patch.


    Regards,
    Rob
    Hi again, I found out that flag_time is a module parameter in linux kernel (see: https://github.com/torvalds/linux/blob/master/drivers/net/ppp/ppp_async.c#L88), so my guess is that is possible to set its value from the outside somehow (i read this
    guide about that: https://tldp.org/LDP/lkmpg/2.6/html/x323.html).

    If this is the case, then it should be possible to add this feature directly to pppd module instead of editing linux kernel; the problem is that I don't know how to do this... Could you maybe point me to the right direction?

    Thanks!

    Some updates: I managed to make this work by manually loading the ppp_async kernel module passing the flag_time=0 parameter to it before starting pppd, like this:

    sudo modprobe ppp_async flag_time=0

    This could be obviously much easier if it can be converted to an options option, something like FLAG_TIME that can be set to an integer, then pppd takes care of loading it.

    Again, I can do that, but I need some help to understand from where to start 😁

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Giovanni@21:1/5 to Federico Frigo on Tue Feb 16 14:49:27 2021
    On 02/16/2021 12:56 PM, Federico Frigo wrote:
    Some updates: I managed to make this work by manually loading the
    ppp_async kernel module passing the flag_time=0 parameter to it
    before starting pppd, like this:

    sudo modprobe ppp_async flag_time=0

    This could be obviously much easier if it can be converted to an
    options option, something like FLAG_TIME that can be set to an
    integer, then pppd takes care of loading it.

    flag_time is already an option for ppp_async and you can load it
    automatically entering a line:

    options ppp_async flag_time=0

    in a file with the extension .conf created in the directory
    /etc/modprobe.d/

    Ciao
    Giovanni
    --
    A computer is like an air conditioner,
    it stops working when you open Windows.
    < http://giovanni.homelinux.net/ >

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Henning Hucke@21:1/5 to Federico Frigo on Tue Feb 16 14:02:43 2021
    On 2021-02-16, Federico Frigo <ffrigo8@gmail.com> wrote:

    [...]
    Again, I can do that, but I need some help to understand from where to start 😁

    # ( modinfo ppp_async | less -S ) && man modprobe.d

    You are not too familiar with the kernel module system, aren't you?
    SMILEY! -> :-)

    Best regards
    Henning Hucke
    --
    "nobody is perfect."
    -- Nobody ;)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Henning Hucke@21:1/5 to Federico Frigo on Tue Feb 16 14:50:01 2021
    On 2021-02-16, Federico Frigo <ffrigo8@gmail.com> wrote:

    [...]
    Again, I can do that, but I need some help to understand from where to start 😁

    # ( modinfo ppp_async | less -S ) && man modprobe.d

    You are not too familiar with the kernel module system, aren't you?
    SMILEY! -> :-)

    Best regards
    Henning (nobody) Hucke
    --
    "nobody is perfect."
    -- Nobody ;)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)