1. Forum
  2. Usenet
  3. COMP.UNIX.SHELL

  • There are too many sshd processes has been started on the server trigge

    From hongyi.zhao@gmail.com@21:1/5 to All on Thu Oct 28 02:20:05 2021
    On client, I run the follownig command to create a ssh based remote port forwarding proxy:

    $ autossh -M 0 -4 -NT \
    -o ServerAliveInterval=30 \
    -o "ServerAliveCountMax=3" \
    -o "ExitOnForwardFailure=yes" \
    -o StrictHostKeyChecking=no \
    -R 21080:localhost:1080 \
    -R 20022:localhost:22 \
    werner@my.remote.server -p 2101

    Then on the server side, I observed there are too many sshd processes has been started:

    werner@Standard-PC-i440FX-PIIX-1996:~$ sudo lsof -i :20022
    COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
    sshd 196794 werner 12u IPv4 1409853 0t0 TCP *:20022 (LISTEN)
    sshd 196794 werner 13u IPv6 1409854 0t0 TCP *:20022 (LISTEN)
    sshd 196794 werner 50u IPv4 1411624 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:56586 (CLOSE_WAIT)
    sshd 196794 werner 55u IPv4 1411630 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:38018 (ESTABLISHED)
    sshd 196794 werner 60u IPv4 1411660 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:47680 (ESTABLISHED)
    sshd 196794 werner 67u IPv4 1411668 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:57342 (ESTABLISHED)
    sshd 196794 werner 70u IPv4 1411671 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:38772 (ESTABLISHED)
    sshd 196794 werner 71u IPv4 1411672 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:48434 (ESTABLISHED)
    sshd 196794 werner 74u IPv4 1411675 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:58096 (ESTABLISHED)
    sshd 196794 werner 75u IPv4 1411711 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:39526 (ESTABLISHED)
    sshd 196794 werner 87u IPv4 1411719 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:49188 (ESTABLISHED)
    sshd 196794 werner 88u IPv4 1411721 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:58850 (ESTABLISHED)
    sshd 196794 werner 89u IPv4 1411722 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:40280 (ESTABLISHED)
    sshd 196794 werner 90u IPv4 1411761 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:49942 (ESTABLISHED)
    sshd 196794 werner 91u IPv4 1411763 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:59604 (ESTABLISHED)
    sshd 196794 werner 92u IPv4 1411764 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:41034 (ESTABLISHED)
    sshd 196794 werner 93u IPv4 1411765 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:50696 (ESTABLISHED)
    sshd 196794 werner 96u IPv4 1411768 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:60358 (ESTABLISHED)
    sshd 196794 werner 97u IPv4 1411802 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:41788 (ESTABLISHED)
    sshd 196794 werner 102u IPv4 1411807 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:51450 (ESTABLISHED)
    sshd 196794 werner 103u IPv4 1411808 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:32880 (ESTABLISHED)
    sshd 196794 werner 104u IPv4 1411809 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:42542 (ESTABLISHED)
    sshd 196794 werner 105u IPv4 1411811 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:52206 (ESTABLISHED)
    sshd 196794 werner 106u IPv4 1411852 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:33638 (ESTABLISHED)
    sshd 196794 werner 107u IPv4 1411853 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:43300 (ESTABLISHED)
    sshd 196794 werner 108u IPv4 1411854 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:52962 (ESTABLISHED)
    sshd 196794 werner 109u IPv4 1411855 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:34392 (ESTABLISHED)
    sshd 196794 werner 110u IPv4 1411856 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:44054 (ESTABLISHED)
    sshd 196794 werner 111u IPv4 1411894 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:53716 (ESTABLISHED)
    sshd 196794 werner 112u IPv4 1411895 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:35148 (ESTABLISHED)
    sshd 196794 werner 113u IPv4 1411896 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:44810 (ESTABLISHED)
    sshd 196794 werner 114u IPv4 1411897 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:54472 (ESTABLISHED)
    sshd 196794 werner 115u IPv4 1411926 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:35906 (ESTABLISHED)
    sshd 196794 werner 116u IPv4 1411927 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:45572 (ESTABLISHED)
    sshd 196794 werner 117u IPv4 1411928 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:55234 (ESTABLISHED)
    sshd 196794 werner 118u IPv4 1411929 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:36668 (ESTABLISHED)
    sshd 196794 werner 119u IPv4 1411930 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:46330 (ESTABLISHED)
    sshd 196794 werner 120u IPv4 1411956 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:55992 (ESTABLISHED)
    sshd 196794 werner 121u IPv4 1411957 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:37422 (ESTABLISHED)
    sshd 196794 werner 122u IPv4 1411958 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:47084 (ESTABLISHED)
    sshd 196794 werner 123u IPv4 1411959 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:56746 (ESTABLISHED)
    sshd 196794 werner 124u IPv4 1411960 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:38176 (ESTABLISHED)
    sshd 196794 werner 125u IPv4 1412012 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:47838 (ESTABLISHED)
    sshd 196794 werner 126u IPv4 1412013 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:57500 (ESTABLISHED)
    sshd 196794 werner 127u IPv4 1412014 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:38930 (ESTABLISHED)
    sshd 196794 werner 128u IPv4 1414997 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:48592 (ESTABLISHED)
    sshd 196794 werner 129u IPv4 1416878 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:58252 (ESTABLISHED)
    sshd 196794 werner 130u IPv4 1417151 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:39684 (ESTABLISHED)
    sshd 196794 werner 131u IPv4 1417152 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:49346 (ESTABLISHED)
    sshd 196794 werner 132u IPv4 1417153 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:59008 (ESTABLISHED)
    sshd 196794 werner 133u IPv4 1418662 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:40438 (ESTABLISHED)
    sshd 196794 werner 134u IPv4 1418702 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:50100 (ESTABLISHED)
    sshd 196794 werner 135u IPv4 1418703 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:59762 (ESTABLISHED)
    sshd 196794 werner 136u IPv4 1418704 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:41192 (ESTABLISHED)
    sshd 196794 werner 137u IPv4 1418705 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:50854 (ESTABLISHED)
    sshd 196794 werner 138u IPv4 1419023 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:60516 (ESTABLISHED)
    sshd 196794 werner 139u IPv4 1419055 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:41946 (ESTABLISHED)
    sshd 196794 werner 140u IPv4 1419056 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:51608 (ESTABLISHED)
    sshd 196794 werner 141u IPv4 1419057 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:33040 (ESTABLISHED)
    sshd 196794 werner 142u IPv4 1419058 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:42702 (ESTABLISHED)
    sshd 196794 werner 143u IPv4 1419060 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:52370 (ESTABLISHED)
    sshd 196794 werner 144u IPv4 1419090 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:33800 (ESTABLISHED)
    sshd 196794 werner 145u IPv4 1419091 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:43462 (ESTABLISHED)
    sshd 196794 werner 146u IPv4 1420396 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:53124 (ESTABLISHED)
    sshd 196794 werner 147u IPv4 1420397 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:34554 (ESTABLISHED)
    sshd 196794 werner 148u IPv4 1420428 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:44216 (ESTABLISHED)
    sshd 196794 werner 149u IPv4 1420429 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:53878 (ESTABLISHED)
    sshd 196794 werner 150u IPv4 1420430 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:35308 (ESTABLISHED)
    sshd 196794 werner 151u IPv4 1420432 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:44970 (ESTABLISHED)
    sshd 196794 werner 152u IPv4 1420433 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:54632 (ESTABLISHED)
    sshd 196794 werner 153u IPv4 1420476 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:36062 (ESTABLISHED)
    sshd 196794 werner 154u IPv4 1420477 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:45724 (ESTABLISHED)
    sshd 196794 werner 155u IPv4 1420478 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:55386 (ESTABLISHED)
    sshd 196794 werner 156u IPv4 1420479 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:36816 (ESTABLISHED)
    sshd 196794 werner 157u IPv4 1420819 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:46478 (ESTABLISHED)
    sshd 196794 werner 158u IPv4 1420848 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:56140 (ESTABLISHED)
    sshd 196794 werner 159u IPv4 1420849 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:37570 (ESTABLISHED)
    sshd 196794 werner 160u IPv4 1420850 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:47234 (ESTABLISHED)
    sshd 196794 werner 161u IPv4 1420851 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:56896 (ESTABLISHED)
    sshd 196794 werner 162u IPv4 1420880 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:38326 (ESTABLISHED)
    sshd 196794 werner 163u IPv4 1420881 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:47988 (ESTABLISHED)
    sshd 196794 werner 164u IPv4 1420882 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:57650 (ESTABLISHED)
    sshd 196794 werner 165u IPv4 1420883 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:39080 (ESTABLISHED)
    sshd 196794 werner 166u IPv4 1420884 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:48742 (ESTABLISHED)
    sshd 196794 werner 167u IPv4 1420926 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:58404 (ESTABLISHED)
    sshd 196794 werner 168u IPv4 1420927 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:39834 (ESTABLISHED)
    sshd 196794 werner 169u IPv4 1420928 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:49496 (ESTABLISHED)
    sshd 196794 werner 170u IPv4 1420930 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:59158 (ESTABLISHED)
    sshd 196794 werner 171u IPv4 1420931 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:40588 (ESTABLISHED)
    sshd 196794 werner 172u IPv4 1422370 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:50250 (ESTABLISHED)
    sshd 196794 werner 173u IPv4 1422371 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:59912 (ESTABLISHED)
    sshd 196794 werner 174u IPv4 1422373 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:41342 (ESTABLISHED)
    sshd 196794 werner 175u IPv4 1422374 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:51004 (ESTABLISHED)
    sshd 196794 werner 176u IPv4 1422375 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:60668 (ESTABLISHED)
    sshd 196794 werner 177u IPv4 1422417 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:42098 (ESTABLISHED)
    sshd 196794 werner 178u IPv4 1422418 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:51760 (ESTABLISHED)
    sshd 196794 werner 179u IPv4 1422419 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:33190 (ESTABLISHED)
    sshd 196794 werner 180u IPv4 1422421 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:42852 (ESTABLISHED)
    sshd 196794 werner 181u IPv4 1422460 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:52514 (ESTABLISHED)
    sshd 196794 werner 182u IPv4 1422461 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:33946 (ESTABLISHED)
    sshd 196794 werner 183u IPv4 1422462 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:43608 (ESTABLISHED)
    sshd 196794 werner 184u IPv4 1422468 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:53270 (ESTABLISHED)
    sshd 196794 werner 185u IPv4 1422472 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:34700 (ESTABLISHED)
    sshd 196794 werner 186u IPv4 1422500 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:44362 (ESTABLISHED)
    sshd 196794 werner 187u IPv4 1422501 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:54024 (ESTABLISHED)
    sshd 196794 werner 188u IPv4 1422502 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:35454 (ESTABLISHED)
    sshd 196794 werner 189u IPv4 1422503 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:45116 (ESTABLISHED)
    sshd 196794 werner 190u IPv4 1422504 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:54778 (ESTABLISHED)
    sshd 196794 werner 193u IPv4 1422545 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:36208 (ESTABLISHED)
    sshd 196794 werner 198u IPv4 1422800 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:45872 (ESTABLISHED)
    sshd 196794 werner 202u IPv4 1422804 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:55534 (ESTABLISHED)
    sshd 196794 werner 204u IPv4 1422806 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:36964 (ESTABLISHED)
    sshd 196794 werner 205u IPv4 1422807 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:46626 (ESTABLISHED)
    sshd 196794 werner 206u IPv4 1422835 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:56288 (ESTABLISHED)
    sshd 196794 werner 207u IPv4 1422836 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:37718 (ESTABLISHED)
    sshd 196794 werner 208u IPv4 1422837 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:47380 (ESTABLISHED)
    sshd 196794 werner 217u IPv4 1422847 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:57042 (ESTABLISHED)
    sshd 196794 werner 224u IPv4 1422880 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:38472 (ESTABLISHED)
    sshd 196794 werner 225u IPv4 1422881 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:48134 (ESTABLISHED)
    sshd 196794 werner 226u IPv4 1422882 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:57796 (ESTABLISHED)
    sshd 196794 werner 227u IPv4 1423179 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:39226 (ESTABLISHED)
    sshd 196794 werner 230u IPv4 1423214 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:48888 (ESTABLISHED)
    sshd 196794 werner 231u IPv4 1423238 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:58554 (ESTABLISHED)
    sshd 196794 werner 234u IPv4 1423262 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:39984 (ESTABLISHED)
    sshd 196794 werner 235u IPv4 1424514 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->143.198.101.180:49646 (ESTABLISHED)


    And this will make SSH forwarding unavailable. Any hints for fixing this problem?

    Regards,
    HZ

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Spiros Bousbouras@21:1/5 to hongy...@gmail.com on Thu Oct 28 11:18:11 2021
    On Thu, 28 Oct 2021 02:20:05 -0700 (PDT)
    "hongy...@gmail.com" <hongyi.zhao@gmail.com> wrote:
    On client, I run the follownig command to create a ssh based remote
    port forwarding proxy:

    $ autossh -M 0 -4 -NT \
    -o ServerAliveInterval=30 \
    -o "ServerAliveCountMax=3" \
    -o "ExitOnForwardFailure=yes" \
    -o StrictHostKeyChecking=no \
    -R 21080:localhost:1080 \
    -R 20022:localhost:22 \
    werner@my.remote.server -p 2101

    Then on the server side, I observed there are too many sshd processes
    has been started:

    werner@Standard-PC-i440FX-PIIX-1996:~$ sudo lsof -i :20022

    Why do you need sudo to run lsof ?

    COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
    sshd 196794 werner 12u IPv4 1409853 0t0 TCP *:20022 (LISTEN)
    sshd 196794 werner 13u IPv6 1409854 0t0 TCP *:20022 (LISTEN)

    [Rest of long output snipped]

    In the whole output I only see one process with PID 196794.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From hongyi.zhao@gmail.com@21:1/5 to Spiros Bousbouras on Thu Oct 28 05:32:02 2021
    On Thursday, October 28, 2021 at 7:18:17 PM UTC+8, Spiros Bousbouras wrote:
    On Thu, 28 Oct 2021 02:20:05 -0700 (PDT)
    "hongy...@gmail.com" <hongy...@gmail.com> wrote:
    On client, I run the follownig command to create a ssh based remote
    port forwarding proxy:

    $ autossh -M 0 -4 -NT \
    -o ServerAliveInterval=30 \
    -o "ServerAliveCountMax=3" \
    -o "ExitOnForwardFailure=yes" \
    -o StrictHostKeyChecking=no \
    -R 21080:localhost:1080 \
    -R 20022:localhost:22 \
    wer...@my.remote.server -p 2101

    Then on the server side, I observed there are too many sshd processes
    has been started:

    werner@Standard-PC-i440FX-PIIX-1996:~$ sudo lsof -i :20022
    Why do you need sudo to run lsof ?

    This is my misuse :-(

    COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
    sshd 196794 werner 12u IPv4 1409853 0t0 TCP *:20022 (LISTEN)
    sshd 196794 werner 13u IPv6 1409854 0t0 TCP *:20022 (LISTEN)
    [Rest of long output snipped]

    In the whole output I only see one process with PID 196794.

    I really not noticed that. I mean so many sshd entries belonging to the same process.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Spiros Bousbouras@21:1/5 to hongy...@gmail.com on Thu Oct 28 12:54:17 2021
    On Thu, 28 Oct 2021 05:32:02 -0700 (PDT)
    "hongy...@gmail.com" <hongyi.zhao@gmail.com> wrote:
    On Thursday, October 28, 2021 at 7:18:17 PM UTC+8, Spiros Bousbouras wrote:
    On Thu, 28 Oct 2021 02:20:05 -0700 (PDT)
    "hongy...@gmail.com" <hongy...@gmail.com> wrote:
    On client, I run the follownig command to create a ssh based remote
    port forwarding proxy:

    $ autossh -M 0 -4 -NT \
    -o ServerAliveInterval=30 \
    -o "ServerAliveCountMax=3" \
    -o "ExitOnForwardFailure=yes" \
    -o StrictHostKeyChecking=no \
    -R 21080:localhost:1080 \
    -R 20022:localhost:22 \
    wer...@my.remote.server -p 2101

    Then on the server side, I observed there are too many sshd processes
    has been started:

    werner@Standard-PC-i440FX-PIIX-1996:~$ sudo lsof -i :20022
    Why do you need sudo to run lsof ?

    This is my misuse :-(

    COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
    sshd 196794 werner 12u IPv4 1409853 0t0 TCP *:20022 (LISTEN)
    sshd 196794 werner 13u IPv6 1409854 0t0 TCP *:20022 (LISTEN)
    [Rest of long output snipped]

    In the whole output I only see one process with PID 196794.

    I really not noticed that. I mean so many sshd entries belonging to the
    same process.

    If I understand correctly the output of lsof , it says that one sshd
    process (with PID 196794) has made multiple connections. I confess that I
    only have a vague idea as to what a "remote port forwarding proxy" does. Is
    it implausible that it would open multiple connections ?

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Eli the Bearded@21:1/5 to hongy...@gmail.com on Thu Oct 28 17:37:23 2021
    In comp.unix.shell, Spiros Bousbouras <spibou@gmail.com> wrote:
    [ "hongy...@gmail.com" <hongyi.zhao@gmail.com> wrote ]
    $ autossh -M 0 -4 -NT \
    -o ServerAliveInterval=30 \
    -o "ServerAliveCountMax=3" \
    -o "ExitOnForwardFailure=yes" \
    -o StrictHostKeyChecking=no \
    -R 21080:localhost:1080 \
    -R 20022:localhost:22 \
    ...
    If I understand correctly the output of lsof , it says that one sshd process (with PID 196794) has made multiple connections. I confess that I only have a vague idea as to what a "remote port forwarding proxy" does. Is it implausible that it would open multiple connections ?

    It is entirely plausible that something that acts as a port forwarding
    agent would accept multiple connections on the same port to forward them
    all. Looking at that, I suspect it is intended to forward ssh and web
    traffic (1080 == SOCKS). One web browser will likely create a bunch of connections for each page. If it has to wait to do those single file
    browsing will be very slow. I conclude that opening multiple connections
    very much an intended use.

    Elijah
    ------
    remembers writing single threaded proxies for studying http back in the day

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Grant Taylor@21:1/5 to hongy...@gmail.com on Thu Oct 28 12:47:26 2021
    On 10/28/21 3:20 AM, hongy...@gmail.com wrote:
    On client, I run the follownig command to create a ssh based remote
    port forwarding proxy:

    $ autossh -M 0 -4 -NT \
    -o ServerAliveInterval=30 \
    -o "ServerAliveCountMax=3" \
    -o "ExitOnForwardFailure=yes" \
    -o StrictHostKeyChecking=no \
    -R 21080:localhost:1080 \
    -R 20022:localhost:22 \
    werner@my.remote.server -p 2101

    Then on the server side, I observed there are too many sshd processes
    has been started:

    As Spiros pointed out, they are all the same sshd process; 196794.

    werner@Standard-PC-i440FX-PIIX-1996:~$ sudo lsof -i :20022
    COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME

    If I've tracked the topology correctly, those all look like
    143.198.101.180 connected to port 20022 on the remote server which is
    forwarded through SSH to port 22 on the client.

    This seems like an SSH attack from 143.198.101.180 to me. Maybe
    password guessing / brute forcing.

    And this will make SSH forwarding unavailable. Any hints for fixing
    this problem?

    I'm guessing that you're either tripping a security feature (number of
    total connections, number in a given time, number from a specific
    source, etc.). I don't see how this would actually prevent the (remote)
    port forwarding from working with SSH. This seems like a denial of
    service type issue.



    --
    Grant. . . .
    unix || die

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From hongyi.zhao@gmail.com@21:1/5 to Grant Taylor on Thu Oct 28 18:34:18 2021
    On Friday, October 29, 2021 at 2:47:16 AM UTC+8, Grant Taylor wrote:
    On 10/28/21 3:20 AM, hongy...@gmail.com wrote:
    On client, I run the follownig command to create a ssh based remote
    port forwarding proxy:

    $ autossh -M 0 -4 -NT \
    -o ServerAliveInterval=30 \
    -o "ServerAliveCountMax=3" \
    -o "ExitOnForwardFailure=yes" \
    -o StrictHostKeyChecking=no \
    -R 21080:localhost:1080 \
    -R 20022:localhost:22 \
    wer...@my.remote.server -p 2101

    Then on the server side, I observed there are too many sshd processes
    has been started:
    As Spiros pointed out, they are all the same sshd process; 196794.
    werner@Standard-PC-i440FX-PIIX-1996:~$ sudo lsof -i :20022
    COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
    If I've tracked the topology correctly, those all look like
    143.198.101.180 connected to port 20022 on the remote server which is forwarded through SSH to port 22 on the client.

    This seems like an SSH attack from 143.198.101.180 to me. Maybe
    password guessing / brute forcing.
    And this will make SSH forwarding unavailable. Any hints for fixing
    this problem?
    I'm guessing that you're either tripping a security feature (number of
    total connections, number in a given time, number from a specific
    source, etc.). I don't see how this would actually prevent the (remote)
    port forwarding from working with SSH. This seems like a denial of
    service type issue.

    Thank you for your in-depth analysis. Basically, my goal is to achieve intranet penetration without public IP. It seems that the ssh remote port forwarding method discussed here is not carefully designed for this purpose. So, maybe I should switch to
    zerotier-one [1].

    [1] https://github.com/zerotier/ZeroTierOne

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Grant Taylor@21:1/5 to hongy...@gmail.com on Fri Oct 29 00:13:16 2021
    On 10/28/21 7:34 PM, hongy...@gmail.com wrote:
    Thank you for your in-depth analysis.

    You're welcome.

    Basically, my goal is to achieve intranet penetration without public
    IP.

    Okay. That seems sensible enough.

    I take it that you do have access to a system with a globally routed IP
    that you are able to use some ports on to get traffic into your
    system(s) without a globally routed IP.

    I've done similar with SSH many times before.

    It seems that the ssh remote port forwarding method discussed here
    is not carefully designed for this purpose.

    I wouldn't say that.

    I'd be more apt to suspect some sort of rate limiting elsewhere that was interfering with connections and that SSH itself wasn't the problem.
    I've push a LOT of things through a LOT of crazy SSH configurations.
    Remote port forwarding is about the most mundane thing that I've done.

    A firewall on the remote system may be filtering additional incoming connections. A VPS / hosting provider may be filtering the number of
    outgoing packets form a system. A firewall on the internal system (ssh
    client) may be filtering things. I'm fairly certain that you can even configure OpenSSH to rate limit connections, including source IP. There
    are a number of things that could be interfering before SSH becoming the
    actual problem.

    So, maybe I should switch to zerotier-one [1].

    I spent a few minutes looking at ZeroTier-One. The website (as opposed
    to the content) looks ... polished. They seem to be using a lot of buzz
    words. Many of them are used in context and correctly. What I don't
    see in about five minutes time is any hint at the technology behind
    their solution. So I find myself being extremely skeptical of what they
    have to offer.

    I would spend about 10 times the effort on SSH before spending any on ZeroTier-One or anything that seems care sales tactics to me.



    --
    Grant. . . .
    unix || die

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From William Unruh@21:1/5 to Grant Taylor on Fri Oct 29 09:43:18 2021
    On 2021-10-29, Grant Taylor <gtaylor@tnetconsulting.net> wrote:
    On 10/28/21 7:34 PM, hongy...@gmail.com wrote:
    .....

    So, maybe I should switch to zerotier-one [1].

    I spent a few minutes looking at ZeroTier-One. The website (as opposed
    to the content) looks ... polished. They seem to be using a lot of buzz words. Many of them are used in context and correctly. What I don't
    see in about five minutes time is any hint at the technology behind
    their solution. So I find myself being extremely skeptical of what they
    have to offer.

    Of course the technology could just be ssh port forwarding hidden behind proprietary cover.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Grant Taylor@21:1/5 to William Unruh on Fri Oct 29 10:36:15 2021
    On 10/29/21 3:43 AM, William Unruh wrote:
    Of course the technology could just be ssh port forwarding hidden
    behind proprietary cover.

    I'm fairly certain that ZeroTier-One is not using /just/ ssh port
    forwarding.

    I say this because of what ssh's port forwarding does vs what
    ZeroTier-One claims they do. First, ssh's port forwarding is inherently
    a singular port, hence it's name. Second, ZeroTier-One claims to
    emulate an Ethernet LAN of connected system, which means that it's an L2 connection. Port forwarding and L2 connections are considerably
    different. Third, the "just works" comments indicate that there is a
    lot more automagic / discovery that happens behind the scenes.

    Yes, ssh can forward multiple ports, but that's additional configuration.

    Yes, ssh can create an L2 VPN tunnel, but that's a lot of additional configuration. The emulated Ethernet LAN would also require bridging
    multiple L2 VPN tunnels together, something that's decidedly outside of
    ssh's purview.

    I'm not aware of any ssh support for auto-discovery / auto-configuration
    beyond DNSSEC protected SSHFP keys or certificates from a trusted SSH
    root CA.

    I think that there is a lot more going on under the hood behind
    ZeroTier-One than meets the eye. I think it's more in the realm of SDN
    and further away from ssh. It may use ssh as part of it's technology
    stack, but I wouldn't bet a cup of coffee that it does. In fact I'd bet
    a cup of coffee that it does not use ssh as part of it's technology stack.



    --
    Grant. . . .
    unix || die

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From William Unruh@21:1/5 to Grant Taylor on Fri Oct 29 19:20:06 2021
    On 2021-10-29, Grant Taylor <gtaylor@tnetconsulting.net> wrote:
    On 10/29/21 3:43 AM, William Unruh wrote:
    Of course the technology could just be ssh port forwarding hidden
    behind proprietary cover.

    I'm fairly certain that ZeroTier-One is not using /just/ ssh port
    forwarding.

    I say this because of what ssh's port forwarding does vs what
    ZeroTier-One claims they do. First, ssh's port forwarding is inherently
    a singular port, hence it's name. Second, ZeroTier-One claims to
    emulate an Ethernet LAN of connected system, which means that it's an L2 connection. Port forwarding and L2 connections are considerably
    different. Third, the "just works" comments indicate that there is a
    lot more automagic / discovery that happens behind the scenes.

    Yes, ssh can forward multiple ports, but that's additional configuration.

    Yes, ssh can create an L2 VPN tunnel, but that's a lot of additional configuration. The emulated Ethernet LAN would also require bridging multiple L2 VPN tunnels together, something that's decidedly outside of
    ssh's purview.

    I'm not aware of any ssh support for auto-discovery / auto-configuration beyond DNSSEC protected SSHFP keys or certificates from a trusted SSH
    root CA.

    I think that there is a lot more going on under the hood behind
    ZeroTier-One than meets the eye. I think it's more in the realm of SDN
    and further away from ssh. It may use ssh as part of it's technology
    stack, but I wouldn't bet a cup of coffee that it does. In fact I'd bet
    a cup of coffee that it does not use ssh as part of it's technology stack.

    What Marketing says and what the technical reality is can be very far
    apart. All we have is the marketing in this case it seems.




    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Grant Taylor@21:1/5 to William Unruh on Fri Oct 29 15:02:01 2021
    On 10/29/21 1:20 PM, William Unruh wrote:
    What Marketing says and what the technical reality is can be very
    far apart. All we have is the marketing in this case it seems.

    Agreed.

    In my experience, when the public information is disproportionately
    marketing, that's because the technical information has been withheld on purpose. Almost every time it's because the technical information
    wouldn't support the weight of the marketing information.



    --
    Grant. . . .
    unix || die

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From hongyi.zhao@gmail.com@21:1/5 to hongy...@gmail.com on Fri Oct 29 21:45:45 2021
    On Saturday, October 30, 2021 at 12:22:05 PM UTC+8, hongy...@gmail.com wrote:
    On Thursday, October 28, 2021 at 8:32:06 PM UTC+8, hongy...@gmail.com wrote:
    On Thursday, October 28, 2021 at 7:18:17 PM UTC+8, Spiros Bousbouras wrote:
    On Thu, 28 Oct 2021 02:20:05 -0700 (PDT)
    "hongy...@gmail.com" <hongy...@gmail.com> wrote:
    On client, I run the follownig command to create a ssh based remote port forwarding proxy:

    $ autossh -M 0 -4 -NT \
    -o ServerAliveInterval=30 \
    -o "ServerAliveCountMax=3" \
    -o "ExitOnForwardFailure=yes" \
    -o StrictHostKeyChecking=no \
    -R 21080:localhost:1080 \
    -R 20022:localhost:22 \
    wer...@my.remote.server -p 2101

    Then on the server side, I observed there are too many sshd processes has been started:

    werner@Standard-PC-i440FX-PIIX-1996:~$ sudo lsof -i :20022
    Why do you need sudo to run lsof ?
    This is my misuse :-(
    Based on my further tries, only the sudo can display the results:

    werner@Standard-PC-i440FX-PIIX-1996:~$ lsof -i :20022 werner@Standard-PC-i440FX-PIIX-1996:~$ sudo lsof -i :20022
    COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
    sshd 417861 werner 12u IPv4 3063124 0t0 TCP *:20022 (LISTEN)
    sshd 417861 werner 13u IPv6 3063125 0t0 TCP *:20022 (LISTEN)
    sshd 417861 werner 14u IPv4 3092408 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->OpenWrt.lan:47194 (CLOSE_WAIT)
    sshd 417861 werner 15u IPv4 3093870 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->OpenWrt.lan:60730 (CLOSE_WAIT)
    sshd 417861 werner 16u IPv4 3099227 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->OpenWrt.lan:36974 (CLOSE_WAIT)
    werner@Standard-PC-i440FX-PIIX-1996:~$ sudo lsof -i :21080
    COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
    sshd 417861 werner 10u IPv4 3063120 0t0 TCP *:21080 (LISTEN)
    sshd 417861 werner 11u IPv6 3063121 0t0 TCP *:21080 (LISTEN) werner@Standard-PC-i440FX-PIIX-1996:~$ lsof -i :21080 werner@Standard-PC-i440FX-PIIX-1996:~$

    Basically, when the problem happens, I will see a lot of the following information:

    $ sudo ss --tcp state CLOSE-WAIT '( sport = 20022 or dst 192.168.10.1 )' Recv-Q Send-Q Local Address:Port Peer Address:Port Process
    42 0 192.168.10.101:20022 192.168.10.1:60930
    42 0 192.168.10.101:20022 192.168.10.1:32776
    42 0 192.168.10.101:20022 192.168.10.1:60956

    Here, 192.168.10.1 is the intranet gateway interface on the router, which has a dynamic public IP address. And 192.168.10.101 is the intranet host which running sshd to do the ssh remote forwarding.


    COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
    sshd 196794 werner 12u IPv4 1409853 0t0 TCP *:20022 (LISTEN)
    sshd 196794 werner 13u IPv6 1409854 0t0 TCP *:20022 (LISTEN)
    [Rest of long output snipped]

    In the whole output I only see one process with PID 196794.
    I really not noticed that. I mean so many sshd entries belonging to the same process.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From hongyi.zhao@gmail.com@21:1/5 to hongy...@gmail.com on Fri Oct 29 21:22:02 2021
    On Thursday, October 28, 2021 at 8:32:06 PM UTC+8, hongy...@gmail.com wrote:
    On Thursday, October 28, 2021 at 7:18:17 PM UTC+8, Spiros Bousbouras wrote:
    On Thu, 28 Oct 2021 02:20:05 -0700 (PDT)
    "hongy...@gmail.com" <hongy...@gmail.com> wrote:
    On client, I run the follownig command to create a ssh based remote
    port forwarding proxy:

    $ autossh -M 0 -4 -NT \
    -o ServerAliveInterval=30 \
    -o "ServerAliveCountMax=3" \
    -o "ExitOnForwardFailure=yes" \
    -o StrictHostKeyChecking=no \
    -R 21080:localhost:1080 \
    -R 20022:localhost:22 \
    wer...@my.remote.server -p 2101

    Then on the server side, I observed there are too many sshd processes
    has been started:

    werner@Standard-PC-i440FX-PIIX-1996:~$ sudo lsof -i :20022
    Why do you need sudo to run lsof ?
    This is my misuse :-(

    Based on my further tries, only the sudo can display the results:

    werner@Standard-PC-i440FX-PIIX-1996:~$ lsof -i :20022 werner@Standard-PC-i440FX-PIIX-1996:~$ sudo lsof -i :20022
    COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
    sshd 417861 werner 12u IPv4 3063124 0t0 TCP *:20022 (LISTEN)
    sshd 417861 werner 13u IPv6 3063125 0t0 TCP *:20022 (LISTEN)
    sshd 417861 werner 14u IPv4 3092408 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->OpenWrt.lan:47194 (CLOSE_WAIT)
    sshd 417861 werner 15u IPv4 3093870 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->OpenWrt.lan:60730 (CLOSE_WAIT)
    sshd 417861 werner 16u IPv4 3099227 0t0 TCP Standard-PC-i440FX-PIIX-1996.lan:20022->OpenWrt.lan:36974 (CLOSE_WAIT)
    werner@Standard-PC-i440FX-PIIX-1996:~$ sudo lsof -i :21080
    COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
    sshd 417861 werner 10u IPv4 3063120 0t0 TCP *:21080 (LISTEN)
    sshd 417861 werner 11u IPv6 3063121 0t0 TCP *:21080 (LISTEN) werner@Standard-PC-i440FX-PIIX-1996:~$ lsof -i :21080 werner@Standard-PC-i440FX-PIIX-1996:~$



    COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
    sshd 196794 werner 12u IPv4 1409853 0t0 TCP *:20022 (LISTEN)
    sshd 196794 werner 13u IPv6 1409854 0t0 TCP *:20022 (LISTEN)
    [Rest of long output snipped]

    In the whole output I only see one process with PID 196794.
    I really not noticed that. I mean so many sshd entries belonging to the same process.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From hongyi.zhao@gmail.com@21:1/5 to hongy...@gmail.com on Fri Oct 29 22:38:53 2021
    On Friday, October 29, 2021 at 9:34:21 AM UTC+8, hongy...@gmail.com wrote:
    On Friday, October 29, 2021 at 2:47:16 AM UTC+8, Grant Taylor wrote:
    On 10/28/21 3:20 AM, hongy...@gmail.com wrote:
    On client, I run the follownig command to create a ssh based remote
    port forwarding proxy:

    $ autossh -M 0 -4 -NT \
    -o ServerAliveInterval=30 \
    -o "ServerAliveCountMax=3" \
    -o "ExitOnForwardFailure=yes" \
    -o StrictHostKeyChecking=no \
    -R 21080:localhost:1080 \
    -R 20022:localhost:22 \
    wer...@my.remote.server -p 2101

    Then on the server side, I observed there are too many sshd processes has been started:
    As Spiros pointed out, they are all the same sshd process; 196794.
    werner@Standard-PC-i440FX-PIIX-1996:~$ sudo lsof -i :20022
    COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
    If I've tracked the topology correctly, those all look like 143.198.101.180 connected to port 20022 on the remote server which is forwarded through SSH to port 22 on the client.

    This seems like an SSH attack from 143.198.101.180 to me. Maybe
    password guessing / brute forcing.
    And this will make SSH forwarding unavailable. Any hints for fixing
    this problem?
    I'm guessing that you're either tripping a security feature (number of total connections, number in a given time, number from a specific
    source, etc.). I don't see how this would actually prevent the (remote) port forwarding from working with SSH. This seems like a denial of
    service type issue.
    Thank you for your in-depth analysis. Basically, my goal is to achieve intranet penetration without public IP. It seems that the ssh remote port forwarding method discussed here is not carefully designed for this purpose. So, maybe I should switch to
    zerotier-one [1].

    [1] https://github.com/zerotier/ZeroTierOne

    I find another wonderful website which collected and organized most of the open-source tools on this topic:

    https://book.hacktricks.xyz/tunneling-and-port-forwarding

    I'm currently studying the following tools mentioned by the above website:

    https://github.com/jpillora/chisel
    https://github.com/sshuttle/sshuttle

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From hongyi.zhao@gmail.com@21:1/5 to hongy...@gmail.com on Sat Oct 30 02:16:39 2021
    On Saturday, October 30, 2021 at 1:38:55 PM UTC+8, hongy...@gmail.com wrote:
    On Friday, October 29, 2021 at 9:34:21 AM UTC+8, hongy...@gmail.com wrote:
    On Friday, October 29, 2021 at 2:47:16 AM UTC+8, Grant Taylor wrote:
    On 10/28/21 3:20 AM, hongy...@gmail.com wrote:
    On client, I run the follownig command to create a ssh based remote port forwarding proxy:

    $ autossh -M 0 -4 -NT \
    -o ServerAliveInterval=30 \
    -o "ServerAliveCountMax=3" \
    -o "ExitOnForwardFailure=yes" \
    -o StrictHostKeyChecking=no \
    -R 21080:localhost:1080 \
    -R 20022:localhost:22 \
    wer...@my.remote.server -p 2101

    Then on the server side, I observed there are too many sshd processes has been started:
    As Spiros pointed out, they are all the same sshd process; 196794.
    werner@Standard-PC-i440FX-PIIX-1996:~$ sudo lsof -i :20022
    COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
    If I've tracked the topology correctly, those all look like 143.198.101.180 connected to port 20022 on the remote server which is forwarded through SSH to port 22 on the client.

    This seems like an SSH attack from 143.198.101.180 to me. Maybe
    password guessing / brute forcing.
    And this will make SSH forwarding unavailable. Any hints for fixing this problem?
    I'm guessing that you're either tripping a security feature (number of total connections, number in a given time, number from a specific source, etc.). I don't see how this would actually prevent the (remote) port forwarding from working with SSH. This seems like a denial of service type issue.
    Thank you for your in-depth analysis. Basically, my goal is to achieve intranet penetration without public IP. It seems that the ssh remote port forwarding method discussed here is not carefully designed for this purpose. So, maybe I should switch to
    zerotier-one [1].

    [1] https://github.com/zerotier/ZeroTierOne
    I find another wonderful website which collected and organized most of the open-source tools on this topic:

    https://book.hacktricks.xyz/tunneling-and-port-forwarding

    The author's GitHub repository is here:

    https://github.com/carlospolop/hacktricks.git


    I'm currently studying the following tools mentioned by the above website:

    https://github.com/jpillora/chisel
    https://github.com/sshuttle/sshuttle

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Grant Taylor@21:1/5 to hongy...@gmail.com on Sat Oct 30 17:24:52 2021
    On 10/29/21 10:45 PM, hongy...@gmail.com wrote:
    Basically, when the problem happens, I will see a lot of the following information:

    How many is "a lot of"? 10s, 100s, 1,000s? You may be running out
    available sockets. There are all sorts of tunables around how long
    stocks stay in CLOSE-WAIT and related states. The idea is turn over the sockets faster.

    $ sudo ss --tcp state CLOSE-WAIT '( sport = 20022 or dst 192.168.10.1 )' Recv-Q Send-Q Local Address:Port Peer Address:Port Process
    42 0 192.168.10.101:20022 192.168.10.1:60930
    42 0 192.168.10.101:20022 192.168.10.1:32776
    42 0 192.168.10.101:20022 192.168.10.1:60956

    Here, 192.168.10.1 is the intranet gateway interface on the router,
    which has a dynamic public IP address. And 192.168.10.101 is the
    intranet host which running sshd to do the ssh remote forwarding.

    Why is your internal system seeing the intranet gateway as the (remote)
    source IP? I would expect it to see the real (remote) client's IP
    address. This hints at a different configuration that I would not use
    unless I had to.

    Traditional port forwarding usually shows the real (remote) client's IP address, not the intranet gateway's IP address. This makes me think
    that the intranet gateway is SNATing incoming traffic to it's own IP. I
    would not do that unless I had a very specific reason to do so.



    --
    Grant. . . .
    unix || die

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From hongyi.zhao@gmail.com@21:1/5 to Grant Taylor on Sat Oct 30 18:52:17 2021
    On Sunday, October 31, 2021 at 7:24:41 AM UTC+8, Grant Taylor wrote:
    On 10/29/21 10:45 PM, hongy...@gmail.com wrote:
    Basically, when the problem happens, I will see a lot of the following information:
    How many is "a lot of"? 10s, 100s, 1,000s? You may be running out
    available sockets. There are all sorts of tunables around how long
    stocks stay in CLOSE-WAIT and related states. The idea is turn over the sockets faster.
    $ sudo ss --tcp state CLOSE-WAIT '( sport = 20022 or dst 192.168.10.1 )' Recv-Q Send-Q Local Address:Port Peer Address:Port Process
    42 0 192.168.10.101:20022 192.168.10.1:60930
    42 0 192.168.10.101:20022 192.168.10.1:32776
    42 0 192.168.10.101:20022 192.168.10.1:60956

    Here, 192.168.10.1 is the intranet gateway interface on the router,
    which has a dynamic public IP address. And 192.168.10.101 is the
    intranet host which running sshd to do the ssh remote forwarding.
    Why is your internal system seeing the intranet gateway as the (remote) source IP? I would expect it to see the real (remote) client's IP
    address. This hints at a different configuration that I would not use
    unless I had to.

    The router OS is OpenWrt for my case. This should be controlled by the following setting:

    Loopback source IP
    Specifies whether to use the external or the internal IP address for reflected traffic.


    Traditional port forwarding usually shows the real (remote) client's IP address, not the intranet gateway's IP address. This makes me think
    that the intranet gateway is SNATing incoming traffic to it's own IP. I would not do that unless I had a very specific reason to do so.

    In my case, the sshd is running on an intranet host residing on the local subnetwork with the gateway 192.168.10.1, which is one of the interface of the OpenWrt router. And I set the DNAT rule on the router to forward the ssh client request to the sshd
    daemon process.

    HZ

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From hongyi.zhao@gmail.com@21:1/5 to hongy...@gmail.com on Sat Oct 30 19:13:33 2021
    On Sunday, October 31, 2021 at 9:52:19 AM UTC+8, hongy...@gmail.com wrote:
    On Sunday, October 31, 2021 at 7:24:41 AM UTC+8, Grant Taylor wrote:
    On 10/29/21 10:45 PM, hongy...@gmail.com wrote:
    Basically, when the problem happens, I will see a lot of the following information:
    How many is "a lot of"? 10s, 100s, 1,000s? You may be running out available sockets. There are all sorts of tunables around how long
    stocks stay in CLOSE-WAIT and related states. The idea is turn over the sockets faster.
    $ sudo ss --tcp state CLOSE-WAIT '( sport = 20022 or dst 192.168.10.1 )' Recv-Q Send-Q Local Address:Port Peer Address:Port Process
    42 0 192.168.10.101:20022 192.168.10.1:60930
    42 0 192.168.10.101:20022 192.168.10.1:32776
    42 0 192.168.10.101:20022 192.168.10.1:60956

    Here, 192.168.10.1 is the intranet gateway interface on the router, which has a dynamic public IP address. And 192.168.10.101 is the intranet host which running sshd to do the ssh remote forwarding.
    Why is your internal system seeing the intranet gateway as the (remote) source IP? I would expect it to see the real (remote) client's IP
    address. This hints at a different configuration that I would not use unless I had to.
    The router OS is OpenWrt for my case. This should be controlled by the following setting:

    Loopback source IP
    Specifies whether to use the external or the internal IP address for reflected traffic.

    If I use the following setting: Use external IP address, then I will see the following result reported by ss:

    $ sudo ss --tcp
    State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
    ESTAB 0 0 192.168.10.101:ssh 60.6.xxx.xxx:40736
    ESTAB 0 0 192.168.10.101:ssh 106.9.xxx.xxx:60126



    Traditional port forwarding usually shows the real (remote) client's IP address, not the intranet gateway's IP address. This makes me think
    that the intranet gateway is SNATing incoming traffic to it's own IP. I would not do that unless I had a very specific reason to do so.
    In my case, the sshd is running on an intranet host residing on the local subnetwork with the gateway 192.168.10.1, which is one of the interface of the OpenWrt router. And I set the DNAT rule on the router to forward the ssh client request to the sshd
    daemon process.

    HZ

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From hongyi.zhao@gmail.com@21:1/5 to Grant Taylor on Sat Oct 30 20:56:16 2021
    On Friday, October 29, 2021 at 2:47:16 AM UTC+8, Grant Taylor wrote:
    On 10/28/21 3:20 AM, hongy...@gmail.com wrote:
    On client, I run the follownig command to create a ssh based remote
    port forwarding proxy:

    $ autossh -M 0 -4 -NT \
    -o ServerAliveInterval=30 \
    -o "ServerAliveCountMax=3" \
    -o "ExitOnForwardFailure=yes" \
    -o StrictHostKeyChecking=no \
    -R 21080:localhost:1080 \
    -R 20022:localhost:22 \
    wer...@my.remote.server -p 2101

    Then on the server side, I observed there are too many sshd processes
    has been started:
    As Spiros pointed out, they are all the same sshd process; 196794.
    werner@Standard-PC-i440FX-PIIX-1996:~$ sudo lsof -i :20022
    COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
    If I've tracked the topology correctly, those all look like
    143.198.101.180 connected to port 20022 on the remote server which is forwarded through SSH to port 22 on the client.

    This seems like an SSH attack from 143.198.101.180 to me. Maybe
    password guessing / brute forcing.

    Really. This is absolutely not the connection initiated by myself.

    And this will make SSH forwarding unavailable. Any hints for fixing
    this problem?
    I'm guessing that you're either tripping a security feature (number of
    total connections, number in a given time, number from a specific
    source, etc.). I don't see how this would actually prevent the (remote)
    port forwarding from working with SSH. This seems like a denial of
    service type issue.

    There seems to be a security risk in my OpenWrt firewall. I now set the following traffic rules in `/etc/config/firewall', which seems fixed the problem:

    config rule
    option src 'wan'
    option dest '*'
    option target 'DROP'
    list src_ip '143.110.224.0/20'
    list src_ip '143.198.96.0/20'

    HZ

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From hongyi.zhao@gmail.com@21:1/5 to hongy...@gmail.com on Sat Oct 30 21:59:03 2021
    On Sunday, October 31, 2021 at 11:56:19 AM UTC+8, hongy...@gmail.com wrote:
    On Friday, October 29, 2021 at 2:47:16 AM UTC+8, Grant Taylor wrote:
    On 10/28/21 3:20 AM, hongy...@gmail.com wrote:
    On client, I run the follownig command to create a ssh based remote
    port forwarding proxy:

    $ autossh -M 0 -4 -NT \
    -o ServerAliveInterval=30 \
    -o "ServerAliveCountMax=3" \
    -o "ExitOnForwardFailure=yes" \
    -o StrictHostKeyChecking=no \
    -R 21080:localhost:1080 \
    -R 20022:localhost:22 \
    wer...@my.remote.server -p 2101

    Then on the server side, I observed there are too many sshd processes
    has been started:
    As Spiros pointed out, they are all the same sshd process; 196794.
    werner@Standard-PC-i440FX-PIIX-1996:~$ sudo lsof -i :20022
    COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
    If I've tracked the topology correctly, those all look like
    143.198.101.180 connected to port 20022 on the remote server which is forwarded through SSH to port 22 on the client.

    This seems like an SSH attack from 143.198.101.180 to me. Maybe
    password guessing / brute forcing.
    Really. This is absolutely not the connection initiated by myself.
    And this will make SSH forwarding unavailable. Any hints for fixing
    this problem?
    I'm guessing that you're either tripping a security feature (number of total connections, number in a given time, number from a specific
    source, etc.). I don't see how this would actually prevent the (remote) port forwarding from working with SSH. This seems like a denial of
    service type issue.
    There seems to be a security risk in my OpenWrt firewall. I now set the following traffic rules in `/etc/config/firewall', which seems fixed the problem:

    config rule
    option src 'wan'
    option dest '*'
    option target 'DROP'
    list src_ip '143.110.224.0/20'
    list src_ip '143.198.96.0/20'

    I have changed to the better method below:

    config redirect
    option target 'DNAT'
    list proto 'tcp'
    option src 'wan'
    option src_dport '20022'
    option dest 'lan'
    option dest_ip '192.168.10.101'
    option dest_port '20022'
    option name 'autossh-remote-forward-ssh-pve-ubuntu'
    option reflection_src 'external'
    option src_dip '106.9.xxx.xxx'
    option src_ip '60.0.0.0/13'

    The CIDR address ranges used above is obtained from https://ipinfo.io/

    HZ

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From hongyi.zhao@gmail.com@21:1/5 to hongy...@gmail.com on Sun Oct 31 01:42:32 2021
    On Saturday, October 30, 2021 at 5:16:42 PM UTC+8, hongy...@gmail.com wrote:
    On Saturday, October 30, 2021 at 1:38:55 PM UTC+8, hongy...@gmail.com wrote:
    On Friday, October 29, 2021 at 9:34:21 AM UTC+8, hongy...@gmail.com wrote:
    On Friday, October 29, 2021 at 2:47:16 AM UTC+8, Grant Taylor wrote:
    On 10/28/21 3:20 AM, hongy...@gmail.com wrote:
    On client, I run the follownig command to create a ssh based remote port forwarding proxy:

    $ autossh -M 0 -4 -NT \
    -o ServerAliveInterval=30 \
    -o "ServerAliveCountMax=3" \
    -o "ExitOnForwardFailure=yes" \
    -o StrictHostKeyChecking=no \
    -R 21080:localhost:1080 \
    -R 20022:localhost:22 \
    wer...@my.remote.server -p 2101

    Then on the server side, I observed there are too many sshd processes
    has been started:
    As Spiros pointed out, they are all the same sshd process; 196794.
    werner@Standard-PC-i440FX-PIIX-1996:~$ sudo lsof -i :20022
    COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
    If I've tracked the topology correctly, those all look like 143.198.101.180 connected to port 20022 on the remote server which is forwarded through SSH to port 22 on the client.

    This seems like an SSH attack from 143.198.101.180 to me. Maybe password guessing / brute forcing.
    And this will make SSH forwarding unavailable. Any hints for fixing this problem?
    I'm guessing that you're either tripping a security feature (number of total connections, number in a given time, number from a specific source, etc.). I don't see how this would actually prevent the (remote)
    port forwarding from working with SSH. This seems like a denial of service type issue.
    Thank you for your in-depth analysis. Basically, my goal is to achieve intranet penetration without public IP. It seems that the ssh remote port forwarding method discussed here is not carefully designed for this purpose. So, maybe I should switch
    to zerotier-one [1].

    [1] https://github.com/zerotier/ZeroTierOne
    I find another wonderful website which collected and organized most of the open-source tools on this topic:

    https://book.hacktricks.xyz/tunneling-and-port-forwarding
    The author's GitHub repository is here:

    https://github.com/carlospolop/hacktricks.git

    Here is another similar wonderful GitHub repository:

    https://github.com/anderspitman/awesome-tunneling


    I'm currently studying the following tools mentioned by the above website:

    https://github.com/jpillora/chisel
    https://github.com/sshuttle/sshuttle

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Grant Taylor@21:1/5 to hongy...@gmail.com on Sun Oct 31 12:41:05 2021
    On 10/30/21 8:13 PM, hongy...@gmail.com wrote:
    If I use the following setting: Use external IP address, then I will
    see the following result reported by ss:

    It seems as if you have found and corrected what I was describing as a
    sub-par configuration.

    $ sudo ss --tcp
    State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
    ESTAB 0 0 192.168.10.101:ssh 60.6.xxx.xxx:40736
    ESTAB 0 0 192.168.10.101:ssh 106.9.xxx.xxx:60126

    That is exactly the type of output that I would expect.

    I wonder if you will have any connection issues now. -- My thought is
    that there was a security setting somewhere that you were tickling
    related to the maximum number of connections from a singular IP. Now
    the connections will appear to be from their real IP, not your gateway's IP.



    --
    Grant. . . .
    unix || die

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Grant Taylor@21:1/5 to hongy...@gmail.com on Sun Oct 31 12:38:51 2021
    On 10/30/21 9:56 PM, hongy...@gmail.com wrote:
    Really. This is absolutely not the connection initiated by myself.

    I assumed that it wasn't you initiating the connection(s).

    But based on the output of netstat / ss, that's what the connection
    looks like. I assume that is a false appearance based on a ... sub-par configuration somewhere.



    --
    Grant. . . .
    unix || die

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From hongyi.zhao@gmail.com@21:1/5 to Grant Taylor on Sun Oct 31 19:22:03 2021
    On Monday, November 1, 2021 at 2:41:10 AM UTC+8, Grant Taylor wrote:
    On 10/30/21 8:13 PM, hongy...@gmail.com wrote:
    If I use the following setting: Use external IP address, then I will
    see the following result reported by ss:
    It seems as if you have found and corrected what I was describing as a sub-par configuration.
    $ sudo ss --tcp
    State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
    ESTAB 0 0 192.168.10.101:ssh 60.6.xxx.xxx:40736
    ESTAB 0 0 192.168.10.101:ssh 106.9.xxx.xxx:60126
    That is exactly the type of output that I would expect.

    I wonder if you will have any connection issues now. -- My thought is
    that there was a security setting somewhere that you were tickling
    related to the maximum number of connections from a singular IP. Now
    the connections will appear to be from their real IP, not your gateway's IP.

    Do you mean something like the following?

    root@OpenWrt:~# sysctl net.nf_conntrack_max
    net.nf_conntrack_max = 16384

    HZ

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Grant Taylor@21:1/5 to hongy...@gmail.com on Sun Oct 31 22:43:23 2021
    On 10/31/21 8:22 PM, hongy...@gmail.com wrote:
    Do you mean something like the following?

    root@OpenWrt:~# sysctl net.nf_conntrack_max
    net.nf_conntrack_max = 16384

    That's not the setting that I was thinking of. But, yes, it's within
    the group of settings.

    tcp_fin_timeout seems to be one of the values I was thinking of.

    There are a number of TCP related sys controls.

    Link - IP Sysctl - /proc/sys/net/ipv4/* Variables
    - https://www.kernel.org/doc/html/latest/networking/ip-sysctl.html

    Admittedly, I've so rarely needed to change any timeout settings.
    Plenty of other settings, but not timeout related.



    --
    Grant. . . .
    unix || die

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From hongyi.zhao@gmail.com@21:1/5 to Grant Taylor on Mon Nov 1 01:41:18 2021
    On Monday, November 1, 2021 at 12:43:28 PM UTC+8, Grant Taylor wrote:
    On 10/31/21 8:22 PM, hongy...@gmail.com wrote:
    Do you mean something like the following?

    root@OpenWrt:~# sysctl net.nf_conntrack_max
    net.nf_conntrack_max = 16384
    That's not the setting that I was thinking of. But, yes, it's within
    the group of settings.

    tcp_fin_timeout seems to be one of the values I was thinking of.

    There are a number of TCP related sys controls.

    Link - IP Sysctl - /proc/sys/net/ipv4/* Variables
    - https://www.kernel.org/doc/html/latest/networking/ip-sysctl.html

    Admittedly, I've so rarely needed to change any timeout settings.
    Plenty of other settings, but not timeout related.

    Me too.

    BTW, now, I use the following tool to complement the SSH based approach discussed here:

    https://github.com/fatedier/frp

    HZ

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)