Hello,
I have setup an Ubuntu 20.04 virtual machine on my second host (host2). When I ssh into the vm from host2 everything works.
But when I ssh into the vm from my main host (Host1) it gets stuck before asking for the password.

ssh verbose (ssh -vvv name@server) gives the output,

debug2: ssh_connect_direct
debug1: Connecting to 192.168.150.139 [192.168.150.139] port 22.
debug1: Connection established.
....
debug1: Local version string SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.4
END OF OUTPUT! Her it stucks.

Each of the systems are in different network segments. VM has a bridged network to the network adapter of its host.

Any help will be great.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

PING 192.168.150.139 (192.168.150.139) 56(84) Bytes Daten.
64 Bytes von 192.168.150.139: icmp_seq=1 ttl=63 Zeit=1.30 ms
Von 192.168.1.1 icmp_seq=2 Host umleiten(New nexthop: 18.1.168.192)
64 Bytes von 192.168.150.139: icmp_seq=2 ttl=63 Zeit=0.988 ms
Von 192.168.1.1 icmp_seq=3 Host umleiten(New nexthop: 18.1.168.192)
64 Bytes von 192.168.150.139: icmp_seq=3 ttl=63 Zeit=1.11 ms
Von 192.168.1.1 icmp_seq=4 Host umleiten(New nexthop: 18.1.168.192)
64 Bytes von 192.168.150.139: icmp_seq=4 ttl=63 Zeit=1.06 ms
^C
--- 192.168.150.139 ping statistics ---
4 Pakete übertragen, 4 empfangen, +3 Fehler, 0% Paketverlust, Zeit 3004ms
rtt min/avg/max/mdev = 0.988/1.114/1.296/0.113 ms

I'm not a network expert but where does this nexthop come from and is that the reason for that behaviour?




bob prohaska schrieb am Samstag, 19. Februar 2022 um 20:34:34 UTC+1:

Knarf Reueh <knarf...@gmail.com> wrote:

Hello,
I have setup an Ubuntu 20.04 virtual machine on my second host (host2). When I ssh into the vm from host2 everything works.
But when I ssh into the vm from my main host (Host1) it gets stuck before asking for the password.

ssh verbose (ssh -vvv name@server) gives the output,

debug2: ssh_connect_direct
debug1: Connecting to 192.168.150.139 [192.168.150.139] port 22.
debug1: Connection established.
....
debug1: Local version string SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.4
END OF OUTPUT! Her it stucks.

Each of the systems are in diff

Knarf Reueh <knarf.reueh@gmail.com> wrote:

Hello,
I have setup an Ubuntu 20.04 virtual machine on my second host (host2). When I ssh into the vm from host2 everything works.
But when I ssh into the vm from my main host (Host1) it gets stuck before asking for the password.

ssh verbose (ssh -vvv name@server) gives the output,

debug2: ssh_connect_direct
debug1: Connecting to 192.168.150.139 [192.168.150.139] port 22.
debug1: Connection established.
....
debug1: Local version string SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.4
END OF OUTPUT! Her it stucks.

Each of the systems are in different network segments. VM has a bridged network to the network adapter of its host.

How does ping behave?

hth,

bob prohaska

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Knarf Reueh <knarf.reueh@gmail.com> wrote:

PING 192.168.150.139 (192.168.150.139) 56(84) Bytes Daten.
64 Bytes von 192.168.150.139: icmp_seq=1 ttl=63 Zeit=1.30 ms
Von 192.168.1.1 icmp_seq=2 Host umleiten(New nexthop: 18.1.168.192)
64 Bytes von 192.168.150.139: icmp_seq=2 ttl=63 Zeit=0.988 ms
Von 192.168.1.1 icmp_seq=3 Host umleiten(New nexthop: 18.1.168.192)
64 Bytes von 192.168.150.139: icmp_seq=3 ttl=63 Zeit=1.11 ms
Von 192.168.1.1 icmp_seq=4 Host umleiten(New nexthop: 18.1.168.192)
64 Bytes von 192.168.150.139: icmp_seq=4 ttl=63 Zeit=1.06 ms
^C
--- 192.168.150.139 ping statistics ---
4 Pakete ?bertragen, 4 empfangen, +3 Fehler, 0% Paketverlust, Zeit 3004ms
rtt min/avg/max/mdev = 0.988/1.114/1.296/0.113 ms

I'm not a network expert but where does this nexthop come from and is that the reason for that behaviour?

I'm no expert either and have no idea what nexthop is. Never seen it on
any of my machines, nothing in the man pages. Perhaps Ubuntu specific?

I suggested the test only because of a problem I've been having with
ssh on FreeBSD: The first symptom was an ssh login stalling after
asking for and receiving a password, with the login subsequently
timing out.

It eventually developed that my machines weren't responnding correctly
to ping, returning only 1% of packets. If that was the case for you it
would be interesting, but clearly you are seeing something different.

Sorry I can't be more help!

bob prohaska

bob prohaska schrieb am Samstag, 19. Februar 2022 um 20:34:34 UTC+1:

Knarf Reueh <knarf...@gmail.com> wrote:

Hello,
I have setup an Ubuntu 20.04 virtual machine on my second host (host2). When I ssh into the vm from host2 everything works.
But when I ssh into the vm from my main host (Host1) it gets stuck before asking for the password.

ssh verbose (ssh -vvv name@server) gives the output,

debug2: ssh_connect_direct
debug1: Connecting to 192.168.150.139 [192.168.150.139] port 22.
debug1: Connection established.
....
debug1: Local version string SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.4
END OF OUTPUT! Her it stucks.

Each of the systems are in different network segments. VM has a bridged network to the network adapter of its host.

How does ping behave?

hth,

bob prohaska

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2022-02-19, bob prohaska <bp@www.zefox.net> wrote:

Knarf Reueh <knarf.reueh@gmail.com> wrote:

PING 192.168.150.139 (192.168.150.139) 56(84) Bytes Daten.
64 Bytes von 192.168.150.139: icmp_seq=1 ttl=63 Zeit=1.30 ms
Von 192.168.1.1 icmp_seq=2 Host umleiten(New nexthop: 18.1.168.192)
64 Bytes von 192.168.150.139: icmp_seq=2 ttl=63 Zeit=0.988 ms
Von 192.168.1.1 icmp_seq=3 Host umleiten(New nexthop: 18.1.168.192)
64 Bytes von 192.168.150.139: icmp_seq=3 ttl=63 Zeit=1.11 ms
Von 192.168.1.1 icmp_seq=4 Host umleiten(New nexthop: 18.1.168.192)
64 Bytes von 192.168.150.139: icmp_seq=4 ttl=63 Zeit=1.06 ms
^C
--- 192.168.150.139 ping statistics ---
4 Pakete ?bertragen, 4 empfangen, +3 Fehler, 0% Paketverlust, Zeit 3004ms
rtt min/avg/max/mdev = 0.988/1.114/1.296/0.113 ms

I'm not a network expert but where does this nexthop come from and is that the reason for that behaviour?

I'm no expert either and have no idea what nexthop is. Never seen it on
any of my machines, nothing in the man pages. Perhaps Ubuntu specific?

umleiten=redirect. Maybe 18.1.168.192 is the host for the vm

I suggested the test only because of a problem I've been having with
ssh on FreeBSD: The first symptom was an ssh login stalling after
asking for and receiving a password, with the login subsequently
timing out.

It eventually developed that my machines weren't responnding correctly
to ping, returning only 1% of packets. If that was the case for you it
would be interesting, but clearly you are seeing something different.

Well it could be. The packet weems to be being sent to 18.1.168.192
rather than to 192.168.150.139. That may confuse the ssh on the vm in
that the address the packet is being sent to is not what it thinks its
own address as, and thus gets confused.

...

bob prohaska schrieb am Samstag, 19. Februar 2022 um 20:34:34 UTC+1:

Knarf Reueh <knarf...@gmail.com> wrote:

Hello,
I have setup an Ubuntu 20.04 virtual machine on my second host (host2). When I ssh into the vm from host2 everything works.
But when I ssh into the vm from my main host (Host1) it gets stuck before asking for the password.

ssh verbose (ssh -vvv name@server) gives the output,

debug2: ssh_connect_direct
debug1: Connecting to 192.168.150.139 [192.168.150.139] port 22.
debug1: Connection established.
....
debug1: Local version string SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.4
END OF OUTPUT! Her it stucks.

Each of the systems are in different network segments. VM has a bridged network to the network adapter of its host.

How does ping behave?

hth,

bob prohaska

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2/19/22 10:07 AM, Knarf Reueh wrote:

Any help will be great.

Compare what sniffers (e.g. tcpdump) see on each end of the connection.

I've seen some strange MTU issues cause this type of problem.

I suspect that one side is sending a packet that the other side is not receiving, thereby stalling the normal flow of the exchange.

If it is an MTU issue, you can artificially lower it for the connection
on one end to see if SSH connections will establish or not.

Aside: I've also seen DNS issues induce a ~35 second delay.



--
Grant. . . .
unix || die

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hello Grant Taylor,
thanks for your reply. As I told, my knowledge about networking is basic, not advanced. I had tcpdump used before so far but was always unable to interpret the output. So ... Good Luck (see above).
I could not find any special but I found a bug in Ubuntu that looks a bit like that behavior. As described there, the ip address of the hop is printed in reverse order. To explain, here is the route as printed from ping called on Host1:
ping -R 192.168.150.139
PING 192.168.150.139 (192.168.150.139) 56(124) Bytes Daten.
64 Bytes von 192.168.150.139: icmp_seq=1 ttl=63 Zeit=1.37 ms
RR: 192.168.1.20 (Host 1)
192.168.1.1 (router and dhcp server)
192.168.150.1 ?? guess a virtual router on host2 that hosts the vm 192.168.150.139 (vm)
192.168.150.139
192.168.1.18 (host2)
192.168.1.20 (host1)

So the route backwards from ..139 to ..1.20 takes another route. The nexthop printed out by ping ist 18.1.168.192 (so in reverse order). Magic.
See the bug described here: https://bugs.launchpad.net/ubuntu/+source/iputils/+bug/1892108

And here are tcpdumps:

TCP Dump on the vm while ssh from Zuse2016 ------------------------------------------------------------------- franz@TrainUB20:~$ sudo tcpdump host 192.168.1.20
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on enp1s0, link-type EN10MB (Ethernet), capture size 262144 bytes 13:41:57.359875 IP Zuse2016.localdomain.43514 > TrainUB20.network.ssh: Flags [S], seq 2909393996, win 64240, options [mss 1460,sackOK,TS val 3232087362 ecr 0,nop,wscale 7], length 0
13:41:57.359974 IP TrainUB20.network.ssh > Zuse2016.localdomain.43514: Flags [S.], seq 2705827111, ack 2909393997, win 65160, options [mss 1460,sackOK,TS val 2859469818 ecr 3232087362,nop,wscale 7], length 0
13:41:58.383658 IP TrainUB20.network.ssh > Zuse2016.localdomain.43514: Flags [S.], seq 2705827111, ack 2909393997, win 65160, options [mss 1460,sackOK,TS val 2859470842 ecr 3232087362,nop,wscale 7], length 0
13:42:00.399674 IP TrainUB20.network.ssh > Zuse2016.localdomain.43514: Flags [S.], seq 2705827111, ack 2909393997, win 65160, options [mss 1460,sackOK,TS val 2859472858 ecr 3232087362,nop,wscale 7], length 0
13:42:04.431668 IP TrainUB20.network.ssh > Zuse2016.localdomain.43514: Flags [S.], seq 2705827111, ack 2909393997, win 65160, options [mss 1460,sackOK,TS val 2859476890 ecr 3232087362,nop,wscale 7], length 0
13:42:12.623670 IP TrainUB20.network.ssh > Zuse2016.localdomain.43514: Flags [S.], seq 2705827111, ack 2909393997, win 65160, options [mss 1460,sackOK,TS val 2859485082 ecr 3232087362,nop,wscale 7], length 0

TCP Dump from Zuse2016 ------------------------------------------------------------------
@Zuse2016:~$ sudo tcpdump host 192.168.150.139 -i enp0s31f6
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on enp0s31f6, link-type EN10MB (Ethernet), capture size 262144 bytes 13:41:57.358577 IP Zuse2016.localdomain.43514 > 192.168.150.139.ssh: Flags [S], seq 2909393996, win 64240, options [mss 1460,sackOK,TS val 3232087362 ecr 0,nop,wscale 7], length 0
13:41:57.359719 IP 192.168.150.139.ssh > Zuse2016.localdomain.43514: Flags [S.], seq 2705827111, ack 2909393997, win 65160, options [mss 1460,sackOK,TS val 2859469818 ecr 3232087362,nop,wscale 7], length 0
13:41:57.359736 IP Zuse2016.localdomain.43514 > 192.168.150.139.ssh: Flags [.], ack 1, win 502, options [nop,nop,TS val 3232087363 ecr 2859469818], length 0
13:41:57.359954 IP Zuse2016.localdomain.43514 > 192.168.150.139.ssh: Flags [P.], seq 1:42, ack 1, win 502, options [nop,nop,TS val 3232087363 ecr 2859469818], length 41
13:41:57.564064 IP Zuse2016.localdomain.43514 > 192.168.150.139.ssh: Flags [P.], seq 1:42, ack 1, win 502, options [nop,nop,TS val 3232087567 ecr 2859469818], length 41
13:41:57.772060 IP Zuse2016.localdomain.43514 > 192.168.150.139.ssh: Flags [P.], seq 1:42, ack 1, win 502, options [nop,nop,TS val 3232087775 ecr 2859469818], length 41
13:41:58.184075 IP Zuse2016.localdomain.43514 > 192.168.150.139.ssh: Flags [P.], seq 1:42, ack 1, win 502, options [nop,nop,TS val 3232088187 ecr 2859469818], length 41
13:41:58.383423 IP 192.168.150.139.ssh > Zuse2016.localdomain.43514: Flags [S.], seq 2705827111, ack 2909393997, win 65160, options [mss 1460,sackOK,TS val 2859470842 ecr 3232087362,nop,wscale 7], length 0
13:41:58.383451 IP Zuse2016.localdomain.43514 > 192.168.150.139.ssh: Flags [.], ack 1, win 502, options [nop,nop,TS val 3232088387 ecr 2859469818], length 0
13:41:59.016148 IP Zuse2016.localdomain.43514 > 192.168.150.139.ssh: Flags [P.], seq 1:42, ack 1, win 502, options [nop,nop,TS val 3232089019 ecr 2859469818], length 41
13:42:00.399470 IP 192.168.150.139.ssh > Zuse2016.localdomain.43514: Flags [S.], seq 2705827111, ack 2909393997, win 65160, options [mss 1460,sackOK,TS val 2859472858 ecr 3232087362,nop,wscale 7], length 0
13:42:00.399509 IP Zuse2016.localdomain.43514 > 192.168.150.139.ssh: Flags [.], ack 1, win 502, options [nop,nop,TS val 3232090403 ecr 2859469818], length 0
13:42:00.680063 IP Zuse2016.localdomain.43514 > 192.168.150.139.ssh: Flags [P.], seq 1:42, ack 1, win 502, options [nop,nop,TS val 3232090683 ecr 2859469818], length 41
13:42:04.200110 IP Zuse2016.localdomain.43514 > 192.168.150.139.ssh: Flags [P.], seq 1:42, ack 1, win 502, options [nop,nop,TS val 3232094203 ecr 2859469818], length 41
13:42:04.431533 IP 192.168.150.139.ssh > Zuse2016.localdomain.43514: Flags [S.], seq 2705827111, ack 2909393997, win 65160, options [mss 1460,sackOK,TS val 2859476890 ecr 3232087362,nop,wscale 7], length 0
13:42:04.431566 IP Zuse2016.localdomain.43514 > 192.168.150.139.ssh: Flags [.], ack 1, win 502, options [nop,nop,TS val 3232094435 ecr 2859469818], length 0
13:42:10.856057 IP Zuse2016.localdomain.43514 > 192.168.150.139.ssh: Flags [P.], seq 1:42, ack 1, win 502, options [nop,nop,TS val 3232100859 ecr 2859469818], length 41

So, any idea?
And: How to change the MTU for the communication?

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2/20/22 9:30 AM, Knarf Reueh wrote:

Hello Grant Taylor,

Hi,

thanks for your reply. As I told, my knowledge about networking is
basic, not advanced. I had tcpdump used before so far but was always
unable to interpret the output. So ... Good Luck (see above).

You're welcome.

No time like the present to learn something new.

I could not find any special but I found a bug in Ubuntu that looks
a bit like that behavior. As described there, the ip address of the
hop is printed in reverse order. To explain, here is the route as
printed from ping called on Host1:

ping -R 192.168.150.139
PING 192.168.150.139 (192.168.150.139) 56(124) Bytes Daten.
64 Bytes von 192.168.150.139: icmp_seq=1 ttl=63 Zeit=1.37 ms
RR: 192.168.1.20 (Host 1)
192.168.1.1 (router and dhcp server)
192.168.150.1 ?? guess a virtual router on host2 that hosts the vm 192.168.150.139 (vm)
192.168.150.139
192.168.1.18 (host2)
192.168.1.20 (host1)

So the route backwards from ..139 to ..1.20 takes another route. The
nexthop printed out by ping ist 18.1.168.192 (so in reverse
order). Magic.

I'm surprised that there are different routes for such a short path.

I wonder if this is simply the reporting of the opposite interfaces.
I'm used to replies from the interface facing the source of the traffic.

[R1 a]---[b R2 c]---[d R3]

E.g. R1 will see replies from the b interface on R2 when tracing to R3. Similarly, R3 will see replies from the c interface on R2 when tracing
to R1.

See the bug described here: https://bugs.launchpad.net/ubuntu/+source/iputils/+bug/1892108

That bug seems to be the way that a particular utility is printing the
output of a specific string. This seems to be unrelated to the problem
you describe above.

Aside: Any time you get an ICMP Redirect, that is an indicating that
you have a sub-optimal routing configuration or a stale routing
configuration or an active attack.

And here are tcpdumps:

tcpdump output removed for brevity.

Aside: Please re-run the tcpdump commands with the "-nn" option to
disable the IP to name resolution. That will make sure that we know the
source & destination of the packets. -- This is important because
there are scenarios where this resolution may be misleading. -- Ask if
you want clarification.

My immediate concern from the two tcpdump outputs is that the first
seems to be showing unidirectional traffic. While the second seems to
be showing bidirectional traffic. This is unexpected to me.

Aside: I'd be tempted to also add another "host" parameter to the
tcpdump command line:

tcpdump -nn host 192.168.1.20 and host 192.168.150.139

Further aside: I'm notorious for adding the "-i ..." interface to the
command line, but I'm omitting it so that it's possible to use the exact
same command on both systems.

So, any idea?

No, not yet.

Please collect new tcpdump data. If at all possible, start the tcpdump sessions /before/ starting the ssh client. This should enable capturing
the TCP three way handshake.

I don't know how many lines of output it would be, but a tcpdump of the
entire ssh session might be helpful. As in from the TCP connection establishment through connection termination.

And: How to change the MTU for the communication?

I usually will rely on iptables TCPMSS target to modify MTU for specific connections. -- I usually have to look it up because I do it infrequently.



--
Grant. . . .
unix || die

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Dear Grant Tailor,

it's very kind that you are helping me. Now it's Monday till Friday and I just have time to do more analysis at evening or even only the next day. So my answers will not arrive in short time.
Maybe that I find time today, to do the tcpdumps, but I think it will be Wednesday evening ( Germany). Hope you will have an eye on it.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hello,

after talking to a friend who is a network specialist I came to the decision, that I first have to setup my network the right way. He told me, that I could not set up a bridged network the way I did and have to assign different IP addresses to the same
link (or something like that).
Anyway: The observed behavior of ssh has nothing to do with ssh itself so my problem is more network related and posting these things here is out of topic.

So thanks everybody for your support.

Regards

Frank

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2/25/22 12:58 PM, Knarf Reueh wrote:

Hello,

Hi,

after talking to a friend who is a network specialist I came to the
decision, that I first have to setup my network the right way. He
told me, that I could not set up a bridged network the way I did and
have to assign different IP addresses to the same link (or something
like that).

As someone who has bridged a lot of different network segments together
using many different technologies, I strongly question the veracity of
your friends statement.

Anyway: The observed behavior of ssh has nothing to do with ssh itself
so my problem is more network related and posting these things here
is out of topic.

If you want to discuss it further, feel free to email men. Or reply
here. -- I feel like there's not a lot of traffic in this newsgroup.
So having some won't be bad. I'd just ask that you prefix your new
thread with "OT" and that we keep it as a thread so that people can
ignore it if they want to not see it.

So thanks everybody for your support.

You're welcome and good luck.



--
Grant. . . .
unix || die

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)