• [Samba] kerberos issue (SPN not found) with windows Hyper-V ( samba

    From Kacper Wirski via samba@21:1/5 to All on Sat Mar 18 14:10:02 2017
    I made some progress with the issue, but didn't solve it completely

    It's basically a kind of bug (i'm not sure if it's on kerberos side or
    samba, I think samba is the culprit here (?).

    Microsoft uses kind of weird SPN for Hyper-V. Weird as there are
    "spaces" in the string - which is kind of unique as far as SPN's go,
    usually SPN form a complete string.

    So I kind of just tried the simplest solution:

    The workaround/fix is this:

    In AD for each Hyper-V host account (machine account that is) in servicePrincipalName attribute as such:

    Hyper-V\ Replica\ Service/<NetbiosName>
    Hyper-V\ Replica\ Service/<FQDN>
    Microsoft\ Virtual\ Console\ Service/<NetbiosName>
    Microsoft\ Virtual\ Console\ Service/FQDN>
    Microsoft\ Virtual\ System\ Migration\ Service/<NetbiosName>
    Microsoft \Virtual\ System\ Migration\ Service/FQDN>

    I edited with ADUC from RSAT suite, but I guess any method will work (samba-tool add spn or windows "setspn -S".

    So basically just add \ after each part of SPN which precedes " ". With
    this settings hyper-v replica and live migration from SOURCE HOST work,
    but i'm stuck at Constrained Delegation.


    I moved my test setup to windows server 2016 and windows server 2016
    hyper-v (free).

    Constrained delegation is setup (with SPN's set as above), protocol in
    hyper-v is set to Kerberos, constrained delegations are used in accord
    to WIndow sserver 2016 specific (in ADUC -> machine account ->
    delegation -> use specific services with ANY PROTOCOL / in pre-2016 it
    was Kerberos Only/ -> choose hosts and SPN's as set above).

    When I'm doing live migration for VM, when logged in at source host it
    works perfectly
    When i'm trying to live migrate VM from remote hyper-v to the one i'm
    logged in I get authentication error. The weirdest thing is the samba
    log, it boggles my mind and if anyone has any idea I'll be very thankful

    notes: BMSRV2$ is machine added to domain with Hyper-V with all spn's
    and settings
    kacper_wirski is DOMAIN USER account
    and in the log there is clearly: "
    /Kerberos: Server not found in database: kacper_wirski@MYDOMAIN.COM.XYZ:
    No such entry in the database/*

    *Which is absurd, as obviously this account exists and is all well and
    fine. Relevant samba log below. When i do from console kinit kacper_wirski@MYDOMAIN.COM.XYZ i have no trouble obtaining ticket etc.
    Maybe constrained delegation should be setup differently (i.e. microsoft guidelines should be implemented differently for samba AD?).
    I tried with different DOMAIN ADMIN account on different host and exact
    same issue with same error in log (root@MYDOMAIN.COM.XYZ: No such entry
    in the database)
    *
    *
    Kerberos: TGS-REQ BMSRV2$@MYDOMAIN.COM.XYZ from ipv4:192.168.1.8:57775
    for kacper_wirski@MYDOMAIN.COM.XYZ [canonicalize, renewable, forwardable] [2017/03/18 13:24:37.782732, 3] ../source4/auth/kerberos/krb5_init_context.c:80(smb_krb5_debug_wrapper)
    Kerberos: samba_kdc_fetch: message2entry failed
    [2017/03/18 13:24:37.782776, 3] ../source4/auth/kerberos/krb5_init_context.c:80(smb_krb5_debug_wrapper)
    Kerberos: Searching referral for kacper_wirski
    [2017/03/18 13:24:37.782800, 3] ../source4/auth/kerberos/krb5_init_context.c:80(smb_krb5_debug_wrapper)
    Kerberos: Server not found in database:
    kacper_wirski@MYDOMAIN.COM.XYZ: No such entry in the database
    [2017/03/18 13:24:37.782819, 3] ../source4/auth/kerberos/krb5_init_context.c:80(smb_krb5_debug_wrapper)
    Kerberos: Failed building TGS-REP to ipv4:192.168.1.8:57775
    [2017/03/18 13:24:37.784201, 3] ../source4/auth/kerberos/krb5_init_context.c:80(smb_krb5_debug_wrapper)
    Kerberos: TGS-REQ BMSRV2$@MYDOMAIN.COM.XYZ from
    ipv4:192.168.1.8:57776 for kacper_wirski@MYDOMAIN.COM.XYZ [canonicalize, renewable, forwardable]
    [2017/03/18 13:24:37.785264, 3] ../source4/auth/kerberos/krb5_init_context.c:80(smb_krb5_debug_wrapper)
    Kerberos: samba_kdc_fetch: message2entry failed
    [2017/03/18 13:24:37.785308, 3] ../source4/auth/kerberos/krb5_init_context.c:80(smb_krb5_debug_wrapper)
    Kerberos: Searching referral for kacper_wirski
    [2017/03/18 13:24:37.785332, 3] ../source4/auth/kerberos/krb5_init_context.c:80(smb_krb5_debug_wrapper)
    Kerberos: Server not found in database:
    kacper_wirski@MYDOMAIN.COM.XYZ: No such entry in the database
    [2017/03/18 13:24:37.785352, 3] ../source4/auth/kerberos/krb5_init_context.c:80(smb_krb5_debug_wrapper)
    Kerberos: Failed building TGS-REP to ipv4:192.168.1.8:57776

    Dnia 2017-03-16 12:17 Kacper Wirski via samba napisał(a):

    Hello,

    I've setup over 6 months ago samba 4 AD on centos 7.3 (self compiled
    from source). Up until now I didn't encounter any undocumented errors. I
    have 3 DC's (all samba 4.5.3) which are working pretty nice with over 60
    windows clients.

    The issue I've stumbled upon is when I added Windows server Hyper-V
    hosts to the domain. Tried with Hyper-V from 2012, 2012r2 and new 2016 -
    all exact same problem.

    I've searched and googled and found one old topic with the same issue in
    samba lists, but no help was given, but also - not enough info was
    supplied.

    The main issue is that Hyper-v Hosts are unable to authenticate each
    other using kerberos for live migration and replication (only two
    features that require kerberos) - windows host gives well documented
    error, that it's unable to authenticate using kerberos.

    I've gathered all the logs, which I think explain the issue quite
    clearly and hopefully someone will be able to give a viable solution.

    domain/realm let's call it:
    mydomain.com.xyz @ MYDOMAIN.COM.XYZ
    hyper-v hosts:
    BM-SRV-5 and BMSRV-WIN10 (both with windows server 2016 standard with
    hyper-v host role installed)
    DC1, DC2, DC3 are my 3 domain controllers (names not really original
    :) )

    Microsoft Hyper-V requires specific SPN's registered for hosts:

    *Microsoft Virtual Console Service**
    **Hyper-V Replica Service**
    **Microsoft Virtual System Migration Service*

    The SPN's should be automatically registered in the AD machine account
    by the windows, but this fails with windows error 14050. This error is
    well documented, but none of the solutions helped, and I think the error
    is with samba AD as I'll try to explain.

    I added the SPN's manually via windows setpsn (for both hyper-v hosts
    of course, mydomain.com.xyz is of course bogus name, real domain is
    something different)

    /setspn -S "Hyper-V Replica Service/BMSRV-WIN10" BMSRV-WIN10//
    //setspn -S "Hyper-V Replica Service/BMSRV-WIN10.mydomain.com.xyz"
    BMSRV-WIN10//
    //
    //setspn -S "Microsoft Virtual System Migration Service/BMSRV-WIN10"
    BMSRV-WIN10//
    //setspn -S "Microsoft Virtual System Migration
    Service/BMSRV-WIN10.mydomain.com.xyz" BMSRV-WIN10//
    //
    //setspn -S "Microsoft Virtual Console Service/BMSRV-WIN10"
    BMSRV-WIN10"//
    //setspn -S "Microsoft Virtual Console
    Service/BMSRV-WIN10.mydomain.com.xyz" BMSRV-WIN10//
    /
    Both windows and samba when queried show correct SPN's:
    output of windows query:

    spn -l BMSRV-WIN10

    Registered ServicePrincipalNames for
    CN=BMSRV-WIN10,CN=Computers,DC=mydomain,DC=com,DC=xyz:
    HOST/BMSRV-WIN10
    HOST/BMSRV-WIN10.mydomain.com.xyz
    Hyper-V Replica Service/BMSRV-WIN10
    Hyper-V Replica Service/BMSRV-WIN10.mydomain.com.xyz
    Microsoft Virtual Console Service/BMSRV-WIN10
    Microsoft Virtual Console Service/BMSRV-WIN10.mydomain.com.xyz
    Microsoft Virtual System Migration Service/BMSRV-WIN10
    Microsoft Virtual System Migration Service/BMSRV-WIN10.mydomain.com.xyz
    RestrictedKrbHost/BMSRV-WIN10
    RestrictedKrbHost/BMSRV-WIN10.mydomain.com.xyz
    TERMSRV/BMSRV-WIN10
    TERMSRV/BMSRV-WIN10.mydomain.com.xyz
    WSMAN/BMSRV-WIN10
    WSMAN/BMSRV-WIN10.mydomain.com.xyz

    output of samba-tool query:
    samba-tool spn list BMSRV-WIN10$

    samba-tool spn list BMSRV-WIN10$
    schema_fsmo_init: we are master[no] updates allowed[no]
    User CN=BMSRV-WIN10,CN=Computers,DC=mydomain,DC=com,DC=xyz has the
    following servicePrincipalName:
    HOST/BMSRV-WIN10
    HOST/BMSRV-WIN10.mydomain.com.xyz
    Hyper-V Replica Service/BMSRV-WIN10
    Hyper-V Replica Service/BMSRV-WIN10.mydomain.com.xyz
    Microsoft Virtual Console Service/BMSRV-WIN10
    Microsoft Virtual Console Service/BMSRV-WIN10.mydomain.com.xyz
    Microsoft Virtual System Migration Service/BMSRV-WIN10
    Microsoft Virtual System Migration
    Service/BMSRV-WIN10.mydomain.com.xyz
    RestrictedKrbHost/BMSRV-WIN10
    RestrictedKrbHost/BMSRV-WIN10.mydomain.com.xyz
    TERMSRV/BMSRV-WIN10
    TERMSRV/BMSRV-WIN10.mydomain.com.xyz
    WSMAN/BMSRV-WIN10
    WSMAN/BMSRV-WIN10.mydomain.com.xyz

    It looks all fine and well (the SPN names are 100% correct verified).

    For the hyper-v features to work (replica and live migration) with
    kerberos I need to setup delegation (it's set - verified it a milion
    times over it's set the right way, just like MS wants it).

    I know that I can obtain tickets to other SPN
    (from windows: *klist cifs/BMSRV-WIN10* grants me a valid ticket for
    example)

    Now cometh the error:
    When I try to run hyper-v replica it fails with error concerning
    kerberos and SPN not being there

    Log from samba DC3 (when trying to start Hyper-V replica from BM-SRV-5
    to BMSRV-WIN.10)

    Kerberos: TGS-REQ BM-SRV-5$@MYDOMAIN.COM.XYZ from
    ipv4:192.168.1.10:56993 for Hyper-V\ Replica\
    Service/BMSRV-WIN10.mydomain.com.xyz@MYDOMAIN.COM.XYZ [canonicalize,
    renewable, forwardable]
    [2017/03/16 10:55:07.246904, 4]
    ../source4/dsdb/samdb/cracknames.c:169(LDB_lookup_spn_alias)
    LDB_lookup_spn_alias: no alias for service Hyper-V Replica Service
    applicable
    [2017/03/16 10:55:07.246971, 3]
    ../source4/auth/kerberos/krb5_init_context.c:80(smb_krb5_debug_wrapper)
    Kerberos: Searching referral for BMSRV-WIN10.mydomain.com.xyz
    [2017/03/16 10:55:07.247028, 3]
    ../source4/auth/kerberos/krb5_init_context.c:80(smb_krb5_debug_wrapper)
    Kerberos: Server not found in database: Hyper-V\ Replica\
    Service/BMSRV-WIN10.mydomain.com.xyz@MYDOMAIN.COM.XYZ: no such entry
    found in hdb
    [2017/03/16 10:55:07.247053, 3]
    ../source4/auth/kerberos/krb5_init_context.c:80(smb_krb5_debug_wrapper)
    Kerberos: Failed building TGS-REP to ipv4:192.168.1.10:56993

    log from wireshark (earlier attempt but same issue, this time when
    trying to start live migration from BM-SRV-5 to BMSRV-WIN10):

    req-body
    Padding: 0
    kdc-options: 40810000 (forwardable, renewable, canonicalize)
    realm: MYDOMAIN.COM.XYZ
    sname
    name-type: kRB5-NT-SRV-INST (2)
    sname-string: 2 items
    SNameString: Microsoft Virtual System Migration Service
    SNameString: BMSRV-WIN10
    till: 2037-09-13 02:48:05 (UTC)
    nonce: 17847174
    etype: 5 items
    enc-authorization-data


    error:
    krb-error
    pvno: 5
    msg-type: krb-error (30)
    ctime: 2017-03-16 08:01:23 (UTC)
    cusec: 128
    stime: 2017-03-16 08:01:23 (UTC)
    susec: 66964
    error-code: eRR-S-PRINCIPAL-UNKNOWN (7)
    realm: <unspecified realm>
    sname
    name-type: kRB5-NT-UNKNOWN (0)
    sname-string: 0 items

    Same errors are when going the other way round,

    So the SPN's are clearly there (both setspn -l and samba-tool spn list
    outputs confirm that), the client sends correct request (as seen by
    wireshark and/or samba log), but suddenly samba is unable to find
    the SPN.
    I'm a complete newbie (well, sort-of) when it comes to kerberos and
    samba, but maybe because the SPN is with spaces, as it's pretty unusual,
    but that's what Microsoft wants/needs?
    I don't know, just a guess :-) . The features offered by hyper-v in AD
    are obviously beneficial and I would love to get them working.
    Any help, workaround or tip - I will be very, very thankful. If more
    info is needed I'll gladly supply logs/whatever is needed.

    Kacper Wirski

    --
    To unsubscribe from this list go to the following URL and read the
    instructions: https://lists.samba.org/mailman/options/samba

    --
    Z poważaniem,
    Kacper Wirski

    tel: + 48 608 421 424

    Babka Medica Sp. z o.o. Sp. k.
    ul. Słomińskiego 19/517, 00-195 Warszawa
    Sąd Rejonowy dla M.St. Warszawy w Warszawie XII Wydział Gospodarczy KRS 0000298042
    NIP 525-234-00-28

    www.babkamedica.pl

    ----------------------------------------------------------------------------


    Informacja zawarta w niniejszej korespondencji jest poufna. Korespondencja skierowana jest wyłącznie do osoby (firmy) wymienionej wyżej. Rozpowszechnianie, kopiowanie, ujawnianie lub przekazywanie osobom trzecim w jakiejkolwiek formie informacji zawartych w niniejszym dokumencie w całości lub w części jest zakazane bez uprzedniej pisemnej (pod rygorem nieważności)
    zgody Babka Medica Sp. z o.o. Sp. k.

    --
    To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kacper Wirski via samba@21:1/5 to All on Sat Mar 18 23:20:02 2017
    After reviewing logs I found that my previous assumption was wrong.

    Situation: - i'm trying to start live migration from hyper-v host A (BMSRV4-HYPERV) to hyper-v host B (BM-SRV-5) from host B (logged in as
    user from DOMAIN ADMINS group).

    Kerberos constrained delegation is set in accordnance to microsoft
    instructions with proper SPN's set (well, proper as in with the
    workaround I wrote earlier).

    Below logs from wireshark and Samba 4 DC (the one that handled request). kacper_wirski user, that belongs to DOMAIN ADMINS group is the one
    "giving" the command. I tried already with different user, also tried
    the other way round (from host B -> to host A when logged into host B).
    Same errors. Tried with different Hyper-V host C, same error

    I have bar to none experience with troubleshooting kerberos (up untli
    now everything was working flawlessly) but reading from the logs I
    understand that generated ticket request from Host A seems ok: it wants
    to "impersonate" kacper_wirski in order to get to SPN on Host B, but
    request fails.
    I admit that I already googled this error and wasted a lot of hours,
    but I really don't know how to handle this situation - wether it's
    kerberos error, or samba error, or microsoft Hyper-V was just built
    that way that it simply will work ONLY with microsoft AD?

    Every bit of advice/tip is greatly appreciated, as I feel i'm running
    out of ideas or options.

    /etc/krb5.conf is basic generated ad DC promo. Overall no issues in the
    domain using kerberos so far (over 6 months now), also used SSO for
    apache so kerberos overall seems ok.

    Logs below (tried my best to trim down).

    Samba 4 log from DC that Host A contacted (one of 3 DC's in domain):

    Log level 5
    Kerberos: TGS-REQ BMSRV4-HYPERV$@MYDOMAIN.COM.XYZ from ipv4:192.168.1.14:64931 for bmsrv4-hyperv$@MYDOMAIN.COM.XYZ
    [canonicalize, renewable, forwardable]
    [2017/03/18 22:00:03.656232, 3] ../source4/auth/kerberos/krb5_init_context.c:80(smb_krb5_debug_wrapper)
    Kerberos: s4u2self BMSRV4-HYPERV$@MYDOMAIN.COM.XYZ impersonating kacper_wirski@MYDOMAIN.COM.XYZ to service
    bmsrv4-hyperv$@MYDOMAIN.COM.XYZ [forwardable]
    [2017/03/18 22:00:03.656262, 3] ../source4/auth/kerberos/krb5_init_context.c:80(smb_krb5_debug_wrapper)
    Kerberos: TGS-REQ authtime: 2017-03-18T21:39:30 starttime: 2017-03-18T22:00:03 endtime: 2017-03-18T22:15:03 renew till: 2017-03-25T21:39:30
    [2017/03/18 22:00:03.657328, 3] ../source4/smbd/service_stream.c:66(stream_terminate_connection)
    Terminating connection - 'kdc_tcp_call_loop:
    tstream_read_pdu_blob_recv() - NT_STATUS_CONNECTION_DISCONNECTED'
    [2017/03/18 22:00:03.657340, 3] ../source4/smbd/process_single.c:114(single_terminate)
    single_terminate: reason[kdc_tcp_call_loop:
    tstream_read_pdu_blob_recv() - NT_STATUS_CONNECTION_DISCONNECTED]
    [2017/03/18 22:00:03.658763, 3] ../source4/auth/kerberos/krb5_init_context.c:80(smb_krb5_debug_wrapper)
    Kerberos: Failed to decrypt enc-authorization-data
    [2017/03/18 22:00:03.658776, 3] ../source4/auth/kerberos/krb5_init_context.c:80(smb_krb5_debug_wrapper)
    Kerberos: Failed parsing TGS-REQ from ipv4:192.168.1.14:64932
    [2017/03/18 22:00:03.658911, 3] ../source4/smbd/service_stream.c:66(stream_terminate_connection)
    Terminating connection - 'dcesrv: NT_STATUS_CONNECTION_DISCONNECTED' [2017/03/18 22:00:03.658920, 3] ../source4/smbd/process_single.c:114(single_terminate)
    single_terminate: reason[dcesrv: NT_STATUS_CONNECTION_DISCONNECTED]


    Wireshark relevant output:

    TGS-REQ (host A -> Samba 4 AD DC):
    Kerberos
    msg-type: krb-ap-req (14)

    ticket
    realm: MYDOMAIN.COM.XYZ
    sname
    name-type: kRB5-NT-SRV-INST (2)
    sname-string: 2 items
    SNameString: krbtgt
    SNameString: MYDOMAIN.COM.XYZ
    enc-part
    etype: eTYPE-ARCFOUR-HMAC-MD5 (23)
    authenticator
    etype: eTYPE-ARCFOUR-HMAC-MD5 (23)
    PA-DATA PA-FOR-USER
    padata-type: kRB5-PADATA-S4U2SELF (129)
    name
    name-type: kRB5-NT-ENTERPRISE-PRINCIPAL (10)
    name-string: 1 item
    KerberosString: kacper_wirski
    realm: MYDOMAIN.COM.XYZ
    cksum
    cksumtype: cKSUMTYPE-HMAC-MD5 (-138)
    auth: Kerberos
    req-body
    Padding: 0
    kdc-options: 40810000 (forwardable, renewable, canonicalize)
    realm: MYDOMAIN.COM.XYZ
    sname
    name-type: kRB5-NT-PRINCIPAL (1)
    sname-string: 1 item
    SNameString: bmsrv4-hyperv$
    etype: 5 items
    ENCTYPE: eTYPE-AES256-CTS-HMAC-SHA1-96 (18)
    ENCTYPE: eTYPE-AES128-CTS-HMAC-SHA1-96 (17)
    ENCTYPE: eTYPE-ARCFOUR-HMAC-MD5 (23)
    ENCTYPE: eTYPE-ARCFOUR-HMAC-MD5-56 (24)
    ENCTYPE: eTYPE-ARCFOUR-HMAC-OLD-EXP (-135)

    TGS-REP KDC -> HOST A

    tgs-rep
    msg-type: krb-tgs-rep (13)
    crealm: MYDOMAIN.COM.XYZ
    cname
    name-type: kRB5-NT-ENTERPRISE-PRINCIPAL (10)
    cname-string: 1 item
    CNameString: kacper_wirski
    ticket
    tkt-vno: 5
    realm: MYDOMAIN.COM.XYZ
    sname
    name-type: kRB5-NT-PRINCIPAL (1)
    sname-string: 1 item
    SNameString: bmsrv4-hyperv$
    enc-part
    etype: eTYPE-ARCFOUR-HMAC-MD5 (23)
    kvno: 1
    enc-part
    etype: eTYPE-ARCFOUR-HMAC-MD5 (23)

    TGS-REQ (Host A -> KDC)

    tgs-req
    pvno: 5
    msg-type: krb-tgs-req (12)
    padata: 2 items
    PA-DATA PA-TGS-REQ
    padata-type: kRB5-PADATA-TGS-REQ (1)
    ticket
    tkt-vno: 5
    realm: MYDOMAIN.COM.XYZ
    sname
    name-type: kRB5-NT-SRV-INST (2)
    sname-string: 2 items
    SNameString: krbtgt
    SNameString: MYDOMAIN.COM.XYZ
    enc-part
    etype: eTYPE-ARCFOUR-HMAC-MD5 (23)
    kvno: 1
    authenticator
    etype: eTYPE-ARCFOUR-HMAC-MD5 (23)
    PA-DATA Unknown:167
    padata-type: Unknown (167)
    padata-value: 3009a00703050010000000
    req-body
    Padding: 0
    kdc-options: 40830000 (forwardable, renewable,
    request-anonymous, canonicalize)
    realm: MYDOMAIN.COM.XYZ
    sname
    name-type: kRB5-NT-SRV-INST (2)
    sname-string: 2 items
    SNameString: Microsoft Virtual System Migration Service
    SNameString: BM-SRV-5
    till: 2017-03-18 21:15:03 (UTC)
    nonce: 478023267
    etype: 5 items
    ENCTYPE: eTYPE-AES256-CTS-HMAC-SHA1-96 (18)
    ENCTYPE: eTYPE-AES128-CTS-HMAC-SHA1-96 (17)
    ENCTYPE: eTYPE-ARCFOUR-HMAC-MD5 (23)
    ENCTYPE: eTYPE-ARCFOUR-HMAC-MD5-56 (24)
    ENCTYPE: eTYPE-ARCFOUR-HMAC-OLD-EXP (-135)
    enc-authorization-data
    etype: eTYPE-ARCFOUR-HMAC-MD5 (23)
    cipher: 0fa4ee9a7e16003266d7566c12c2f50748e50435090ee9e2...
    additional-tickets: 1 item
    Ticket
    realm: MYDOMAIN.COM.XYZ
    sname
    name-type: kRB5-NT-PRINCIPAL (1)
    SNameString: bmsrv4-hyperv$
    enc-part
    etype: eTYPE-ARCFOUR-HMAC-MD5 (23)
    kvno:
    and final TGS-REP (KDC -> HOST A)
    krb-error
    pvno: 5
    msg-type: krb-error (30)
    ctime: 2017-03-18 21:00:03 (UTC)
    cusec: 481
    stime: 2017-03-18 21:00:03 (UTC)
    susec: 658781
    error-code: eRR-BAD-INTEGRITY (31)
    realm: <unspecified realm>
    sname
    name-type: kRB5-NT-UNKNOWN (0)
    sname-string: 0 items




    --
    To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Luke Bigum via samba@21:1/5 to All on Sun Mar 19 21:50:01 2017
    Hello,

    This won't be a very helpful reply, but I can confirm I've had the exact same issue. I ran into this a few years ago and could not get HyperV migrations to work with a Samba DC. I even went so far as to install a Windows DC just to prove to myself that
    it is supposed to work, and it does, perfectly (with ADDC it even creates all the SPNs for you auto-magically).

    Unfortunately at the time I was focused on a Windows VM Disaster Recovery problem, so ended up dropping HyperV entirely in favour of KVM and DRBD. As such, I never raised a bug with Samba or Catalyst about this - I probably should have :-/ Sorry I can't
    be of more help other than to add my voice to "there is a bug somewhere in Samba".

    --
    Luke Bigum
    Lead Engineer

    Information Systems

    ----- Original Message -----
    From: "Kacper Wirski via samba" <samba@lists.samba.org>
    To: samba@lists.samba.org
    Sent: Saturday, 18 March, 2017 22:10:01
    Subject: Re: [Samba] kerberos issue (SPN not found) with windows Hyper-V ( samba 4.5.3 AD)

    After reviewing logs I found that my previous assumption was wrong.

    Situation: - i'm trying to start live migration from hyper-v host A (BMSRV4-HYPERV) to hyper-v host B (BM-SRV-5) from host B (logged in as
    user from DOMAIN ADMINS group).

    Kerberos constrained delegation is set in accordnance to microsoft
    instructions with proper SPN's set (well, proper as in with the
    workaround I wrote earlier).

    Below logs from wireshark and Samba 4 DC (the one that handled request). kacper_wirski user, that belongs to DOMAIN ADMINS group is the one
    "giving" the command. I tried already with different user, also tried
    the other way round (from host B -> to host A when logged into host B).
    Same errors. Tried with different Hyper-V host C, same error

    I have bar to none experience with troubleshooting kerberos (up untli
    now everything was working flawlessly) but reading from the logs I
    understand that generated ticket request from Host A seems ok: it wants
    to "impersonate" kacper_wirski in order to get to SPN on Host B, but
    request fails.
    I admit that I already googled this error and wasted a lot of hours,
    but I really don't know how to handle this situation - wether it's
    kerberos error, or samba error, or microsoft Hyper-V was just built
    that way that it simply will work ONLY with microsoft AD?

    Every bit of advice/tip is greatly appreciated, as I feel i'm running
    out of ideas or options.

    /etc/krb5.conf is basic generated ad DC promo. Overall no issues in the
    domain using kerberos so far (over 6 months now), also used SSO for
    apache so kerberos overall seems ok.

    Logs below (tried my best to trim down).

    Samba 4 log from DC that Host A contacted (one of 3 DC's in domain):

    Log level 5
    Kerberos: TGS-REQ BMSRV4-HYPERV$@MYDOMAIN.COM.XYZ from ipv4:192.168.1.14:64931 for bmsrv4-hyperv$@MYDOMAIN.COM.XYZ
    [canonicalize, renewable, forwardable]
    [2017/03/18 22:00:03.656232, 3] ../source4/auth/kerberos/krb5_init_context.c:80(smb_krb5_debug_wrapper)
    Kerberos: s4u2self BMSRV4-HYPERV$@MYDOMAIN.COM.XYZ impersonating kacper_wirski@MYDOMAIN.COM.XYZ to service
    bmsrv4-hyperv$@MYDOMAIN.COM.XYZ [forwardable]
    [2017/03/18 22:00:03.656262, 3] ../source4/auth/kerberos/krb5_init_context.c:80(smb_krb5_debug_wrapper)
    Kerberos: TGS-REQ authtime: 2017-03-18T21:39:30 starttime: 2017-03-18T22:00:03 endtime: 2017-03-18T22:15:03 renew till: 2017-03-25T21:39:30
    [2017/03/18 22:00:03.657328, 3] ../source4/smbd/service_stream.c:66(stream_terminate_connection)
    Terminating connection - 'kdc_tcp_call_loop:
    tstream_read_pdu_blob_recv() - NT_STATUS_CONNECTION_DISCONNECTED'
    [2017/03/18 22:00:03.657340, 3] ../source4/smbd/process_single.c:114(single_terminate)
    single_terminate: reason[kdc_tcp_call_loop:
    tstream_read_pdu_blob_recv() - NT_STATUS_CONNECTION_DISCONNECTED]
    [2017/03/18 22:00:03.658763, 3] ../source4/auth/kerberos/krb5_init_context.c:80(smb_krb5_debug_wrapper)
    Kerberos: Failed to decrypt enc-authorization-data
    [2017/03/18 22:00:03.658776, 3] ../source4/auth/kerberos/krb5_init_context.c:80(smb_krb5_debug_wrapper)
    Kerberos: Failed parsing TGS-REQ from ipv4:192.168.1.14:64932
    [2017/03/18 22:00:03.658911, 3] ../source4/smbd/service_stream.c:66(stream_terminate_connection)
    Terminating connection - 'dcesrv: NT_STATUS_CONNECTION_DISCONNECTED' [2017/03/18 22:00:03.658920, 3] ../source4/smbd/process_single.c:114(single_terminate)
    single_terminate: reason[dcesrv: NT_STATUS_CONNECTION_DISCONNECTED]


    Wireshark relevant output:

    TGS-REQ (host A -> Samba 4 AD DC):
    Kerberos
    msg-type: krb-ap-req (14)

    ticket
    realm: MYDOMAIN.COM.XYZ
    sname
    name-type: kRB5-NT-SRV-INST (2)
    sname-string: 2 items
    SNameString: krbtgt
    SNameString: MYDOMAIN.COM.XYZ
    enc-part
    etype: eTYPE-ARCFOUR-HMAC-MD5 (23)
    authenticator
    etype: eTYPE-ARCFOUR-HMAC-MD5 (23)
    PA-DATA PA-FOR-USER
    padata-type: kRB5-PADATA-S4U2SELF (129)
    name
    name-type: kRB5-NT-ENTERPRISE-PRINCIPAL (10)
    name-string: 1 item
    KerberosString: kacper_wirski
    realm: MYDOMAIN.COM.XYZ
    cksum
    cksumtype: cKSUMTYPE-HMAC-MD5 (-138)
    auth: Kerberos
    req-body
    Padding: 0
    kdc-options: 40810000 (forwardable, renewable, canonicalize)
    realm: MYDOMAIN.COM.XYZ
    sname
    name-type: kRB5-NT-PRINCIPAL (1)
    sname-string: 1 item
    SNameString: bmsrv4-hyperv$
    etype: 5 items
    ENCTYPE: eTYPE-AES256-CTS-HMAC-SHA1-96 (18)
    ENCTYPE: eTYPE-AES128-CTS-HMAC-SHA1-96 (17)
    ENCTYPE: eTYPE-ARCFOUR-HMAC-MD5 (23)
    ENCTYPE: eTYPE-ARCFOUR-HMAC-MD5-56 (24)
    ENCTYPE: eTYPE-ARCFOUR-HMAC-OLD-EXP (-135)

    TGS-REP KDC -> HOST A

    tgs-rep
    msg-type: krb-tgs-rep (13)
    crealm: MYDOMAIN.COM.XYZ
    cname
    name-type: kRB5-NT-ENTERPRISE-PRINCIPAL (10)
    cname-string: 1 item
    CNameString: kacper_wirski
    ticket
    tkt-vno: 5
    realm: MYDOMAIN.COM.XYZ
    sname
    name-type: kRB5-NT-PRINCIPAL (1)
    sname-string: 1 item
    SNameString: bmsrv4-hyperv$
    enc-part
    etype: eTYPE-ARCFOUR-HMAC-MD5 (23)
    kvno: 1
    enc-part
    etype: eTYPE-ARCFOUR-HMAC-MD5 (23)

    TGS-REQ (Host A -> KDC)

    tgs-req
    pvno: 5
    msg-type: krb-tgs-req (12)
    padata: 2 items
    PA-DATA PA-TGS-REQ
    padata-type: kRB5-PADATA-TGS-REQ (1)
    ticket
    tkt-vno: 5
    realm: MYDOMAIN.COM.XYZ
    sname
    name-type: kRB5-NT-SRV-INST (2)
    sname-string: 2 items
    SNameString: krbtgt
    SNameString: MYDOMAIN.COM.XYZ
    enc-part
    etype: eTYPE-ARCFOUR-HMAC-MD5 (23)
    kvno: 1
    authenticator
    etype: eTYPE-ARCFOUR-HMAC-MD5 (23)
    PA-DATA Unknown:167
    padata-type: Unknown (167)
    padata-value: 3009a00703050010000000
    req-body
    Padding: 0
    kdc-options: 40830000 (forwardable, renewable,
    request-anonymous, canonicalize)
    realm: MYDOMAIN.COM.XYZ
    sname
    name-type: kRB5-NT-SRV-INST (2)
    sname-string: 2 items
    SNameString: Microsoft Virtual System Migration Service
    SNameString: BM-SRV-5
    till: 2017-03-18 21:15:03 (UTC)
    nonce: 478023267
    etype: 5 items
    ENCTYPE: eTYPE-AES256-CTS-HMAC-SHA1-96 (18)
    ENCTYPE: eTYPE-AES128-CTS-HMAC-SHA1-96 (17)
    ENCTYPE: eTYPE-ARCFOUR-HMAC-MD5 (23)
    ENCTYPE: eTYPE-ARCFOUR-HMAC-MD5-56 (24)
    ENCTYPE: eTYPE-ARCFOUR-HMAC-OLD-EXP (-135)
    enc-authorization-data
    etype: eTYPE-ARCFOUR-HMAC-MD5 (23)
    cipher: 0fa4ee9a7e16003266d7566c12c2f50748e50435090ee9e2...
    additional-tickets: 1 item
    Ticket
    realm: MYDOMAIN.COM.XYZ
    sname
    name-type: kRB5-NT-PRINCIPAL (1)
    SNameString: bmsrv4-hyperv$
    enc-part
    etype: eTYPE-ARCFOUR-HMAC-MD5 (23)
    kvno:
    and final TGS-REP (KDC -> HOST A)
    krb-error
    pvno: 5
    msg-type: krb-error (30)
    ctime: 2017-03-18 21:00:03 (UTC)
    cusec: 481
    stime: 2017-03-18 21:00:03 (UTC)
    susec: 658781
    error-code: eRR-BAD-INTEGRITY (31)
    realm: <unspecified realm>
    sname
    name-type: kRB5-NT-UNKNOWN (0)
    sname-string: 0 items




    --
    To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
    ---

    LMAX Exchange, Yellow Building, 1A Nicholas Road, London W11 4AN http://www.LMAX.com/

    Recognised by the most prestigious business and technology awards

    2016 Best Trading & Execution, HFM US Technology Awards
    2016, 2015, 2014, 2013 Best FX Trading Venue - ECN/MTF, WSL Institutional Trading Awards

    2016, 2015 Winner, Deloitte UK Technology Fast 50
    2015, 2014, 2013, One of the UK's fastest growing technology firms, The Sunday Times Tech Track 100
    2016, 2015 Winner, Deloitte EMEA Technology Fast 500
    2015, 2014, 2013 Best Margin Sector Platform, Profit & Loss Readers' Choice Awards

    ---

    FX and CFDs are leveraged products that can result in losses exceeding your deposit. They are not suitable for everyone so please ensure you fully understand the risks involved.

    This message and its attachments are confidential, may not be disclosed or used by any person other than the addressee and are intended only for the named recipient(s). This message is not intended for any recipient(s) who based
  • From Kacper Wirski via samba@21:1/5 to All on Sun Mar 19 23:10:01 2017
    To be more precise, live migration works, but only when started directly
    from the hyper-v that I'm directly logged into (via RDP for example), as
    in: logged into host A, I can live migrate from host A to host B, to
    migrate the other way round I have to log into host B first and start
    migration there.

    Bug must be somewhere within kerberos constrained delegation or maybe
    it's still connected to the weird hyper-v SPN's (weird as in: with
    whitespaces within their names) and my proposed workaround to get
    anything started (hyper-v replica and live migration with
    aforementioned limitations).

    I've tried to read up more on possible causes for the kerberos error,
    but I'm too much of a newbie to have any chance to truly understand
    what's not working. Or even if it's supposed to work at all.

    Anyway I made a further test and I created a simple 2 node Hyper-V
    failover cluster on top of those hyper-v hosts used earlier, and within
    the cluster live migration works perfectly in every direction imagined
    (when using failover cluster manager, I can live migrate VM from host a
    to host b, then directly from host b to host a without changing machines
    i'm logged into), but then again it is overall a lot complicated
    solution which I'm not ready to use in production yet.

    Overall some issues I did manage to solve, so I guess that's that. :-)


    W dniu 2017-03-19 o 21:18, Luke Bigum via samba pisze:
    Hello,

    This won't be a very helpful reply, but I can confirm I've had the exact same issue. I ran into this a few years ago and could not get HyperV migrations to work with a Samba DC. I even went so far as to install a Windows DC just to prove to myself that
    it is supposed to work, and it does, perfectly (with ADDC it even creates all the SPNs for you auto-magically).

    Unfortunately at the time I was focused on a Windows VM Disaster Recovery problem, so ended up dropping HyperV entirely in favour of KVM and DRBD. As such, I never raised a bug with Samba or Catalyst about this - I probably should have :-/ Sorry I can'
    t be of more help other than to add my voice to "there is a bug somewhere in Samba".



    --
    To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)