GSS-TSIG not working and have no idea why

[#1]

Struggling a bit in my lab today. Previously I have had GSS-TSIG signed updates working fine from Infoblox DHCP into MS DNS, but for some reason today, I just cannot get it working. I have upgraded to NIOS 8.5.2 in case it was a bug, but despite everything I get a message saying "server unknown" when it's trying to negotiate the security context. The diagnostics also complain about the same issue, but when I run dig the server name is resolved no problem.

I'm sure it's something stupidly easy but I can't see the wood for the trees and need another set of eyes...

iblab (A) > show dhcp_gss_tsig config
System time: Fri Jun 25 16:23:14 UTC 2021

DHCP GSS-TSIG configuration for this member:
  KDC address              192.168.2.71
  KDC IP                   192.168.2.71
  Member principal         DNS/win2k8-1.ad.cn.corp@AD.CN.CORP
  Member domain            AD.CN.CORP
  GSS-TSIG                 enabled
  DDNS updates             enabled
  DHCP service             enabled

Test KDC using member configuration? (y or n): y
Requesting TGT for DNS/win2k8-1.ad.cn.corp@AD.CN.CORP from KDC 192.168.2.71...
Successfully obtained test TGT.

Credentials cache: FILE:/tmp/krb5_cache.7892
        Principal: DNS/win2k8-1.ad.cn.corp@AD.CN.CORP
    Cache version: 4

Server: krbtgt/AD.CN.CORP@AD.CN.CORP
Client: DNS/win2k8-1.ad.cn.corp@AD.CN.CORP
Ticket etype: aes256-cts-hmac-sha1-96, kvno 2
Ticket length: 1082
Auth time:  Jun 25 16:23:43 2021
End time:   Jun 26 02:23:15 2021
Ticket flags: initial, pre-authenticated
Addresses: addressless

Successful test. Test TGT destroyed.

This member is configured to update the following zones:
  ad.cn.corp on 192.168.2.71 as DNS/win2k8-1.ad.cn.corp@AD.CN.CORP
  168.192.in-addr.arpa on 192.168.2.71 as DNS/win2k8-1.ad.cn.corp@AD.CN.CORP

Test configured zones? (y or n): y

Next zone is ad.cn.corp on 192.168.2.71.
Test this zone? (y or n): y
Testing external zone ad.cn.corp on NS 192.168.2.71...
DNS principal is DNS/win2k8-1.ad.cn.corp@AD.CN.CORP.
Derived FQDN is win2k8-1.ad.cn.corp.
Error: FQDN does not resolve to nameserver IP.
Error: Nameserver is not authoritative for zone.
Errors were detected. Zone configuration may be invalid.

Next zone is 168.192.in-addr.arpa on 192.168.2.71.
Test this zone? (y or n): y
Testing external zone 168.192.in-addr.arpa on NS 192.168.2.71...
DNS principal is DNS/win2k8-1.ad.cn.corp@AD.CN.CORP.
Derived FQDN is win2k8-1.ad.cn.corp.
Error: FQDN does not resolve to nameserver IP.
Error: Nameserver is not authoritative for zone.
Errors were detected. Zone configuration may be invalid.
iblab (A) > dig win2k8-1.ad.cn.corp.

; <<>> DiG 9.11.3-S3 <<>> +noedns win2k8-1.ad.cn.corp.
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 17228
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;win2k8-1.ad.cn.corp.           IN      A

;; ANSWER SECTION:
win2k8-1.ad.cn.corp.    3600    IN      A       192.168.2.71

;; Query time: 0 msec
;; SERVER: 192.168.2.61#53(192.168.2.61)
;; WHEN: Fri Jun 25 17:24:27 BST 2021
;; MSG SIZE  rcvd: 53

iblab (A) > dig ad.cn.corp. ns

; <<>> DiG 9.11.3-S3 <<>> +noedns ad.cn.corp. ns
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 33127
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;ad.cn.corp.                    IN      NS

;; ANSWER SECTION:
ad.cn.corp.             3600    IN      NS      win2k8-2.ad.cn.corp.
ad.cn.corp.             3600    IN      NS      win2k8-1.ad.cn.corp.

;; Query time: 0 msec
;; SERVER: 192.168.2.61#53(192.168.2.61)
;; WHEN: Fri Jun 25 17:24:36 BST 2021
;; MSG SIZE  rcvd: 74

iblab (A) > dig ad.cn.corp. soa

; <<>> DiG 9.11.3-S3 <<>> +noedns ad.cn.corp. soa
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 13179
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;ad.cn.corp.                    IN      SOA

;; ANSWER SECTION:
ad.cn.corp.             3600    IN      SOA     win2k8-1.ad.cn.corp. postmaster.no.email.please. 635170316 3600 600 2592000 3600

;; Query time: 0 msec
;; SERVER: 192.168.2.61#53(192.168.2.61)
;; WHEN: Fri Jun 25 17:24:41 BST 2021
;; MSG SIZE  rcvd: 99

iblab (A) >

[#2]

Hello Paul,

Seeing the diagnostic test result pasted, i wonder if the KDC is able to resolve its own name ? Can you try this from the CLI of your Iblab?  : 

iblab (A) > dig @192.168.2.71 win2k8-1.ad.cn.corp

That's what'll happen in the end to verify zone authority(during the config test). As you've specified 192.168.2.71 to be the authoritative server for ad.cn.corp & 168.192.in-addr.arpa zones under Configure DDNS, your Iblab would try to compare the above dig result with this IP. If it doesn't match, then the test would throw that error in the end.

If that dig result doesn't resolve to 192.168.2.71 as of now, can you ensure this result & try again to see if it helps in getting the secure updates through ?

Now if it doesn't help, I would take a traffic capture duirng the failed attempts & compare the krb error from the capture with something like this link , which might give a bit more clear insight around the problem. 

Best regards,

[#3]

Hi, thanks for replying, yes I did check that the server name itself resolves, here is the dig output to both windows servers...

iblab (A) > dig @192.168.2.71 win2k8-1.ad.cn.corp

; <<>> DiG 9.11.3-S3 <<>> +noedns @192.168.2.71 win2k8-1.ad.cn.corp
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 25501
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;win2k8-1.ad.cn.corp.           IN      A

;; ANSWER SECTION:
win2k8-1.ad.cn.corp.    3600    IN      A       192.168.2.71

;; Query time: 0 msec
;; SERVER: 192.168.2.71#53(192.168.2.71)
;; WHEN: Mon Jun 28 09:36:37 BST 2021
;; MSG SIZE  rcvd: 53

iblab (A) >

iblab (A) > dig @192.168.2.72 win2k8-1.ad.cn.corp

; <<>> DiG 9.11.3-S3 <<>> +noedns @192.168.2.72 win2k8-1.ad.cn.corp
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 28617
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;win2k8-1.ad.cn.corp.           IN      A

;; ANSWER SECTION:
win2k8-1.ad.cn.corp.    3600    IN      A       192.168.2.71

;; Query time: 1 msec
;; SERVER: 192.168.2.72#53(192.168.2.72)
;; WHEN: Mon Jun 28 09:38:36 BST 2021
;; MSG SIZE  rcvd: 53

iblab (A) >

[#4]

Also both the forward and reverse zones I am trying to update resolve fine, there is something weird going on here, maybe some kind of bug...

iblab (A) > dig @192.168.2.71 ad.cn.corp ns

; <<>> DiG 9.11.3-S3 <<>> +noedns @192.168.2.71 ad.cn.corp ns
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 50715
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 2

;; QUESTION SECTION:
;ad.cn.corp.                    IN      NS

;; ANSWER SECTION:
ad.cn.corp.             3600    IN      NS      win2k8-1.ad.cn.corp.
ad.cn.corp.             3600    IN      NS      win2k8-2.ad.cn.corp.

;; ADDITIONAL SECTION:
win2k8-1.ad.cn.corp.    3600    IN      A       192.168.2.71
win2k8-2.ad.cn.corp.    3600    IN      A       192.168.2.72

;; Query time: 0 msec

Enter <return> for next page or q<return> to cancel the command.

;; SERVER: 192.168.2.71#53(192.168.2.71)
;; WHEN: Mon Jun 28 09:41:14 BST 2021
;; MSG SIZE  rcvd: 106

iblab (A) > dig @192.168.2.71 168.192.in-addr.arpa. ns

; <<>> DiG 9.11.3-S3 <<>> +noedns @192.168.2.71 168.192.in-addr.arpa. ns
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49697
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 2

;; QUESTION SECTION:
;168.192.in-addr.arpa.          IN      NS

;; ANSWER SECTION:
168.192.in-addr.arpa.   3600    IN      NS      win2k8-2.ad.cn.corp.
168.192.in-addr.arpa.   3600    IN      NS      win2k8-1.ad.cn.corp.

;; ADDITIONAL SECTION:
win2k8-2.ad.cn.corp.    3600    IN      A       192.168.2.72
win2k8-1.ad.cn.corp.    3600    IN      A       192.168.2.71

;; Query time: 0 msec

Enter <return> for next page or q<return> to cancel the command.

;; SERVER: 192.168.2.71#53(192.168.2.71)
;; WHEN: Mon Jun 28 09:41:21 BST 2021
;; MSG SIZE  rcvd: 126

iblab (A) >

[#5]

Hello Paul,

Thank you for your reply. If this happened right after a product upgrade, then there are chances of a change in feature architecture or may be a bug. But if this happened all in sudden in a working lab/prod environment where it has been working well, may be some environmental variables changed resulting in the situation(Just assuming, since i think its too early to assume this to be a bug). Can you look into the traffic capture to see what the the KDC is complaining about ? This page has the error descriptions to give a basic lead. Just a sample of error(Please ignore the error seen) :

Here, 192.168.29.225 is my KDC. I'll try to replicate the error that you see, in my lab meanwhile(Trial/error). 

Best regards,