Query refused when DNS member is set in Dashboard dig widget

[#1]

When using the Dig Request widget from the Dashboards - Status tab (we're on NIOS 8.5.4) and using a grid member with DNS running, the results always come back with a status of refused. I've tried multiple record types and setting the name server to localhost, 127.0.0.1, hostname of the server, and IP of the server, with the same results. The only time it's worked is when I've used the anycast IP that's set on the server.

When I remote into the appliance and run digs, it answers unless the source address of the query is set to 127.0.1.0 (same as used in the widget).

; <<>> DiG 9.11.3-S3 <<>> +noedns login.microsoftonline.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 22360
;; flags: qr rd ra; QUERY: 1, ANSWER: 10, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;login.microsoftonline.com.     IN      A

;; ANSWER SECTION:
login.microsoftonline.com. 81   IN      CNAME   ak.privatelink.msidentity.com.
ak.privatelink.msidentity.com. 50 IN    CNAME   www.tm.ak.prd.aadg.trafficmanager.net.
www.tm.ak.prd.aadg.trafficmanager.net. 5 IN A   40.126.28.12
www.tm.ak.prd.aadg.trafficmanager.net. 5 IN A   40.126.28.21
www.tm.ak.prd.aadg.trafficmanager.net. 5 IN A   40.126.28.22
www.tm.ak.prd.aadg.trafficmanager.net. 5 IN A   40.126.7.35
www.tm.ak.prd.aadg.trafficmanager.net. 5 IN A   40.126.28.13
www.tm.ak.prd.aadg.trafficmanager.net. 5 IN A   40.126.28.23
www.tm.ak.prd.aadg.trafficmanager.net. 5 IN A   40.126.28.18
www.tm.ak.prd.aadg.trafficmanager.net. 5 IN A   40.126.28.14

;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Fri Apr 29 16:14:49 UTC 2022
;; MSG SIZE  rcvd: 262

; <<>> DiG 9.11.3-S3 <<>> +noedns -b 127.0.1.0 login.microsoftonline.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: REFUSED, id: 49899
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;login.microsoftonline.com.     IN      A

;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Fri Apr 29 16:14:59 UTC 2022
;; MSG SIZE  rcvd: 43

; <<>> DiG 9.11.3-S3 <<>> +noedns -b 127.0.0.1 login.microsoftonline.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 20107
;; flags: qr rd ra; QUERY: 1, ANSWER: 10, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;login.microsoftonline.com.     IN      A

;; ANSWER SECTION:
login.microsoftonline.com. 62   IN      CNAME   ak.privatelink.msidentity.com.
ak.privatelink.msidentity.com. 31 IN    CNAME   www.tm.ak.prd.aadg.trafficmanager.net.
www.tm.ak.prd.aadg.trafficmanager.net. 204 IN A 40.126.28.12
www.tm.ak.prd.aadg.trafficmanager.net. 204 IN A 40.126.28.11
www.tm.ak.prd.aadg.trafficmanager.net. 204 IN A 40.126.28.14
www.tm.ak.prd.aadg.trafficmanager.net. 204 IN A 40.126.28.22
www.tm.ak.prd.aadg.trafficmanager.net. 204 IN A 40.126.7.35
www.tm.ak.prd.aadg.trafficmanager.net. 204 IN A 40.126.28.23
www.tm.ak.prd.aadg.trafficmanager.net. 204 IN A 40.126.28.19
www.tm.ak.prd.aadg.trafficmanager.net. 204 IN A 40.126.28.21

;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Fri Apr 29 16:15:08 UTC 2022
;; MSG SIZE  rcvd: 262

Interestingly, when I query from my workstation terminal using the IP of the server, the query is refused, but when I use the server name, it resolves properly.

; <<>> DiG 9.16.11 <<>> @10.251.8.250 login.microsoftonline.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: REFUSED, id: 32884
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;login.microsoftonline.com.     IN      A

;; Query time: 78 msec
;; SERVER: 10.251.8.250#53(10.251.8.250)
;; WHEN: Fri Apr 29 11:17:53 Central Daylight Time 2022
;; MSG SIZE  rcvd: 43

; <<>> DiG 9.16.11 <<>> @nwkinfi4t001.domain.local login.microsoftonline.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 36158
;; flags: qr rd ra; QUERY: 1, ANSWER: 10, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1220
; COOKIE: 494553ad5f52d6647adf2258626c0fbaf2c9329ab4a95e4b (good)
;; QUESTION SECTION:
;login.microsoftonline.com.     IN      A

;; ANSWER SECTION:
login.microsoftonline.com. 5    IN      CNAME   ak.privatelink.msidentity.com.
ak.privatelink.msidentity.com. 5 IN     CNAME   www.tm.ak.prd.aadg.trafficmanager.net.
www.tm.ak.prd.aadg.trafficmanager.net. 30 IN A  40.126.28.14
www.tm.ak.prd.aadg.trafficmanager.net. 30 IN A  40.126.28.22
www.tm.ak.prd.aadg.trafficmanager.net. 30 IN A  40.126.7.35
www.tm.ak.prd.aadg.trafficmanager.net. 30 IN A  40.126.28.23
www.tm.ak.prd.aadg.trafficmanager.net. 30 IN A  40.126.28.19
www.tm.ak.prd.aadg.trafficmanager.net. 30 IN A  40.126.28.21
www.tm.ak.prd.aadg.trafficmanager.net. 30 IN A  40.126.28.12
www.tm.ak.prd.aadg.trafficmanager.net. 30 IN A  40.126.28.11

;; Query time: 62 msec
;; SERVER: 10.251.8.150#53(10.251.8.150)
;; WHEN: Fri Apr 29 11:18:02 Central Daylight Time 2022
;; MSG SIZE  rcvd: 301

I talked to my SE and he doesn't see the same behavior on his lab. We looked through the named.conf on both servers and there's nothing preventing access, as far as we can see.

options {
	recursion yes;
	max-recursion-depth 7;
	max-recursion-queries 150;
	listen-on { 127.0.0.1; 10.251.8.150; };
	query-source address 10.251.8.150 port *; 
	notify-source 10.251.8.150 port *; 
	transfer-source 10.251.8.150; 
	minimal-responses yes;
	resolver-query-timeout 10;
	allow-recursion { any; 127.0.1.0; };
};

# Internal
view "1" {  # Internal
    match-clients { key DHCP_UPDATER1; !all_dns_views_updater_keys; any; };
    match-destinations { any; };
    recursion yes; 
    additional-from-cache yes;
    lame-ttl 600;
    max-cache-ttl 86400;
    max-ncache-ttl 900;
    max-udp-size 1220;
    edns-udp-size 1220;
    filter-aaaa-on-v4 yes;

I will be opening a ticket for this, since we couldn't think of anything else to check, but if anyone has any ideas, I'd love to hear them!

[#2]

Looking at your output when running dig from your workstation dig is going towards different IP's.

;; SERVER: 10.251.8.250#53(10.251.8.250)
;; WHEN: Fri Apr 29 11:17:53 Central Daylight Time 2022

;; SERVER: 10.251.8.150#53(10.251.8.150)
;; WHEN: Fri Apr 29 11:18:02 Central Daylight Time 2022

But I have similar issue in my lab, NIOS 8.6.1.

When query from my Windows client using dig there is a REFUSED from NIOS and using nslookup I do get a reply.. Settings on DNS server are default. No difference with no cookie, edns or ad flags..

Doing dig from another NIOS in the lab is working both from standard terminal and expertmode with edns and cookie.

My Windows client is using DiG 9.16.20.

NIOS is DiG 9.11.3-S3

There is firewall in between my Windows client and lab but since nslookup is working and dig is not it feels like my issue could be somehow similar to yours.

If you do a case please share your findings.

Cheers

[#3]

Hi, Just wondering, do you have any other view configured in the grid? Thanks

[#4]

We do have other views, but only one is assigned to this member.

[#5]

Good catch! It does resolve properly if I use the correct IP of that member.

I haven't opened that ticket yet (other issues) but I'll get around to it eventually. Probably. Maybe.

[#6 · Accepted solution]

AFAIK ,the "default" view must be assigned to the Grid Member you are querying using the dig widget in order to get a non-refused response.

[#7]

This is exactly it, thank you! It's good to know the cause.

Unfortunately, when our grid was initially set up, the default view was used for public facing zones, not internal, which has caused (is causing) other pains...