How safe is it to restart DNS services during the workday?

[#1]

I'm trying to ascertain the risks involved in, say, creating a new zone and restarting DNS services to bring it to life during the workday. Those who I know that work as Infoblox admins tell me that they do it from time to time with no ill effects, not that they necessarily do it daily. Obviously, the cache usually takes a hit, but is that enough to cause client failures, especially if restart groups don't do everything simultaneously?

Thanks

[#2]

Hello,

The DNS restart usually takes just for a few seconds and of course, the services won't be served during that time. If your clients are configured with a secondary DNS server and that you are restarting the servers in a sequential order, I guess the impact could be minimal. Again, deciding to restart the DNS service during business hours depends on how tolernant the production environments can be to this. 

[#3]

I suggest two things:

Only restart IF NEEDED.  If you force a restart, those services which do not have pending changes will still be impacted.

Do not restart all services simultaneously.  This would impact services across the entire grid.  As long as clients are using more than one DNS service for example, then if a request fails to one member because it's restarting, then the request would go to another member that should still be in service.

[#4]

> if a request fails to one member because it's restarting, then the request would go to another member that should still be in service.

Which of course assumes that the application re-transmits the DNS query after a certain time-out, so you will either see a short delay whilst the DNS query is retransmitted, or the application will fail. One would hope that the application would retransmit but who knows how they will behave?

Personally I would try and delay any restarts until an off-peak time of the day, all our changes are done under change control anyway, and a restart indicates that something quite fundamental has changed (e.g. new zone or ACL change) - so that definitely should be done under change control and the restart done during an agreed change window.

Something else you have to consider, an Infoblox DNS/DHCP restart is not the same a reload - in BIND for instace you can issue an rndc reload or rndc reconfig command to reload the config, but the DNS server process (named) actually keeps running, so it's a lot less intrusive. But Infoblox actually stops and restarts the DNS/DHCP daemons - I always thought this was quite drastic but guess it guarantees that any changes get picked up (e.g. changes to the underlying network config such as new loopback interfaces).

It would be great if Infoblox had a less severe restart option, i.e. it can already detect what changes require a restart so if something only required a reload than a restart then I'm sure logic could be built into the system to do that. I would feel more comfortable doing a reload rather than a restart during working hours.