DNS DHCP IPAM

Reply
Highlighted

DHCP Leases - Abandoned IP addresses

Authority
Posts: 18
29623     0

Has anyone else encountered problems with IP addresses within DHCP ranges that for one reason or another have been 'abandoned'? I know that this can happen if a server offers an address, and the client rejects it.

The problem I keep encountering every so often is that the range utilisation statistics in the GUI do not take abandoned addresses into account. At first glance it can appear that a range is only 50% utilised but in reality it could be the case most of that free 50% is not being leased out as the servers have marked them as abandoned at some stage in the past. The first I know about it is when users complain of not being able to obtain a lease and I find both DHCP servers are reporting "peer holds all free leases", despite there being lots of unassigned addresses within the range.

My fix is to go into Data Management / DHCP / Leases, filter on the relevant subnet, and start deleting any leases I find that are in state 'Abandoned', which is really a bodge fix.It would be really useful if Infoblox presented this information in the Utilisation statistics when viewing the DHCP ranges.

 

Hi Andrew,

GAgyapong
Techie
Posts: 11
29624     0

Hi Andrew,

I have a very large environment, and I very often encounter the very same "abandoned" IP issue.

In my environment we discovered that the devices causing this abandonement issues were Android devices, well documented in the Princeton University article below. 

https://www.net.princeton.edu/android/android-stops-renewing-lease-keeps-using-IP-address-11236.html

Of course, this still does not address the fact that Infoblox is not including the "abandoned" address in the utilization stats.  I completely agree with you on that point.  The utilization bar is useless if it can accurately give us the real IP use.

Anyway I hope the Princeton article perhaps, helps you identify the root of the issue.

Thanks for the reply and for

Authority
Posts: 18
29624     0

Thanks for the reply and for posting that link. We have also found this problem exists on Windows PCs in power saving mode, which the majority of our staff use. Windows likes to offload ARP to the network card, and unfortunately the network card keeps responding to ARP requests on IP addresses for which the PC no longer has a lease for. It continues until someone wakes up the PC and the machine negotiates a new lease. Very annoying!

 

Thanks for sharing the

GAgyapong
Techie
Posts: 11
29624     0

Thanks for sharing the Windows ARP issue.  I was not aware of this change in Windows.  Do you know which version/s of Windows desktops you witnessed this behavior by a chance?

Windows 7 enterprise edition.

Authority
Posts: 18
29624     0

Windows 7 enterprise edition. 

https://social.technet.microsoft.com/Forums/windowsserver/en-US/b9d5cb1c-cddc-4769-b36e-dd8ee9304990...

Our systems team are in the process of rolling out a powershell script that will hopefully disable the ARP OFFLOAD feature.

Re: DHCP Leases - Abandoned IP addresses

Expert
Posts: 164
29624     0

Hi, I've just been doing some testing, from what I am seeing, the DHCP server will start using abandoned leases when all free addresses have been used up - however, if clients are still responding to pings on the abandoned leases, then obviously the DHCP server still can't issue them.

 

What I am trying to say is that it shouldn't be necessary to trawl through the subnets deleting abandoned leases as the server will attempt to use them when it needs to. I think it's only really necessary if you are doing a migration and need to delete abandoned leases because you are waiting for the legitimate clients to renew their leases and repopulate the lease database.

 

If manually deleting abandoned leases fixed it for you, then I'm wondering if there's something else going on here.

Paul Roberts
PCN (UK) Ltd

All opinions expressed are my own and not representative of PCN Inc./PCN (UK) Ltd. E&OE

Re: DHCP Leases - Abandoned IP addresses

JEarickson
Techie
Posts: 6
29624     0

Here is a perl API script that I run every night at 4 AM via cron on a Linux host, to kill off abandoned leases.

Note the "your-password-here" part below, otherwise generic to any Infoblox system.  If you want to see what it is doing, then uncomment the data-dumper lines.

 

#!/usr/bin/perl

# Always use 'strict' and 'warnings'
#use strict;
use warnings;

use Getopt:Smiley Frustratedtd;
#use IO:Smiley Frustratedocket:Smiley FrustratedSL qw(debug3);
use Data:Smiley Very Happyumper;
use Infoblox;

#---subroutine prototypes
sub usage();

#---global variables
my $session;
my ($server, $username, $password);
my $num_abandoned = 0;

#===================
#=== subroutines ===
#===================
sub usage()
{
    print "Find and remove abandoned leases\n";
    print "Usage is: $0 [-d number] -s server -u username\n";
    print "\t-d\tdebug level\n";
    print "\t-s\tInfoblox server name\n";
    print "\t-u\tusername\n";
    exit 1;
}

#====================
#=== main routine ===
#====================

getopts("d:s:u:") || &usage;
$server = $Getopt:Smiley Frustratedtd:Smiley Surprisedpt_s;
$username = $Getopt:Smiley Frustratedtd:Smiley Surprisedpt_u;

#---prompt for the password
#if ( $server and $username) {
#    unless ( $password ) {
#        system "stty -echo";
#        print "Password: ";
#        chomp($password = <STDIN>);
#        print "\n";
#        system "stty echo";
#        exit unless $password;
#    }
#}
#else {
#    &usage;
#}

#---create the session handler
$session = Infoblox:Smiley Frustratedession->new(
    "master" => $Getopt:Smiley Frustratedtd:Smiley Surprisedpt_s,
    "username" => $Getopt:Smiley Frustratedtd:Smiley Surprisedpt_u,
    "password" => "your-password-here",
    "timeout" => 3600,
);

#---See if the connection attempt worked
if ( $session->status_code() ) {
    printf("session[%d] %s\n", $session->status_code(), $session->status_detail());
    exit 1;
}

#---pull network info for all networks
my @network_array = $session->search(
    object => "Infoblox:Smiley Very HappyHCP::Network",
);
if ( $session->status_code() ) {
    printf("network[%d] %s\n", $session->status_code(), $session->status_detail());
}

#print "network data dump\n";
#print Dumper (\@network_array);

#print "===============================================\n";

#---sort network array of hashes, first by third octet of IP, then
#---by fourth octet/mask of IP
my @keys = sort {
(split/\./,$a->{network})[2] <=> (split/\./,$b->{network})[2] ||
(split/[\.\/]/,$a->{network})[3] <=> (split/[\.\/]/,$b->{network})[3]
} @network_array;

#---for each network
foreach my $key (@keys)
{
    #---pull DHCP range for network
    my @ranges= $session->get(
        object => "Infoblox:Smiley Very HappyHCP::Range",
        network => $key->{'network'}
    );

    #---if network has a DHCP range, look for abandoned leases
    if (! $session->status_code() ) {
        #print "$key->{'network'}: $key->{'comment'} \n";
        #print "range data dump\n";
        #print Dumper (\@ranges);

        #---cover all DHCP ranges in this network
        foreach my $range (@ranges) {
            #print "\tDHCP range: $range->{'start_addr'} - $range->{'end_addr'}\n";

            #---get IPAM objects for this network
            my @ipam_objects = $session->get(
                object => 'Infoblox::IPAM::Address',
                network => $key->{'network'},
                start_addr => $range->{'start_addr'},
                end_addr => $range->{'end_addr'}
            );
            #print Dumper (\@ipam_objects);

            #---for every IP address in the DHCP range
            foreach my $ip (@ipam_objects) {
                #---found an abandoned lease, stomp it
                if($ip->{'lease_state'} eq 'abandoned') {
                    #print "\t$ip->{'ip_address'}: $ip->{'lease_state'}\n";
                    #print Dumper (\$ip);

                    #---ping the system to see if it is alive
                    system("ping -q -c 1 -w 1 $ip->{'ip_address'} > /dev/null");
                    if($? >> 8 == 0){
                        print "\thost $ip->{'ip_address'} pings!\n";
                    }
                    #---no ping, remove it
                    else {
                        $num_abandoned++;
                        my $desired_lease = $session->get(
                            object => 'Infoblox:Smiley Very HappyHCP::Lease',
                            ip_address => $ip->{'ip_address'}
                        );
                        $session->remove($desired_lease)
                            or die("Remove dhcp lease failed: ", $session->status_code() . ":" . $session->status_detail());
                        print "\tdhcp lease for $ip->{'ip_address'} removed\n";
                    }
                }
            }
        }
    }
}
print "$num_abandoned abandoned leases found and crushed\n";
$session->logout();
exit;

Re: DHCP Leases - Abandoned IP addresses

GHorne Community Manager
Community Manager
Posts: 254
29624     0

That script is definitely the worst way to solve the problem. Aside from the fact that deleting abandoned leases shouldn't be the solution to the overall problem, The script will create a lot of overhead: It gets ALL the networks, then ALL the ranges then gets ALL the IP addresses, the GETs a lease.

 

You are better off just getting all the DHCP ranges, then using that to get all the leases within that range, then just flitering on those that are Abandoned.

Re: DHCP Leases - Abandoned IP addresses

GHorne Community Manager
Community Manager
Posts: 254
29624     0

The DHCP server will report a lease as abanonded either because the client says the address is in use  (because it saw an ARP or PING on the subnet) or because the server sees it as in use (either via an ARP or a PING)

 

But however you look at it an abandoned lease is a problem on the client end and shouldn't be reflected in the DHCP statistics count. That value is a record of the DHCP leases and addresses that the server is managing.

 

If you staticly assign a device inside a DHCP range, and thus that address gets marked as abanonded, should that address then get marked as 'USED' and hence imply that there is an active lease at that address ? (which there isn't) The same logic applies for PC's who let their lease expire, but contiune to use the address.

 

As Paul says, Abandoned leases still get issued, but only as a last resort, and only if the address really isn't in use.

 

There are probably 2 workarounds/fixes to this:

- We could report an additional statistic : abandoned leases per subnet

- use discovery which will reprt the address as 'in conflict' which is a more accurate idea of what is going on.

 

but realistlcally a snapshot isn't going to help, you would want to track this over time so see if the # of abandoned leases is changing. If it isn't doing down, then you need to search and destroy devices holding onto addresses that shouldn't.

 

FYI - there may be a way to do this with the reporting server, I will have to check.

 

Re: DHCP Leases - Abandoned IP addresses

Authority
Posts: 18
29624     0

My experience at our site, on more than one occasion has been that Infoblox does not start issuing out the Abandoned leases, which was what led us to have to manually clear out the Abandoned ones. I acknowledge that the manual says it should start to re-use them as a last resort, but I have seen at least twice scenarios where all that remains free to lease out are the previously abandoned ones, and both DHCP failover members are reporting "peer holds all free leases". 

 

This is what prompted me to suggest that the addresses in abandoned state need to be reflected in some way in the utilisation stats, as you can end up in a situation where neither server thinks it has any free addresses to lease out, but the utilisation percentages look healthy.

Re: DHCP Leases - Abandoned IP addresses

Authority
Posts: 21
29624     0

We also seem to be seeing this problem.  It seems to be most serious on certain of our wireless LANs - oddly not those which should be the busiest.  We have wondered about this, but I had always assumed that the Infoblox DHCP server was behaving properly and clearing the abandoned leases.  We seem to now have a clear case of a large network in a fairly low traffic area which is spewing quite a few "peer holds all free leases" with no clear cause.

 

Does the DHCP server log any sort of message when it reclaims an abandoned lease?  I haven't been able to find any.

 

I'd say that the posting of abandoned counts alongside the utilization information would be very helpful.  Has anyone opened a ticket on this or even better got an RFE number that we can jump on?

Re: DHCP Leases - Abandoned IP addresses

[ Edited ]
Authority
Posts: 21
29624     0

I must say, your script gets very interesting once copied and pasted:

 

use Getopt:Smiley Frustratedtd;
#use IO:Smiley Frustratedocket:Smiley FrustratedSL qw(debug3);
use Data:Smiley Very Happyumper;
 
$server = $Getopt:Smiley Frustratedtd:Smiley Surprisedpt_s;
$username = $Getopt:Smiley Frustratedtd:Smiley Surprisedpt_u;

 

 

A little quick editing should clean it up.

Re: DHCP Leases - Abandoned IP addresses

ssmcdonald
Techie
Posts: 1
29624     0

We had the same problem when we migrated from QIP to IB for DHCP.  We found the issue was due to IP tracking turned on by default on some of our Cisco switches.  IP tracking kept an arp table on the switches of all places.  So when the client did an ARP on its local network to check to see if the DHCP Offer was not used, the switch replied to the ARP, so the DHCP Client sent back a notice to IB who inturn put the address in an abandoned state.

 

We have netmri and ran a job to look for IP tracking on all our Cisco gear.  We then opened a TAC case with Cisco on the best way to remove/ disable tracking for each type of appliance.

 

Once we removed the tracking, we cleared all the abandoned addresses and havent had the problem since.

 

Hope this helps.

Re: Windows 7 enterprise edition.

GAgyapong
Techie
Posts: 11
29624     0

Thanks you AEverett!

Showing results for 
Search instead for 
Do you mean 

Recommended for You