Most efficient way to extract DHCP Leases

[#1]

Hello,

I am trying to integrate with an Infoblox DDI in order to extract all of the leases periodically and currently im using the "/lease" API endpoint and extracting it all page by page.
My problem is that this Infoblox has a very large amount of active leases at all time (100k+) and it's taking a long time to extract all of it. I'm also afraid of abusing the API and hurting the overall performance of the server.
So i have 2 questions:
1. Is what im doing currently hurting the performance of the Infoblox server?
2. Is there a way to do what im doing faster and more efficiently? Maybe a different API endpoint i can use? Or perhaps an easy way to parallelize the API calls?

Thanks,
Nir

[#2 · Accepted solution]

Possibly the most efficient is the pull the lease file from each appliance.  You'd need to do this for each DHCP appliance.  API calls still go against the gridmaster but you must specify which DHCP member you want to pull the data from.

#
# download lease log of a member as a CSV file
# start time and end time seem to be mandatory, even though docs don't say so
#

curl -k1 -u admin:infoblox -X POST \
'https://192.168.1.2/wapi/v2.6/fileop?_function=getleasehistoryfiles' \
-H "Content-Type: application/json" \
-d '{"member": "gm.example.org", "start_time": "1514764800", "end_time" :"1533686400"}'

# returns:

{  "token": "eJytUMFuwyAMvfMj3aUkNA1NduuUVZtUtVI7aUcrAZJaSoABqdp9/WDSdtpxB9Czn+33bCGMvYNT\nA4mfMNoHN4tgHLGMPAjUvelGc6NGq/TC3SpPmja0cFI9sSsiALoZx4AagEgUgdiCPEi7JueFull0\ndwg4qQWxJdmxsmCcl7zmdJ0XPK8Y8efF7MZI89hwCcH6xyxjOa0p45RnKQMSo7UAPY4K0GROfQDK\nZXN8P+yP22a5yqvNOpMXYfeq9eoFffR/p8Jf6fAZJ2+SCMqIqqjxZ3fk6lQl42YRszwWZmlMO6gs\nTPZffDD2IwFKCyNRDym7ilpPr4cEi9+Cych0M5bO2GzftnB63qW4JD6cLOPpxGwTyR7VKD0EA8JM\ntnXfXRU5pJW71gJqOwe4KufR6MTVkevoF4DuoAQ=\n", 
    "url": "https://192.168.1.2/http_direct_file_io/req_id-DOWNLOAD-20874/dhcpLeaseHistory.csv.gz" }

# download file

curl -k1 -u admin:infoblox -H "Content-type:application/force-download" -O \
 'https://192.168.1.2/http_direct_file_io/req_id-DOWNLOAD-20874/dhcpLeaseHistory.csv.gz'
 
 # and then close the file using the token
 
curl -k1 -u admin:infoblox -X POST 'https://192.168.1.2/wapi/v2.6/fileop?_function=downloadcomplete' \
-H "Content-Type: application/json" -d '{ "token": "eJytUMFuwyAMvfMj3aUkNA1NduuUVZtUtVI7aUcrAZJaSoABqdp9/WDSdtpxB9Czn+33bCGMvYNT\nA4mfMNoHN4tgHLGMPAjUvelGc6NGq/TC3SpPmja0cFI9sSsiALoZx4AagEgUgdiCPEi7JueFull0\ndwg4qQWxJdmxsmCcl7zmdJ0XPK8Y8efF7MZI89hwCcH6xyxjOa0p45RnKQMSo7UAPY4K0GROfQDK\nZXN8P+yP22a5yqvNOpMXYfeq9eoFffR/p8Jf6fAZJ2+SCMqIqqjxZ3fk6lQl42YRszwWZmlMO6gs\nTPZffDD2IwFKCyNRDym7ilpPr4cEi9+Cych0M5bO2GzftnB63qW4JD6cLOPpxGwTyR7VKD0EA8JM\ntnXfXRU5pJW71gJqOwe4KufR6MTVkevoF4DuoAQ=\n"}'

[#3]

Hi. How can I know the member with a call to wapi?. I obtain this error: 

{ "Error": "AdmConDataError: None (IBDataError: IB.Data:Get DHCP Lease History Files : No logging member specified in Grid DHCP Properties.)",
"code": "Client.Ibap.Data",
"text": "Get DHCP Lease History Files : No logging member specified in Grid DHCP Properties."
}