API & Integration

Reply
Highlighted
Accepted Solution

Importing a CSV file to Infoblox using webAPI?

Authority
Posts: 41
12092     0

Hi All,

 

I'm trying to import a csv file to infoblox via webapi.

I'm still not sure about the follwoing two fields.

Few things I’m not sure..

  • File_name : Is it just a name or complete source ? I would imagine it to be complete path of the on windows.
  • Where would the file go once its uploaded?

command:

https://mydomain/wapi/v2.1/csvimporttask?action=START&file_name="my_path_on_windows\test.csv"

Error:

{ "Error": "AdmConProtoError: Field is not searchable: action", 
  "code": "Client.Ibap.Proto", 
  "text": "Field is not searchable: action"
}

please advise
Nidhi

 

Re: Importing a CSV file to Infoblox using webAPI?

GHorne Community Manager
Community Manager
Posts: 254
12093     0

Import a CSV file

 

 

 POST /wapi/v1.5/fileop?_function=uploadinit
 Content-Type: application/json

 

 You will get back a URL for the upload, with a path of

 /http_direct_file_io/...  and a token for the upload session.  This is

 where you will be sending the file.

 

 Upload the CSV file to the URL.  The 'filedata' field is the actual

 contents of the file.  So you need to load this file into memory with

 whatever api client you are using.

 

 

   POST /http_direct_file_io/...

   {
     "name" : "my-file.csv",
     "filedata" : my-file.csv
   }

 

 Start the CSV import job

 

 You just use the original token, as this will have all the internal

 references to the filedata and it's location on the infoblox appliance.

 

 

 POST /wapi/v1.5/fileop?_function=csv_import
 Content-Type: application/json

    {
        "action":"START",
        "operation":"INSERT",
        "token": "eJydULFuwyAQ3e9...=",
        "on_error": "CONTINUE"
    }

 

 

Re: Importing a CSV file to Infoblox using webAPI?

Adviser
Posts: 131
12093     0

To add to what Geoff wrote, below is a complete Python script to initiate upload of a CSV file, upload the contents of the file, and start the CSV import task. I've tested this on Mac OS X, but not on Windows or Linux/Unix.

 

Frank

 

 

#!/usr/local/bin/python

"""
Import CSV data from a file.

This script uploads a CSV-formatted file (in Infoblox CSV format) into
the grid and starts a CSV import task.

To use this script change the 'url' variable to contain the domain
name or IP address of the grid master, and change the 'id' variable to
contain a userid with WAPI access to the grid master. (The script will
prompt for the corresponding password when run.)

If your grid master uses a TLS/SSL certificate from a commercial CA
then set the variable 'valid_cert' to True.

This script should work for NIOS 6.12 and later (WAPI 1.7 and later).
"""


# Import the required Python modules.
import requests
import json
import csv
import getpass
import sys
import os


# Set parameters to access the NIOS WAPI.
url = 'https://gm.example.com/wapi/v1.7/'
id = 'api'  # Userid with WAPI access
valid_cert = False  # True if GM uses certificate from commercial CA


# Helper functions.
def sanitize_filename(pathname):
    """Return sanitized filename without path information."""

    # Get the base filename without the directory path, convert dashes
    # to underscores, and get rid of other special characters.
    filename = ''
    for c in os.path.basename(pathname):
        if c == '-':
            c = '_'
        if c.isalnum() or c == '_' or c == '.':
            filename += c
    return filename


# Prompt for the API user password.
pw = getpass.getpass('Password for user ' + id + ': ')

# If running on Windows avoid error due to a self-signed cert.
if sys.platform.startswith('win') and not valid_cert:
    requests.packages.urllib3.disable_warnings()

# The CSV file we want to import (in the local filesystem).
csv_data = '/Users/jdoe/csv-data.csv'

# Initiate a file upload operation, providing a filename (with
# alphanumeric, underscore, or periods only) for the CSV job manager.
req_params = {'filename': sanitize_filename(csv_data)}
r = requests.post(url + 'fileop?_function=uploadinit',
                  params=req_params,
                  auth=(id, pw),
                  verify=valid_cert)
if r.status_code != requests.codes.ok:
    print r.text
    exit_msg = 'Error {} initiating upload: {}'
    sys.exit(exit_msg.format(r.status_code, r.reason))
results = r.json()

# Save the authentication cookie for use in subsequent requests.
ibapauth_cookie = r.cookies['ibapauth']

# Save the returned URL and token for subsequent requests.
upload_url = results['url']
upload_token = results['token']


# Upload the data in the CSV file.

# Specify a file handle for the file data to be uploaded.
req_files = {'filedata': open(csv_data,'r')}

# Specify the name of the file (not used?).
req_params = {'name': sanitize_filename(csv_data)}

# Use the ibapauth cookie to authenticate instead of userid/password.
req_cookies = {'ibapauth': ibapauth_cookie}

# Perform the actual upload. (NOTE: It does NOT return JSON results.)
r = requests.post(upload_url,
                  params=req_params,
                  files=req_files,
                  cookies=req_cookies,
                  verify=valid_cert)
if r.status_code != requests.codes.ok:
    print r.text
    exit_msg = 'Error {} uploading file: {}'
    sys.exit(exit_msg.format(r.status_code, r.reason))


# Initiate the actual import task.
req_params = {'token': upload_token,
              'doimport': True,
              'on_error': 'STOP',
              'operation': 'INSERT',
              'update_method': 'OVERRIDE'}
r = requests.post(url + 'fileop?_function=csv_import',
                  params=req_params,
                  cookies=req_cookies,
                  verify=valid_cert)
if r.status_code != requests.codes.ok:
    print r.text
    exit_msg = 'Error {} starting CSV import: {}'
    sys.exit(exit_msg.format(r.status_code, r.reason))
results = r.json()

# Record cvsimporttask object reference for possible future use.
csvimporttask = results['csv_import_task']['_ref']
print csvimporttask

Re: Importing a CSV file to Infoblox using webAPI?

Authority
Posts: 41
12093     0

 Hi 

 

Firstly, thank you so much for the detailed reply of my questions.

I have bene looking into archive but didnt get anything so useful.

 

i have been trying the same method as you have mentioned in your example but not getting anywhere.

POST /https://xyz/wapi/v2.1/fileop?_function=uploadinit

 

May sound a dumb question but how can I use the POST directly via browser?

OR

Has to be used with curl only as mentioned in the refrrence doc?

 

 

 

Please advice

Nidhi

 

Re: Importing a CSV file to Infoblox using webAPI?

Authority
Posts: 41
12093     0

 Hi 
 
Firstly, thank you so much for the detailed reply of my questions.
I have bene looking into archive but didnt get anything so useful.
 
i have been trying the same method as you have mentioned in your example but not getting anywhere.
POST /https://xyz/wapi/v2.1/fileop?_function=uploadinit
 
May sound a dumb question but how can I use the POST directly via browser?
OR
Has to be used with curl only as mentioned in the refrrence doc?
 
  
Please advice
Nidhi
 

Re: Importing a CSV file to Infoblox using webAPI?

[ Edited ]
Authority
Posts: 41
12093     0

Thanks Frank and Geoff!!

 

Appreciate your responses and help!

 

Will let you know if still there is any concerns,Would really appreciate if you can respond to my question.

 

thats is how to use POST directly in a URL.

 

Thanks again!

Nidhi

Re: Importing a CSV file to Infoblox using webAPI?

[ Edited ]
GHorne Community Manager
Community Manager
Posts: 254
12093     0

You can't use a POST in a url/browser, you need a client like curl or postman or a browser plugin

Re: Importing a CSV file to Infoblox using webAPI?

Moderator BRampling
Moderator
Posts: 69
12093     0

I use the Chrome Advanced REST Client extension to do in-browser REST queries, including POST. 

 

https://chrome.google.com/webstore/detail/advanced-rest-client/hgmloofddffdnphfgcellkdfbfbjeloo?hl=e...

 

It's pretty easy to use and gives you a nice interface when testing the REST API.

Re: Importing a CSV file to Infoblox using webAPI?

Adviser
Posts: 131
12093     0

A minor error in the script I posted: The line

 

req_files = {'filedata': open(csv_data,'r')}

should be

req_files = {'filedata': open(csv_data,'rb')}

 Per the documentation at python-requests.org, you should explicitly open the file as binary. On Mac OS X this shouldn't make a difference, but it might on other platforms, including Windows.

 

Frank

 

Re: Importing a CSV file to Infoblox using webAPI?

Authority
Posts: 41
12093     0

Thanks for the update Frank!

 

-Nidhi

Re: Importing a CSV file to Infoblox using webAPI?

Authority
Posts: 41
12093     0

Thank you so much!

 

Appreciate your help for the in-browser REST quesries.

Re: Importing a CSV file to Infoblox using webAPI?

Authority
Posts: 41
12093     0

Thanks for the clarification.

 

-Nidhi

 

Re: Importing a CSV file to Infoblox using webAPI?

[ Edited ]
mukulg27
Techie
Posts: 2
12093     0

Hello Frank,

You have posted a very detailed solution for CSV import to Infoblox web API and I have one quick question in the same.  

In your solution you directly assigned the complete path of the csv file to import to variable "csv_data". Can we pass the complete data of the file into csv_data instead of passing the fath of the file ?

 

Looking forward for your response.

 

Thanks & Regards,

 

Mukul

Re: Importing a CSV file to Infoblox using webAPI?

Moderator BRampling
Moderator
Posts: 69
12093     0

You cannot simply add data to that variable. The CSV import requires you to upload a file to the URL provided by the uploadinit funtion.

Re: Importing a CSV file to Infoblox using webAPI?

mukulg27
Techie
Posts: 2
12093     0

To upload the CSV file to the URL provided by the uploadinit function, do we need to pass the local path of the CSV file on the local system (where it exist) or the actual CSV file data ? 

Re: Importing a CSV file to Infoblox using webAPI?

tgraham2
Techie
Posts: 3
12093     0

I tried the above python code which died (due to user error). When I rerun I get:

 

{ "Error": "AdmConProtoError: Too many CSV import tasks are in the UPLOADED state. Only 1 UPLOADED tasks are allowed for each user.", 
  "code": "Client.Ibap.Proto", 
  "text": "Too many CSV import tasks are in the UPLOADED state. Only 1 UPLOADED tasks are allowed for each user."
}

 

Obiously I need to cancel the upload task so two questions: 1)

how do I get the task id for the upload task and 2) how do I cancel so that I can re-upload?

Thanks

 

 

 

Re: Importing a CSV file to Infoblox using webAPI?

jeffwarn
Member
Posts: 0
12093     0

Sorry to bump such an old thread, but does anyone know if there's a way to check on the CSVImport status?

 

The reason being is, there's a way to pull down the CSV_ERROR file, but you have to wait until the CSVImport job is finished.

 

The Perl module has this method available to it: https://ipam.illinois.edu/api/doc/Infoblox/Grid/CSVImportStatus.html#description

 

I have the code that runs after the CSVImportTask is done to pull down the error file, but I can't seem to figure out how to code the proper waiting time after the _function=csv_import runs in the python script provided as the example here.

 

While dropping a time.sleep(N) in the script after the csvimporttask variable is possible, there are times when a larger (2000+ line) CSV import can take anywhere from 3-5 minutes to complete, whereas a smaller import may only take a few seconds and I'd prefer not to put a ~5 minute sleep for every single csvimport task if possible.

 

Any ideas how to achieve this?

Re: Importing a CSV file to Infoblox using webAPI?

Adviser
Posts: 56
12093     0

Hi,

 

When you run the csv_import function it returns an output with the reference of the CSV import task, as shown below

WAPI Call:
curl -k -u admin:infoblox -H 'content-type: application/json' -X POST "https://grid-master/wapi/v2.7/fileop?_function=csv_import" -d '{"operation":"UPDATE","action":"START","on_error":"CONTINUE","update_method":"MERGE","separator":"TAB","token":"eJylUMtOwzAQvPtH2kuTrNu8uBWVSkiooBbOq8R2ykqJbWwHtX+PXQRHLhxs7e7M7GOEMPaKTp1Z\n/ITRPrhZBOOYBbYUpAfTj+aSGa3SC1erPNt1ocOjGpjlTCD2M42BNCKTJAKza7aUdsNOC3Wx5K4Y\naFILZku2h3Jdl7DZNJA1RVtxzvxpMbsxolXkv4dg/V2eQ5FBW2W84BmHPFVRUtwu4ECjQjK5Ux9I\ncvX28vS83a2KlkNRceDQFDXUdU6TNe6bHVvXaQrJGDVxyN/SSGoTXcYTYwxFVOQ++tGdVR4m+/9t\nAH76o9LCSNLnVOVx0P3jIYXrX8Jk5E2SzNxtX7d4fNinvGQ+HC1UyWioIziQGqXHYFCYyXbupmrY\nIR3edxZJ2zngp3KejE5YG7E++wIsS5yX\n"}'

Output:
{
    "csv_import_task": {
        "_ref": "csvimporttask/b25lLmNzdl9pbXBvcnRfdGFzayQ1:5",
        "admin_name": "admin",
        "file_name": "import_file",
        "file_size": 215,
        "import_id": 5,
        "lines_failed": 0,
        "lines_processed": 0,
        "lines_warning": 0,
        "on_error": "CONTINUE",
        "operation": "UPDATE",
        "separator": "TAB",
        "start_time": 1537512228,
        "status": "PENDING",
        "update_method": "MERGE"
    }
}

Using the CSV import task reference, you can query and fetch the status of your task

WAPI Call:
curl -k -u admin:infoblox -X GET "https://grid-master/wapi/v2.7/csvimporttask/b25lLmNzdl9pbXBvcnRfdGFzayQ1:5"

Output:
{
    "_ref": "csvimporttask/b25lLmNzdl9pbXBvcnRfdGFzayQ1:5",
    "admin_name": "admin",
    "end_time": 1537512233,
    "file_name": "import_file",
    "file_size": 215,
    "import_id": 5,
    "lines_failed": 0,
    "lines_processed": 3,
    "lines_warning": 0,
    "on_error": "CONTINUE",
    "operation": "UPDATE",
    "separator": "TAB",
    "start_time": 1537512233,
    "status": "COMPLETED",
    "update_method": "MERGE"
}

You can even just fetch the status and keep polling it till it reaches the COMPLETED or FAILED state.

WAPI Call:
curl -k -u admin:infoblox -X GET "https://grid-master/wapi/v2.7/csvimporttask/b25lLmNzdl9pbXBvcnRfdGFzayQ1:5?_return_fields=status"

Output:
{
    "_ref": "csvimporttask/b25lLmNzdl9pbXBvcnRfdGFzayQ1:5",
    "status": "COMPLETED"
}

Hope this helps,

Krishna

Re: Importing a CSV file to Infoblox using webAPI?

jeffwarn
Member
Posts: 0
12093     0

Thanks! That was easy enough.. If anyone is interested in the downloading of the csv_error file, please let me know and I'll post the additional code.

Re: Importing a CSV file to Infoblox using webAPI?

mirkogeraci
Techie
Posts: 2
12093     0

Hi,

 

I'm trying to import Name server group that ' exported previously from GUI.

The error is the following:

{
"Error": "AdmConProtoError: Function csv_import\" is not valid for this object",
"code": "Client.Ibap.Proto",
"text": "Function csv_import\" is not valid for this object"
}

It seems that is not possible to import csv that contains  some kind of object (member, name server group ) 

Do you know is it correct?

Regards

Mirko

Re: Importing a CSV file to Infoblox using webAPI?

Antoine5931
Techie
Posts: 4
12093     0

Hello,

 

Thanks a lot for all yours sharing.

 

Is it possible to export objects from a view/zone, and to have it directly in the CSV format, ready to be imported ?

I would like perform an export from a specific view/zone, and import to a specific view/zone.

It will be nice if you have an idea Smiley Happy For the moment, i'm only able to export it in a Json format, not CSV.

 

Thanks in advance, and have a nice day.

Re: Importing a CSV file to Infoblox using webAPI?

Adviser
Posts: 56
12093     0

Hi,

 

You can export all the host records within a view/zone into a CSV using the following set of WAPI calls.

 

Initiate a CSV export of all the host records in the zone 'demo.com' within the network view 'test':
curl -k -u admin:infoblox -H 'content-type: application/json' -X POST "https://grid-master/wapi/v2.9/fileop?_function=csv_export" -d '{"_object": "record:host","view":"default.test"}'

Sample output:
{
    "token": "eJytUEFuwyAQvPOR5BLb62A77i2VG7VSlUhJpR5XNuAUyQYKJEp+36VSe+2lB9Cys8PsjBDW3dGr\nM6NLWBOiv4hoPXPAlkKb0Q6TvWXWqHTi3anAuj72eFQjcyUTiMNFT1EbRCa1iMyt2VI6zk4LdXPa\n3zHqWS2Yq9gOKl40bcuhyXgFZV0BC6fFxU8E10T4iNGFhzyHIoO2zsqizErIUxelpvUijnpSqG3u\n1SdqueoO7/vXw7ZbQVG2RV0V0HC+WXOeP1tyQoa8DJkIVxJokpaWVG1I6q8PaKxNBEleqYaCOHmg\nYPqzyuPs/msrgB8VVEZYqc05dUuSe3zZp3L9OzBbmYKElG23fdvi8WmX3hUL8eigTrlDQ+Co1SQD\nRovCzq7336wN26cAht6hNu4S8ap80NYkrCVsyL4AsxyhcQ==\n",
    "url": "https://10.196.202.21/http_direct_file_io/req_id-DOWNLOAD-1029065017448344/Hostrecords.csv"
}

Download the CSV file:
curl -k -u admin:infoblox -H 'content-type: application/force-download' "https://grid-master/http_direct_file_io/req_id-DOWNLOAD-1029065017448344/Hostrecords.csv" -o "Hostrecords.csv"

Remove the stored file using the token:
curl -k -u admin:infoblox -H 'content-type: application/json' -X POST "https://grid-master/wapi/v2.9/fileop?_function=downloadcomplete" -d '{"token" : "eJytUEFuwyAQvPOR5BLb62A77i2VG7VSlUhJpR5XNuAUyQYKJEp+36VSe+2lB9Cys8PsjBDW3dGr\nM6NLWBOiv4hoPXPAlkKb0Q6TvWXWqHTi3anAuj72eFQjcyUTiMNFT1EbRCa1iMyt2VI6zk4LdXPa\n3zHqWS2Yq9gOKl40bcuhyXgFZV0BC6fFxU8E10T4iNGFhzyHIoO2zsqizErIUxelpvUijnpSqG3u\n1SdqueoO7/vXw7ZbQVG2RV0V0HC+WXOeP1tyQoa8DJkIVxJokpaWVG1I6q8PaKxNBEleqYaCOHmg\nYPqzyuPs/msrgB8VVEZYqc05dUuSe3zZp3L9OzBbmYKElG23fdvi8WmX3hUL8eigTrlDQ+Co1SQD\nRovCzq7336wN26cAht6hNu4S8ap80NYkrCVsyL4AsxyhcQ==\n"}'

Hope you find this useful,

Krishna

 

Re: Importing a CSV file to Infoblox using webAPI?

Antoine5931
Techie
Posts: 4
12093     0

Hi Krishna

Thanks for your answer, yes, it will helpful for my use case.

 

I have to now translate it in Python, like on this thread:
https://community.infoblox.com/t5/API-Integration/Importing-a-CSV-file-to-Infoblox-using-webAPI/m-p/...

I use this scrip to perform the import, it works nicely.


Don't hesitate to share if you already did it Smiley Happy

Have a nice day,

Antoine

Re: Importing a CSV file to Infoblox using webAPI?

technic123
Techie
Posts: 2
12093     0

I dont have any idea about this but one of my friends was found this website very helpful he was facing the issue

printer not activated error code 30 

in any printer software from here he got a solution.

Re: Importing a CSV file to Infoblox using webAPI?

jowens1
Techie
Posts: 2
12093     0

Hey @jeffwarn, could you please post the code for downloading the error file?

 

Also, my status always shows as COMPLETED even if there is an error. Any ideas what's going on with that?

Showing results for 
Search instead for 
Do you mean 

Recommended for You