Grid database restore [Python] [ Edited ]

[#1]

Dear All,

I am receiving the following error on Grid Restore using AWS Lambda function written in Python:

{ "Error": "AdmConDataError: None (IBDataError: IB.Data:Restore Database : Grid Restore Failed - Extracting file failed, /storage/tmp/http_direct_file_io/req_id-UPLOAD-1090/errs)",
"code": "Client.Ibap.Data",
"text": "Restore Database : Grid Restore Failed - Extracting file failed, /storage/tmp/http_direct_file_io/req_id-UPLOAD-1090/errs"
}

From in the Infoblox.log:

[2020/02/20 17:13:04.379] (5314 /infoblox/one/bin/restore_node) : Grid Restore started
[2020/02/20 17:13:04.477] (5314 /infoblox/one/bin/restore_node) : untarring of the file failed
[2020/02/20 17:13:04.490] (5314 /infoblox/one/bin/restore_node) :
[2020/02/20 17:13:04.491] (5314 /infoblox/one/bin/restore_node) : gzip: stdin: not in gzip format
[2020/02/20 17:13:04.491] (5314 /infoblox/one/bin/restore_node) : tar: Child returned status 1
[2020/02/20 17:13:04.491] (5314 /infoblox/one/bin/restore_node) : tar: Error is not recoverable: exiting now

Please find attached the Python code:

import boto3
import datetime
import io
import os
import json
from botocore.vendored import requests
import urllib3
import tarfile
import gzip
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)


s3_bucket_name = os.environ['BUCKET_NAME']
username = 'admin'
password = 'xyz'
api_version = 'v2.7'
gm_ip = os.environ["gm_ip"]
gmc_ip = os.environ["gmc_ip"]

def lambda_handler(event, context):
    file_name = event['file_name']

    try:
        infoblox_backup(gm_ip, file_name)
    except:
        infoblox_backup(gmc_ip, file_name)

# Helper functions.
def sanitize_filename(pathname):
    """Return sanitized filename without path information."""

    # Get the base filename without the directory path, convert dashes
    # to underscores, and get rid of other special characters.
    filename = ''
    for c in os.path.basename(pathname):
        if c == '-':
            c = '_'
        if c.isalnum() or c == '_' or c == '.':
            filename += c
    return filename

def infoblox_backup(ip, file_name):
    url = 'https://' + ip
    multipart_headers = {
        'content-type': 'multipart-formdata',
    }
    json_headers = {
        'content-type': 'application/json',
    }
    session = requests.Session()
    session.auth = (username, password)

    # Download Backup File from S3
    s3 = boto3.resource('s3')
    s3.Bucket(s3_bucket_name).download_file(file_name, '/tmp/' + 'dev_infoblox_backup_restore.bak')

    # The BAK file we want to import (downloaded from s3 to Lambda /tmp/).
    bak_data = '/tmp/' + 'dev_infoblox_backup_restore.bak'

    # Initiate a file upload
    initiate_backup_path = "wapi/{}/fileop?_function=uploadinit".format(api_version)
    initiate_backup_url = '{}/{}'.format(url,initiate_backup_path)

    req = session.post(initiate_backup_url, headers=json_headers, verify=False)
    
    # url and token from results above
    resp = req.json()
    token = resp.get('token')
    upload_url = resp.get('url')

    # Specify a file handle for the file data to be uploaded.
    req_files = {'filedata': open(bak_data,'rb')}

    # Perform the actual upload. (NOTE: It does NOT return JSON results.)
    req = session.post(upload_url, headers=multipart_headers, files=req_files, verify=False)
    resp = req.text
        
    # Restore the database
    restore_backup_path = "wapi/{}/fileop?_function=restoredatabase".format(api_version)
    restore_backup_url = '{}/{}'.format(url,restore_backup_path)
    token_replace = token.replace("\n","")
    restore_backup_payload = f'{{"mode":"NORMAL","token":"{token_replace}"}}'

    # Initiate the actual import task.
    req = session.post(restore_backup_url, headers=json_headers, data=restore_backup_payload, verify=False)
    resp = req.text
    print(resp)

def get_account_alias():
    ''' Returns AWS Account Alias or Account Id if no Alias is set '''
    session = boto3.Session()
    iam = session.client('iam')
    client = session.client("sts")
    response = iam.list_account_aliases()
    account_id = client.get_caller_identity()["Account"]
    try:
        account_aliase = response['AccountAliases'][0]
        return account_aliase
    except:
        # No Account Alias Set return Account id instead
        return account_id
if __name__ == "__main__":
    lambda_handler({},{})

I confirm I have successfully managed to do Restore manually using UI using the file I am downloading from AWS S3.

May you please advise?

Thank you in advance.

Slobodanka

[#2]

Hi, have you managed to get some response? I am experiencing same problem, but with ansible.

Thanks,

M.

[#3]

Any luck on this?  

'{ "Error": "AdmConDataError: None (IBDataError: IB.Data:Restore Database : Grid Restore Failed - Extracting file failed, /storage/tmp/http_direct_file_io/req_id-UPLOAD-0318142430515476/errs)", \n "code": "Client.Ibap.Data", \n "text": "Restore Database : Grid Restore Failed - Extracting file failed, /storage/tmp/http_direct_file_io/req_id-UPLOAD-0318142430515476/errs"\n}'

Failing with file extraction as well.

[#4]

Hello everyone,

It seems like there's a problem with the file upload. Please ensure that no headers are used with the file upload block. In my Python script, I utilize the session class and directly employ requests to bypass headers from the session. If anyone requires it, I have a functional Python script available. Check this out if you're interested:

https://github.com/Farseencbv/infoblox-python-scripts.git