01-05-2017 01:21 PM
I've been doing a lot of work in the lab environment exporting csv files via the RESTful API. One thing I have noticed is that when I export a zone's records, the csv file almost always contains duplicate hostrecords. I discovered this by watching the csv import jobs (imported the file into a different view) and looking at the error count. I then downloaded the errors and all of them were due to "Insertion aborted due to The record <insert record name here> already exists."
I then went back and looked at the original csv file that was exported, and indeed, it contains duplicate hostrecords. Any idea why the csv export would contain duplicate records? I don't know if this is a database problem or something specific to the csv export process. This has me concerned because my lab environment is an exact copy of our production environment.
For now I will have to perform extra processing. I'll have to sort the lines in the csv file and remove the duplicates. Also have to set the import to "on error" "continue" or else it will abend.
I'd like to find out why I'm getting duplicate records. Has anybody else seen this? I'm running NIOS 7.3.8.
Solved! Go to Solution.
01-05-2017 01:39 PM - edited 01-05-2017 01:43 PM
I just verified this issue. The unprocessed csv export file does contain duplicate records. About 17% of the records are duplicate and the issue is repeatable. Same duplicate records according to sample I checked. The weird thing is that I do not get any duplicate records if I perform the csv export from the web gui. This might be a bug with the API.
02-02-2017 11:46 AM
Can you provide a sample of the duplicates ?
the CSV export often puts out both host records and 'hostaddress' objects
(the latter being redundant and useful for corner case situations)
so you may just be seeing these.
02-02-2017 11:54 AM
You are exactly right. I'm getting duplicate host records and hostaddress records. As long as my CSV imports are set to NOT stop on error, they proceed and complete. I came to the conclusion the duplicates don't hurt anything. I now preprocess my CSV files before importing them into a different DNS view to remove the duplicates.
I still get duplicates when I import a forward and reverse zone because the hostrecord creates the reverse entry for me. Then, when importing the reverse zone, some of the records are already there, some aren't. This has been a good learning experience.