08-09-2017 10:08 AM - edited 08-09-2017 10:12 AM
I’m struggling to stay under my 20 gig per day limit and still get the syslog information I need into the reporter.
For the Reporting Properties, Report Category. There are many tools and documentation to see what categories are driving up my usage and what reports will stop working if I disable that category.
For syslog categories I seem to be flying blind and syslog is by far my largest consumer of the 20 gig.
I’m stuck turning off things that I think might make a difference, wait a day and see if that specific syslog category made a significate difference.
If it did, I then go through my reports and dashboards and see what reports no longer have data and decide if that is something I can do without or not.
Are there any reports in place to tally up the syslog categories per day?
Any easy way to pull the syslog category to report dependency?
A splunk query that can put in the Infoblox “Syslog Report Category” and just match all the syslog events currently in the reporting database? Just so I can try and see what I’m turning off, before I turn it off.
08-10-2017 01:26 PM
It doesn't appear that the Infoblox catagory information makes it into the reporting server. It seems everything gets dropped into the syslog catagory local1 or local6. Sounds a Request for Enhancement is in order.
08-15-2017 07:19 PM
To the best of my knowledge, none of the predefined reports/dashboards use syslog data. Turning off syslog data indexing completely however, will affect any and every custom reports/dashboards you or other admins may have crafted using data from 'index=ib_syslog'.
Additionally, while working on a grid with reporting, running NIOS version 8.2.1, which also has reporting clustering setup, I just noticed that any changes we make to "Administration-->Reporting-->Grid Reporting Properties-->Syslog Data-->Logging Category" section does not take effect. If I make the changes under "Grid-->Grid Manager-->Reporting", on a per member basis, everything works correctly.
While I am immediately not certain whether this an issue specific to this grid or whether it has anything to do with reporting clustering setup, I shall review it further and reach out to Engineering if required.
Regarding an ability to review something like "Syslog Volume Usage Trend per Category" similar to the current "Reporting Volume Usage Trend per Category" report sounds like a good candidate for an RFE.
08-16-2017 07:03 AM
I've seen some issues where I didn't think that the changes took place either in
Administration-->Reporting-->Grid Reporting Properties-->Syslog Data-->Logging Category
That was part of my struggle I think. It seems like maybe some combination / order of restarting the reporting service AND the related actual service (named, dhcpd... etc) seem to make it take affect later. Or maybe some not until the nightly "daily data generation" on the same grid reporting properties tab?
I'm not sure but there is some variability there that makes managing this fairly frustrating.
08-17-2017 08:48 AM
I think I am seeing the changes in the severity choice take affect and maybe not in the category?
08-30-2017 07:33 AM - edited 09-06-2017 09:43 AM
In playing with this some more, it appears that over half of my syslog data appears to be falling into the "Non-Categorized" heading. That may be why I am having so much difficulity determining what affect changing any of the other categories has on the over all data size.
08-30-2017 03:08 PM
If you havent alreay, it might be worth using the Splunk native "patterns" evaluation to see if there is any clues on what you might be able to turn off. In the example below, you can see that syslog based query logging is sending a bunch of messages and accounts for almost %60 of the syslog traffic.
09-06-2017 10:00 AM - edited 09-06-2017 10:01 AM
I had some more time to play with this and I was close to getting everything I needed until I got to the "Recursive DNS Performance & Troubleshoot Dashboard". Almost all the events used to populate the "bottom half" of the dashboard, appear to fall under the "Non-catergorized" heading.
The Non-catergorized heading for me is about 10 gig of data per day. There really needs a little more granularity there to be useful.
Quick fix would be another heading for those kinds of DNS messages. They don't really fall into any of the other DNS listings.
Or the ability to define your own REGEX's and categories in that list as an RFE?
09-18-2017 04:24 PM
When we turned on Syslog for 16 minutes it generated approximately .85 gb per minute of syslog data. That means that having syslog going into the reporting appliance maybe a bridge too far.
09-19-2017 06:01 AM
I agree. Infoblox really needs to work on the granularity of the syslog controls. To populate the reports that I would really like to have it looks like I need about 30mb of syslog messages total per day. However, they are included in the "unclassified" heading which runs about 10gb a day.
What is even more frustrating is that they are standard DNS message logs and there are several different DNS categories, but these are not in any of them and fall through with all the noise is the catch all category.
09-27-2017 12:21 PM