Reply
Highlighted

DNS Top Client - New Top IP not seen in the last week

Adviser
Posts: 81
3954     2

When it comes to Top Clients, a common question is about baselining:

"Yes these IPs are my proxy / SIEM / etc and are know to be quite talkative, What I want to know is if a new IP appear in the Top"

 

Here is how to do this:

1) Join the search for the last 1,2, ... hour to the last 7,30 days:

 

index=ib_dns sourcetype="ib:dns:query:top_clients" earliest=-1h latest=now | stats sum(COUNT) as Countlast1h by CLIENT
| join type=outer [search index=ib_dns sourcetype="ib:dns:query:top_clients" earliest=-7d latest=-1h |stats sum(COUNT) as CountEarlier by CLIENT]

all top.png

 

2) do the same and display only the client that were not seen before:

index=ib_dns sourcetype="ib:dns:query:top_clients" earliest=-5h latest=now | stats sum(COUNT) as Countlast5h by CLIENT
| join type=outer [search index=ib_dns sourcetype="ib:dns:query:top_clients" earliest=-7d latest=-5h |stats sum(COUNT) as CountEarlier by CLIENT] | where isnull(CountEarlier)

new top.png

 

And now we can include this a top Client dashboard or build an alert on this result

Check out our new Tech docs website at http://docs.infobox.com for latest documentation on Infoblox products.

Re: DNS Top Client - New Top IP not seen in the last week

Expert
Posts: 181
3955     2

I really like the idea of this report, but I struggled with with the variability of the clients and servers within an enterprise. I was getting a huge amount of noise in this report mainly because of this:

In an office of a few hundred PC users pointed to an Infoblox member for DNS and when DHCP leases are relatively short, clients are get new IP's fairly regularly. And, when all clients are "well behaved" from a DNS perspective, a members top DNS talkers is likely to always be in flux, with new IP's showing up and disappearing all the time. Even going back a week to validate will likely find some PC's that got a new IP that has not been used in a week, but now shows up for the first time for a few hours on the top talkers list.


My solution was to create a dashboard based on this report where we could set up a query per second threshold. I really don't care about a new client that is only doing a few QPS. I also created timeframe filters for both look back and look over time widows. Finally,another filter to select the members to run the report over. This is now a very powerful, near real time trouble shooting tool. I'm also working to tune a possible aleart based on this as well.

Thanks for your work getting this started.

Re: DNS Top Client - New Top IP not seen in the last week

Adviser
Posts: 97
3955     2



My solution was to create a dashboard based on this report where we could set up a query per second threshold. I really don't care about a new client that is only doing a few QPS. I also created timeframe filters for both look back and look over time widows. Finally,another filter to select the members to run the report over. This is now a very powerful, near real time trouble shooting tool. I'm also working to tune a possible aleart based on this as well.

 

Sounds really powerful! Care to share the report code?

Re: DNS Top Client - New Top IP not seen in the last week

Expert
Posts: 181
3955     2

Here is my dashboard code, the code could use some work and some input validation but the basic functionality is there.

<form>
  <label>Top-New-DNS-Clients-V3</label>
  <fieldset submitButton="true">
    <input type="time" token="OverLast">
      <label>Look for top DNS clients in this Time Range</label>
      <default>
        <earliest>-1h</earliest>
        <latest>now</latest>
      </default>
    </input>
    <input type="time" token="ValidateTime">
      <label>That were not top clients in this time range</label>
      <default>
        <earliest>-1d</earliest>
        <latest>now</latest>
      </default>
    </input>
    <input type="text" token="QueryCutOff">
      <label>Only if their queryies per second were over:</label>
      <default>1</default>
    </input>
    <input type="multiselect" token="Members">
      <search>
        <query>index=ib_dns_summary report=si_dns_member_qps_trend_per_day | stats count by orig_host</query>
      </search>
      <fieldForLabel>orig_host</fieldForLabel>
      <fieldForValue>orig_host</fieldForValue>
      <default>addnsfwd01.corp.cat.com</default>
      <prefix>(</prefix>
      <suffix>)</suffix>
      <valuePrefix>host="</valuePrefix>
      <valueSuffix>"</valueSuffix>
      <delimiter>OR </delimiter>
    </input>
  </fieldset>
  <row>
    <panel>
      <table>
        <title>New_DNS-Top-Clients</title>
        <search>
          <query>index=ib_dns sourcetype="ib:dns:query:top_clients" $Members$ earliest=$OverLast.earliest$ latest=$OverLast.latest$ | eval QPS = exact(COUNT/600) |
              where QPS &gt; $QueryCutOff$ | stats avg(QPS) as Rate by CLIENT |sort - Rate |
            join type=outer [search index=ib_dns sourcetype="ib:dns:query:top_clients" $Members$ earliest=$ValidateTime.earliest$  latest=$OverLast.earliest$ |
            stats sum(COUNT) as CountEarlier by CLIENT] |  where isnull(CountEarlier)</query>
        </search>
        <option name="wrap">undefined</option>
        <option name="rowNumbers">undefined</option>
        <option name="drilldown">row</option>
      </table>
    </panel>
  </row>
</form>

Re: DNS Top Client - New Top IP not seen in the last week

[ Edited ]
Adviser
Posts: 81
3955     2

Big kudos to Marc Br. for his inputs:

 

This version allows to compare the QPS during last hour vs average QPS for the past week and:

 

1) show the client IPs that have more than double

index=ib_dns sourcetype="ib:dns:query:top_clients" earliest=-1h latest=now | stats sum(eval(COUNT/3600)) as Countlast1h by CLIENT| join type=outer [search index=ib_dns sourcetype="ib:dns:query:top_clients" earliest=-7d latest=-1h |stats sum(eval(COUNT/7/24/3600)) as CountEarlier by CLIENT]| where Countlast1h > 2*CountEarlier

 

2) show the client IPs in the top 100 that have more than double

index=ib_dns sourcetype="ib:dns:query:top_clients" earliest=-1h latest=now | stats sum(eval(COUNT/3600)) as Countlast1h by CLIENT| head 100 | join type=outer [search index=ib_dns sourcetype="ib:dns:query:top_clients" earliest=-7d latest=-1h |stats sum(eval(COUNT/7/24/3600)) as CountEarlier by CLIENT | head 100]| where Countlast1h > 2*CountEarlier

 

Capture d’écran 2016-10-26 à 21.35.00.png

 

Check out our new Tech docs website at http://docs.infobox.com for latest documentation on Infoblox products.
Showing results for 
Search instead for 
Do you mean 

Recommended for You