Blog

Building a Splunk Map

August 16, 2016 | Susan Bilder

Visualizing incoming web traffic on a geographic map provides valuable insights for security monitoring, customer activity, and website traffic. Splunk® provides the ability to turn log data that contains IP addresses (e.g. firewall logs, web server logs) into a real-time activity map.

To create a basic Splunk map, you simply specify which type of data you want to examine and how you want to drill down into that data. For example, if you have an Apache web server and want to display a map of web client requests counted by the city the request came from, you would use the following query:

sourcetype=access_combined | iplocation clientip | geostats count by City

The query works as follows:

1. Data is restricted to only access_combined web log data, which tells Splunk to select all Apache format web logs.

2. The name for the field containing the incoming IP address in Splunk’s access_combined CIM model is “clientip”. The iplocation command reads in the clientip for each record, looks up the geographic location for the IP address, and adds the following fields to each record:

  • Country
  • City
  • Region
  • Latitude (lat)
  • Longitude (lon)

3. The geostats command then sorts the data into bins based on latitude and longitude, and plots the data on a map. The “count by City” argument for geostats is then used to populate the pie chart at each location.
basemap

4. Hovering over the pie chart will display a pop up showing the breakdown of traffic by City.basemap_hover

There are a couple of refinements that can further enhance the value of the map:

1. Eliminate internal traffic
You can use a wildcard or CIDR notation to specify ranges of IP addresses – for example, to eliminate traffic from internal 192.168.0.0/16 addresses, you could use either of the following:

sourcetype=access_combined clientip != 192.168.0.0/16

or

sourcetype=access_combined clientip != 192.168.*

 

2. Display an alternate value if City is blank
iplocation is not always able to assign names to the City, Region, or Country fields when it looks up an IP address. On the map, all points where the City is blank are grouped together under the name “VALUE”.
To provide more accurate data, Splunk can use the eval command and the if function to copy the values from the Region or Country fields to the City field. For example:

eval City= if( City = "", Region, City)

will assign the value for Region to City if City is blank, and keep the current value if one exists. For this map we want to use Region if the value for City is blank, and use Country if both Region and City are blank. The full command to assign defaults for Country, Region and City is:

eval Country = if( Country="", "N/A", Country),
Region = if (Region="", Country, Region),
City = if (City="", Region, City)

For our map this can be shortened to use nested loops:

eval City = if (City = "", if(Region = "", if(Country = "", "N/A", Country), Region), City)

 

3. Display data for more cities in pie charts.

In our default command, geostats is limited to keeping count of 10 cities, and all other cities will be grouped under the name “OTHER”. The “globallimit” argument can be used to change the number of cities geostats counts, with a value a 0 indicating that all cities should be displayed:

geostats count by City globallimit=0

Chaining these together yields:

sourcetype=access_combined clientip != 192.168.0.0/16 |
iplocation clientip |
eval City = if (City = "", if(Region = "", if(Country = "", "N/A", Country), Region), City) |
geostats count by City globallimit=0

 

all_cities_resolved
This map is just one example of the many visualizations available with Splunk – if you have any questions about how to customize Splunk dashboards for your needs or would like to get started with a Splunk  please contact Splunk Sales

Want to learn more?

Download a FREE trial of Splunk  - Download a copy today and see how Splunk makes it simple to collect, analyze, and act upon the untapped value of big data.

Start Your Free 60 Day Trial of  Splunk Today!

 

Sign Up for the Blog

Heroix will never sell or redistribute your email address.