Splunking The Modern Honey Network: Adding Context Using Threat Feeds (Part 2)

<p>Last week, <a href="https://www.anomali.com/blog/splunking-the-modern-honey-network-getting-value-from-your-honeypots-data-p">I showed how to ingest Modern Honey Network data into Splunk and visualise it with the MHN Splunk App</a>.</p><p>Hopefully you’ve been getting lots of additional value on-top of the Modern Honey Network web app. I’m sure you’re now ready to ramp things up.</p><p>As with all security events — including honeypot events— context is key. For example, an IP trying once to SSH into your machine is probably hoping to exploit default login credentials (<em>hopefully not you!</em>). Whereas an IP seen trying to SSH 10 times every second might be a more serious, targeted attack.</p><p>In this post I’m going to show you how to add context the MHN data you’re Splunking using <a href="https://www.anomali.com/products/threatstream">Anomali Threatstream</a> data via the <a href="https://splunkbase.splunk.com/app/1723/">Anomali Threatstream Community Splunk App</a> (<strong>for free</strong>). Doing so will help you easily identify and quickly investigate the most critical events seen by your honeypot network.</p><h2>A brief intro…</h2><p>The Anomali ThreatStream Community App for Splunk leverages threat intelligence from Anomali’s ThreatStream platform to identify potential threats and breaches. The Community App does this by downloading and matching ThreatStream’s intelligence (known indicators of compromise (IOC)) against your event data in Splunk.</p><h2>Configuring the Anomali ThreatStream Community App with MHN data</h2><h3>0. Prerequisites</h3><p>You have a Splunk instance with Modern Honey Network data being ingested. <a href="https://www.anomali.com/blog/splunking-the-modern-honey-network-getting-value-from-your-honeypots-data-p">Here’s a walkthrough detailing how to do this</a>.</p><h3>1. Install the Anomali ThreatStream Community Splunk App</h3><p><a href="https://splunkbase.splunk.com/app/1723/">Download the Anomali ThreatStream Community Splunk App here</a>.</p><p>Navigate to: <em>Apps > Manage Apps > Install App from File</em>. Follow the instructions to upload the app you’ve just downloaded.</p><h3>2. Sign up for a free Anomali Threatstream Account</h3><p><img alt="" src="https://cdn.filestackcontent.com/nGPRimqRqOqFsalzrOzI" style="width: 600px; height: 317px;"/></p><p>In order to download data from Threatstream into Splunk to perform matching you will need a free Anomali Threatstream account. If you don’t already have an account, you can sign up for one whilst configuring the app: <em>Apps > Anomali Threatstream Community > Help > Run setup</em>.</p><h3>3. Get familiar with the App lookups</h3><p><img alt="" src="https://cdn.filestackcontent.com/n26Kc5VTcCQFj9jYOPIz" style="width: 600px; height: 284px;"/></p><p>You will notice the app ships with a number of lookups for different functions within the app when navigating to: <em>Settings > Lookups > Anomali Threatstream Community App</em>.</p><p>The lookups are separated by IOC type and include: URL, MD5 hash, IP, email, and domain. Each lookup file has differing column headers. For example the the tsi_ip lookup has the following headers:</p><pre> _key, asn, classification, confidence, country, date_first, date_last, detail, id, itype, lat, lon, lookup_key_value, maltype, org, resource_uri, severity, source, actor, campaign, tipreport, _time, last_time, link, type</pre><p>The app lookups ship with a small sample of historic IOC data from Threatstream. You can view this in Splunk by running the “inputlookup” command. For example, to view the “tsi_ip” lookup run:</p><pre> | inputlookup tsi_ip</pre><p>Note: It may take up to 24 hours for the download of new lookup data from Threatstream for matching.</p><h3>3. Start matching lookups against MHN events</h3><p>The default MHN Server Splunk key / value log file only contains IP and MD5 fields, so I will only use these two lookup files to perform matches against.</p><p>Start by matching MHN data against the “tsi_ip” lookup:</p><pre> source=”*mhn-splunk.log*” | lookup tsi_ip lookup_key_value AS src OUTPUTNEW confidence AS ts_confidence, itype AS ts_itype, detail AS ts_detail | search ts_confidence=”*”</pre><p>Lets break down this search:</p><pre> source=”*mhn-splunk.log*”</pre><p>First returns all data Splunk holds from the MHN log file.</p><pre> | lookup tsi_ip lookup_key_value AS src OUTPUTNEW confidence AS ts_confidence, itype AS ts_itype, detail AS ts_detail |</pre><p>The “lookup” command is then used to call the “tsi_ip” lookup, telling Splunk that the “lookup_key_value” column in the lookup should be matched against the “src” field values in our MHN log. The “lookup_key_value” contains the malicious IP values from Threatstream, whilst the “src” field in our MHN data is the origin IP of a MHN event. If the two match there is an event that likely warrants investigation.</p><p>The “OUTPUTNEW” function then adds additional context to the MHN event by allowing us to enrich it with lookup data. You can see a list of the fields available in each lookup to enrich a MHN event by running a Splunk search: “| inputlookup LOOKUP_NAME”. In the example above; confidence, itype and detail fields are used from the lookup. I rename each field with a “ts_” prefix so that I can easily identify fields from the lookup in search. The fields added from the lookup can then be used in search, as per the final |.</p><pre> | search ts_confidence=”*”</pre><p>Once the matching has completed another search is run, this time to filter only events that have a “ts_confidence” field. Events will only have a “ts_confidence” field if they successfully matched against Threatstream IOCs.</p><h3>4. Search on</h3><p>You now know how to enrich MHN event data with Threatstream IOCs. However, depending on the matches returned, you might want to add more context. Time, for example, as discussed at the start of this post.</p><pre> source=”*mhn-splunk.log*” | lookup tsi_ip lookup_key_value AS src OUTPUTNEW confidence AS ts_confidence, itype AS ts_itype, detail AS ts_detail | search ts_confidence=”*” | transaction src maxpause=5s</pre><p>The search above adds a “transaction” command to the end of the search created before.</p><pre> | transaction src maxpause=5s</pre><p>After events that match against Threatstream IOCs are returned (| search ts_confidence=”*”), the transaction command further filters results by requiring events grouped by IP (src) that have been seen within 5 seconds of each other (maxpause=5s). For example, the search would return events from an IP that visited a honeypot at 12:00:01 and 12:00:02 (1 sec pause), but not an IP that visited a honeypot at 12:00:01 and 12:00:11 (10 sec pause).</p><h3>4. Visualise</h3><p><img alt="" src="https://cdn.filestackcontent.com/I7gg28sRT2Ny5seGbww3" style="width: 600px; height: 284px;"/></p><p>Data alone is good. It’s often much easier to understand and share in more visual form.</p><pre> source=”*mhn-splunk.log*” | lookup tsi_ip lookup_key_value AS src OUTPUTNEW confidence AS ts_confidence, itype AS ts_itype, detail AS ts_detail | search ts_confidence=”*” | timechart count</pre><p>Using the “timechart” command, a graph is generated showing a daily count of MHN events that match Threatstream IOCs. If there is a sudden spike, it may be indicative of a targeted attack, for example. <a href="http://docs.splunk.com/Documentation/Splunk/6.5.1/Viz/Visualizationreference">Splunk has many visualisation options including maps, line graphs, and tables</a>.</p><h2>Further reading</h2><p>If you’re new to Splunk, a good place to start would be to <a href="https://docs.splunk.com/Documentation/Splunk/6.5.1/SearchTutorial/WelcometotheSearchTutorial">explore Splunk’s search language</a> — specifically the <a href="https://docs.splunk.com/Documentation/Splunk/6.5.1/SearchReference/Lookup">lookup</a> and <a href="https://docs.splunk.com/Documentation/Splunk/6.5.1/SearchReference/Transaction">transaction</a> commands introduced in this post.</p><p>The Anomali Threatstream Community App is a powerful tool to identify threats to your network. I’ve skimmed over its additional functionality in this post. <a href="https://splunkbase.splunk.com/app/1723/#/details">Read more about what the app does and how to configure the additional functionality</a>.</p><h2>Exploring The Modern Honey Network</h2><p>Over the next few weeks I will be posting a series of guides about how to get value out of the data being generated by your honeypots. <a href="https://twitter.com/anomali">You can get updated about new MHN posts by Anomali on Twitter</a>.</p><p><strong>Next up:</strong> <a href="https://www.anomali.com/blog/splunking-the-modern-honey-network-honeypot-alert-automation-part-3">Automated Honeypot Alerts</a></p>