Addressing Threat Blindness

<p>In just four years since launching Anomali we’ve seen Threat Intelligence become a standard element of enterprise security programs. Last week we published a Ponemon Institute report on “<a href="https://www.anomali.com/resources/whitepapers/the-value-of-threat-intelligence-2017-ponemon-report">The Value of Threat Intelligence</a>” (our 2nd year sponsoring this research) – in it we found:</p><div class="feature-box fbox-plain bottommargin-sm"><div class="fbox-icon"><i>80<span style="font-size: 28px;">%</span></i></div><h3 style="padding:0px 10px;">of enterprises now leverage threat intelligence in their security programs</h3></div><div class="feature-box fbox-plain bottommargin"><div class="fbox-icon"><i>84<span style="font-size: 28px;">%</span></i></div><h3 style="padding:0px 10px;">indicated threat intelligence is “essential to a strong security posture”</h3></div><p>Despite this rapid adoption we still see organizations struggling to take full advantage of threat intelligence. Fully 68% of Ponemon respondents said threat intelligence is too voluminous and complex. This speaks to the real promise of threat intelligence – what matters isn't the list of threats itself, but which of those threats are active in my environment. This requires finding the cross section between my threat feeds and my network activity.</p><h3>New Versus Old Threats</h3><p>Most organizations subscribe to numerous threat feeds, whether from open source, premium/3rd party, ISACs, government sharing, etc. Security teams will typically collect and accumulate millions of IOCs (indicators of compromise) from their various threat sources. Every day new threats are added to the list. As it turns out, we need to handle newly discovered threats differently than previously known threats. Here’s why:</p><p><strong>Previously known threats:</strong> All previously <strong>known</strong> threats need to be monitored daily to make sure we haven't become a target. It's like routine health checks - we need to verify that nothing bad happened today from any of these threats.</p><p><strong>Newly discovered threats: </strong>Newly discovered threats discovered threats are a different beast altogether. These threats became <em>known</em> today, but they didn't become <em>bad</em> today. They may have been active for weeks, months or years. Attackers do their best to stay under the radar. When new threats are discovered it’s not enough to be on the lookout for them going forward. Perhaps more importantly, we need to go back in time to see we’ve already been targeted by these actors.</p><h3>Organizations Flying Blind</h3><p>The challenge for security teams is how to realistically monitor for known threats and assess exposure to new threats on a daily basis.</p><p>Consider the previously known threats. Sounds easy enough to simply alert against any matches against my threat list. Here’s the issue – if you’re a moderate sized enterprise you’re likely generating 1 billion or more log events per day. If you’ve got a (relatively small) threat list of, say, 1 million indicators then you need to compare 1 billion events against 1 million threats. That’s 1,000 TRILLION comparisons per day!</p><p style="text-align: center;"><img alt="" src="https://cdn.filestackcontent.com/viajATVyT6CkMaDo9RYa"/></p><p>Now consider newly discovered threats. Here you might get 10, 100, 1000 new threats on a given day. The challenge here isn’t the daily monitoring – it’s going back to assess prior exposure. Given how long attacks often take (200 days or more), it’s important to be able to back at least 1 year to get a clear picture of possible prior exposure. Let’s do the math: 1 billion events/day, 365 days, 100 new threats/day = 36.5 trillion comparisons.</p><p>The Ponemon survey asked respondents how much historical data they maintain online (e.g., in a SIEM). 72% keep 3 months or less. Plus, running a query over that amount of data would take hours or even days. The end result is organizations are pretty much flying blind with respect to the vast majority of these known threats. Our solution for this is Real Time Forensics.</p><h3>Real Time Forensics</h3><p>Anomali innovated the concept of Real Time Forensics (RTF) to address this fundamental threat visibility issue. RTF is the core technology that powers <a href="https://www.anomali.com/products/match">Anomali Match</a>. RTF is an extraordinarily powerful engine that can perform searches over massive amounts of data instantly. In just a few seconds RTF can literally:</p><ul class="iconlist"><li><i class="icon-ok text-blue">​</i> Identify all matches for millions of IOCs across billions of events</li><li><i class="icon-ok text-blue">​</i> Search years of historical data and return all matches</li></ul><p>RTF does this WITHOUT duplicating log storage. It integrates with existing log repositories/sources such as SIEMs, syslog, Netflow/sFlow and AWS S3.</p><p>We developed RTF with three key objectives in mind:</p><table style="width: 100%;"><tbody><tr><td style="width:90px;padding:10px;"><img alt="Visibility" src="https://cdn.filestackcontent.com/MP8YdyYlSLSsd1MjErSM" style="width: 90px;"/></td><td style="padding:10px;"><strong>Visibility:</strong> providing complete visibility into all threats, all network activity, for all time</td></tr><tr><td style="width:90px;padding:10px;"><img alt="Clarity" src="https://cdn.filestackcontent.com/DMWVGzm9SQaXGDpYDnR7" style="width: 90px;"/></td><td style="padding:10px;"><strong>Clarity:</strong> integrating threat context from ThreatStream to provide a complete picture of the threat and how to respond</td></tr><tr><td style="width:90px;padding:10px;"><img alt="Automation" src="https://cdn.filestackcontent.com/Dfa0SAlBREuIByOIOnpc" style="width: 90px;"/></td><td style="padding:10px;"><strong>Automation:</strong> automatically evaluating new and existing threats; alerting security teams to real, active threats in their networks</td></tr></tbody></table><p>These objectives aren’t new, but RTF’s capabilities give them a whole new meaning. The second an organization gets a hold of indicators from the latest network breach they can identify whether or not they were affected. Unmanageable data sets are no longer an obstacle to full threat visibility.</p><p>As it turns out, the future of security isn’t the ability to look forward, but the ability to look back.</p>