Hi Jeff;
One comment - This list is generally for comments on the STIX specification itself. Below seems like a topic for implementation, more suited for the cti-users list? They have different subscribers, so it's significant.
On your question - without getting into the weeds of how your specific SIEM or detection system may or may not work, being an SME in this area I have some thoughts on your problem. There are two sides to the problem typically, both on the threat feed side as well as the detection side.
- On the threat feed side, very, very few threat feeds give a reliable TTL, or any, TTL for their threat indicators, even though we do have valid_from and valid_until in STIX 2. If you have examples of threat feeds that reliably populate this information, that would be good to have actually as a counter-example, as it hasn't been my experience. Obviously, if the threat feed is not providing this information, the SIEM (or any down stream consumer) is not going to be able to act on it.
- Assuming that TTL existed, it then opens up the question of how you would get notified if it ever renewed. To do that, you have to periodically re-poll the collection for updates and you would have to rely on the server fully implementing versioning as expected. Again, from my experience this is a rarity, many TI providers are still re-publishing data on change with new UUIDs and not actually doing STIX 2 versioning properly.
- Down at the SIEM/detection side, the issue becomes one of "where is this data going to be actioned". Normally, the indicators are being read out and used to update white/black lists of some sort in the detection system, then are being used inside rules, searches, or reports. Some vendors have TTLs available in these lists, some do not. Some have a TTL for the entire list, some have it per-element. Sometimes you can set it directly, other times it is a rolling window TTL based on the time of insert. Due to all of these dynamics - there are limitations with how the detection system can realistically support a valid_until being provided by a feed, even if it exists.
As a general work-around to all of these issues - what we often see people do is simply periodically *download the entire collection*, and outright *replace* the white/black list with it. This ensures you always have the up to date information, and don't have to deal with any of the above problems. However the down-side to this approach is you have a lot of data to deal with on every update. Another in-the-middle approach is to use a filter to fetch the last X days of data from the feed, and then insert it into the detection system with an X days TTL.
I hope the above helps somewhat... at the end of the day, the whole reason we are here is to try to make it so the above problems go away across industry as vendors align on standards. It takes time to get there however...
-
Jason Keirstead
Chief Architect - IBM Security Threat Management
www.ibm.com/security
"Would you like me to give you a formula for success? It's quite simple, really. Double your rate of failure."
- Thomas J. Watson
----- Original message -----
From: Jeff LoSpinoso <jlospinoso@childrensplace.com>
Sent by: <cti-comment@lists.oasis-open.org>
To: JG <jg@ctin.us>, "cti-comment@lists.oasis-open.org" <cti-comment@lists.oasis-open.org>
Cc:
Subject: [EXTERNAL] RE: [cti-comment] STIX/TAXII Data Life Cycling
Date: Wed, Feb 26, 2020 12:54 PM
Hello There,
Just wondering if you had any insights to share.
Regards,
Jeff
From: Jeff LoSpinoso
Sent: Friday, February 21, 2020 2:44 PM
To: JG <jg@ctin.us>; cti-comment@lists.oasis-open.org
Subject: RE: [cti-comment] STIX/TAXII Data Life Cycling
Hello Jane,
So hereâs what Iâve got. Our SIEM platform has a built-in TAXII client. I set up a feed from a TAXII server where the SIEM platform basically just creates a list of IPv4 addresses. Our SIEM vendor has told us that we must apply a time to live so the list does not grow to larger over time. They state that the only way an indicator can be removed from the list is by way of a TTL enforced on the list where this TTL is pulled from thin air having nothing to do with any dates provided by the TXF. My response to them is that the only way something should be removed from the downstream copy of the Threat Feed is if the Threat Feed says so.
A simple analogy. If an Airline maintains a copy of the DHS no-Fly list for their own use, they should not remove a bad guys name off the list just because itâs been there for more than X number of days. It should only be removed when DHS says it should be removed.
So what are the mechanisms for a TAXII Client to stay in sync with its authoritative feed? Similar to how Active Directory replicates objects (including deletions) I would imagine.
Iâve searched high and low and canât find any clues as to how this works. To my sad surprise our SIEM vendor, a household name in IT, clearly hasnât a clue as demonstrated by their urging to just arbitrarily drop indicators based on a random TTL.
Any insights would be greatly appreciated. Enjoy you weekend wherever you are.
Regards,
Jeff
CAUTION: This email originated outside of our organization. Before clicking any links or attachments, please confirm that you recognize the sender and know that the content is safe.
Jeff:
Thanks for reaching out to the community.
Can you tell us a little more about your specific Use Case? For specific indicators or STIX Domain Objects (SDOs) we have a revoke property that can be invoked by the original producer of that SDO. We also have a Modified property if a producer finds out additional information and wants to update.
From the consumer side, it would help to know a little more. Is your Use Case related to duplicates? Or is it related to an indicator or set of indicators that you no longer need for your analysis? Please describe the problem you are trying to solve.
Jane Ginn
On 2/20/2020 9:55 AM, Jeff LoSpinoso wrote:
Hello,
How are indicators removed from a collection, more specifically how does a consumer of a TAXII Feed know when to remove an indicator from its synchronized copy of the collection?
Your insights would be most appreciated.
Regards,
Jeff LoSpinoso, CISSP
TCP Cybersecurity Team
919.244.9359
--
*****************************
Jane Ginn, MSIA, MRP
Secretary, OASIS CTI TC
Sponsor, TAC TC
Sponsor, BP TC
001 (928) 399-0509
*****************************