How to determine if a STIX indicator is unique with a large dataset?

[julian kÃster Larsen <jullefis@gmail.com>]

Hi, i would appreciate advice on how to store millions (10~) of STIX indicators in a way thatÂprevents duplicates.

As patterns can be written in various ways, i have yet to come up with a solution myself.

My current idea was to make use of theÂfind_equivalent_patterns()Âmethod from the python stix2 library:

https://stix2.readthedocs.io/en/latest/api/equivalence/stix2.equivalence.pattern.html#module-stix2.equivalence.pattern

With this solution however i would potentially have to iterate over lot of STIX indicators to determine if the STIX indicator, A, is unique.

My current bad solution queries the DB and collects STIX indicators that contain the same Object paths and/or constants as A, and then makes use ofÂfind_equivalent_patterns()Âto correctly verify if any of these patterns are equal.

To avoid confusion here is an example of a STIX indicator pattern:

https://imgur.com/MsG39x8

Regards Julian