cti-stix message
[Date Prev]
| [Thread Prev]
| [Thread Next]
| [Date Next]
--
[Date Index]
| [Thread Index]
| [List Home]
Subject: Re: [cti-stix] Re: [EXT] [cti-stix] ability to use UUID5 in STIX2 identifier
- From: "Jason Keirstead" <Jason.Keirstead@ca.ibm.com>
- To: "Wunder, John A." <jwunder@mitre.org>
- Date: Mon, 4 Feb 2019 14:08:18 -0400
I would also support this.
I have learned more about the inner
workings of UUID4/5 and I don't have any reservations about it anymore.
The odds of collision with a properly-implemented UUID5 are on-par with
UUID4
As far as John's comment below - all
this means IMHO is the library has to force you to provide a namespace
(ie make it a mandatory argument in your constructor or whatever).
-
Jason Keirstead
Lead Architect - IBM Security Connect
www.ibm.com/security
"Things may come to those who wait, but only the things left by those
who hustle." - Unknown
From:
"Wunder, John
A." <jwunder@mitre.org>
To:
Sergey Polzunov <sergey@eclecticiq.com>,
"cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org>
Date:
02/04/2019 12:22 PM
Subject:
[cti-stix] Re:
[EXT] [cti-stix] ability to use UUID5 in STIX2 identifier
Sent by:
<cti-stix@lists.oasis-open.org>
I've been thinking a lot about this and I think it
makes sense.
One of the concerns we had at the time we chose UUID4 is that users of
libraries like python-stix would need to remember to set the UUID5 namespace
-- or, if they don't and python-stix has some default namespace, different
tools using the libraries could have overlapping IDs. This would also apply
to users of the new Java libraries that I've seen come out. It might mean
these libraries requiring that people set a unique namespace before creating
any objects, vs. now where it can just go ahead and create IDs by default.
I'd be curious what other people think about this problem and how we can
help avoid it becoming an issue (especially given how many people use those
libraries).
John
ïOn 2/4/19, 11:12 AM, "cti-stix@lists.oasis-open.org on behalf of
Sergey Polzunov" <cti-stix@lists.oasis-open.org on behalf of sergey@eclecticiq.com>
wrote:
Hey everybody!
Current STIX2 spec definition of an`identifier` for STIX2
objects is as follows:
> An identifier universally and uniquely identifies a
SDO, SRO, Bundle, or Marking Definition. Identifiers MUST follow the form
object-type--UUIDv4, where object-type is the exact value (all type names
are lowercase strings, by definition) from the type property of the object
being identified or referenced and where the UUIDv4 is an RFC 4122-compliant
Version 4 UUID. The UUID MUST be generated according to the algorithm(s)
defined in RFC 4122, section 4.4 (Version 4 UUID) [RFC4122].
â from http://docs.oasis-open.org/cti/stix/v2.0/cs01/part1-stix-core/stix-v2.0-cs01-part1-stix-core.html#_Toc496709265
I think the requirement to have UUID4 brings more problems
than benefits. It makes STIX1->STIX2 transition difficult, hurting existing
STIX1 users.
I will try to show it in these 2 use cases.
Use case 1
----------
Imagine that I'm a client of an intelligence
provider A. I've been a client for a long time and I have received intelligence
in STIX1.2, which I stored in my DB. I fetch new intelligence daily, downloading
only fresh data. Often fresh data links to old objects for context.
Provider A decides to upgrade and switch to
STIX2. In addition to an old STIX1.2 feed, provider creates new STIX2 feed
with the same data. In STIX2 all objects have new identifiers and Provider
A does not bother to supply a mapping of STIX1.2 ids to STIX2 ids. Now
I, as a client, have 2 options:
- clean slate option: drop all old data from
this provider and re-fetch everything. That will work if Provider A is
the only provider I use or if I never referenced Provider A's data from
my own intelligence. Not a great plan.
- new era option: leave my STIX1.2 data graph
in place and start consuming new STIX2 feed from today. This option has
one big issue: new STIX2 data will not be connected to STIX1.2 data I already
have, because STIX2 ids are all different. If I want to deduce connection,
I need to deduplicate the data against my existing STIX1.2 DB. This means
my ingestion pipeline must be smart enough to compare STIX1.2 objects to
STIX2 objects and be fast enough to do that for every new STIX2 object.
This will be difficult to implement and will have a huge performance penalty.
Use case 2
----------
Imagine that I'm a NCSC. I receive intelligence
from providers, combine it and distribute it to my clients. My providers
are still on STIX1.2 but my clients want STIX2, so I must convert STIX1.2
I receive into STIX2. Full STIX1.2 entities I can transform easily but
what do I do with IDREFs I have in my STIX1.2 data?
I can generate new STIX2 id every time I see
new STIX1.2 IDREF in incoming data and store STIX1.2->STIX2 mapping
somewhere to be used next time I see this IDREF. This is painful and will
require additional resources, but it is doable. But it will only work until
the moment my providers switch to STIX2 and start sending me full objects
for those IDREFs with new random STIX2 identifiers! I can not predict these
identifiers and I can't match them with the ones I generated. So my thinking
is - what is the point in even bothering with old IDREFs? I will just drop
them, sending my clients sometimes disconnected STIX2 entities, hoping
that they will figure it out.
Proposed solutions if UUID5 is allowed in STIX2 identifiers:
Use case 1 solution
-------------------
There can be a guideline that will recommend
providers to use old STIX1.2 IDs as input for new STIX2 identifiers. If
STIX2 identifiers are predictable I, as a client, can greatly simplify
my deduplication logic. I can run DB migration once to calculate STIX2
identifiers for all my STIX1.2 objects and use these on ingestion for deduplication.
Appending STIX2 data to my STIX1.2 DB will be much easier.
I'm also interested in pushing Provider A to
adopting this STIX2 identifier generation practice because it will save
me money.
Use case 2 solution
-------------------
WIth UUID5 I have a way out: I can generate new STIX2 ids
from old STIX1.2 ids! I can parse IDREF value, that looks like `[ns prefix]:[construct
type]-[GUID]`, and use provider's namespace / construct type to build new
STIX2 identifier. The logic will be like this:
- full IDREF will be input for UUID5 function
- for STIX1.2 types that were split (like TTP),
I do not know exact STIX2 type Provider would use for old TTP. My solution
here would be to play safe and create relations for all possible types:
for IDREF to TTP, I will create 4 relations: one to a possible Tool object,
one to Malware, one to Attack Pattern and one to Identity. It is an overhead
but it is a small price for keeping interconnected intelligence graph.
Again, when time comes and my providers move
to STIX2, I'm interested in pushing them to adopt this id generation schema
for old objects, because it will save me, as NCSC, money.
To reiterate, I would like to propose:
- a change in STIX2 spec to allow both UUID5 and UUID4 to
be used in an identifier of SDO, SRO, MarkingDefinition and Custom Object
entities;
- creating a guideline, complimentary to the spec, that would
explain how STIX1.2 ids can be transformed into STIX2 for easier transition.
Practicalities:
UUID5 ids require use of a namespace. UUID5 RFC (https://tools.ietf.org/html/rfc4122#section-4.3)
defines some generic namespaces (https://tools.ietf.org/html/rfc4122#appendix-C)
but does not prohibits the use of custom ones. I suggest this algorithm:
- namespace UUID5 is generated by using predefined
`NameSpace_URL` namespace and producer's URL;
- for old objects, GUID part of STIX2 identifier
is namespaced UUID5 generated from old STIX1.2 id
- for new objects, GUID part of STIX2 identifier
is either namespaced UUID5 with random UUID4 string, or just random UUID4.
Example python code for generating UUID5 with custom namespace:
In [1]: import uuid
...:
...: stix12_id = 'eclecticiq:threat-actor-07fa8672-4bca-46e1-a60f-023882b4a473'
...: namespace_uuid = uuid.uuid5(uuid.NAMESPACE_URL,
'https://urldefense.proofpoint.com/v2/url?u=https-3A__eclecticiq.com_ns&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=k6Q07xZDujljzkKqZUfupXFUDIHGIiq-Sl_u1bw0hyA&m=cvP-VddGmd9zTZUjb6OSCUczFxCjDL1cA586YiCE8YI&s=tQbiU4LJBfzo5lmgDPo4k6EjM9ZKKwE6AzhNphzBRcM&e=')
...: stix2_uuid = uuid.uuid5(namespace_uuid,
stix12_id)
...: stix2_id = 'threat-actor--{}'.format(stix2_uuid)
...:
...: print("new STIX2 id: {}".format(stix2_id))
new STIX2 id: threat-actor--adee573a-12e9-5dd3-958b-0040d32c6b3e
BONUS: python functions to convert STIX1.2 IDREFs into STIX2
identifiers - https://gist.github.com/traut/fd4b9b8de3c2aa0e161d68c4099656e5
Thank you,
Sergey Polzunov
EclecticIQ
[Date Prev]
| [Thread Prev]
| [Thread Next]
| [Date Next]
--
[Date Index]
| [Thread Index]
| [List Home]