Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties

["Jason Keirstead" <Jason.Keirstead@ca.ibm.com>]

What would be the proposed third value used in the tuple that would not break versioning?

-
Jason Keirstead
STSM, Product Architect, Security Intelligence, IBM Security Systems
www.ibm.com/security | www.securityintelligence.com

Without data, all you are is just another person with an opinion - Unknown


Paul Patrick ---05/03/2016 01:37:13 PM---Folks, I don’t mean to cause a stir or re-open topics, but on today’s call during the discussion abo

From: Paul Patrick <ppatrick@isightpartners.com>
To: "Jordan, Bret" <bret.jordan@bluecoat.com>, Patrick Maroney <Pmaroney@Specere.org>
Cc: "Wunder, John A." <jwunder@mitre.org>, "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org>
Date: 05/03/2016 01:37 PM
Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties
Sent by: <cti-stix@lists.oasis-open.org>

---

Folks,

I don’t mean to cause a stir or re-open topics, but on today’s call during the discussion about identifiers there appeared to be some confusion about how Patrick had proposed having deterministic identifiers.  If I heard the comments correctly, there seemed to be the opinion that you had to hash the entire object, which would include the revision information, in order to generate the identifier.  But as I read Pat’s proposal, that isn’t what he proposed if you look back below at his email.

In Pat’s proposal, he proposed using a tuple of organization namespace (aka domain name), type of object, and one or more value(s)  as input to the uuid5 algorithm.  The organization namespace and type of object was used to significantly the possibility of collisions that could occur with the key value(s) (e.g., IP address value) when the tuple contents were hashed using the SHA1 per the uuid5 algorithm.  Because the entire object, including revision information, isn’t used to determine the hash there isn’t the issue of breaking relationships.

From an implementors point of view, I've been doing a very similar approach for a while without any issues. It avoids the need to maintain a list of identifiers that we’ve used before and thus simplifies the effort to support.  In addition, the generated identifiers are valid UUIDs with the same number of bytes, etc.  Per RFC 4122, there is no way, beyond making sure that the timestamp portion of the UUID is in the future, to validate if a UUID is valid much less enforce that the UUID had been generated using the uuid4 algorithm or not.  So attempting to enforce that the uuid portion of identifiers are generated by the uuid4 algorithm is pointless.

So I submit that if someone wants to use the uuid3 or uuid5 algorithm, they should not be prohibited to do so but our text should clearly state that it is RECOMMENDED that the uuid4 algorithm be used.

I’m happy to tweak the text in the document accordingly, but wanted to raise what I believe was confusion on this topic to the group before doing any text changes.


Paul Patrick

From: <cti-stix@lists.oasis-open.org> on behalf of "Jordan, Bret" <bret.jordan@bluecoat.com>
Date: Monday, May 2, 2016 at 6:01 PM
To: Patrick Maroney <Pmaroney@Specere.org>
Cc: "Wunder, John A." <jwunder@mitre.org>, "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org>
Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties
Resent-From: <Paul.Patrick@FireEye.com>

Pat,

Are you planning then to build your deterministic IDs without the use of the versioning information? Because, if you do use versioning information for your IDs, to help with all the things you outline below (which you would need to actually find real duplicates), your deterministic IDs will change with ever revision of the object. And thus all of your relationships will break.

Thanks,

Bret



Bret Jordan CISSP
Director of Security Architecture and Standards | Office of the CTO
Blue Coat Systems
PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."

On Apr 30, 2016, at 13:27, Patrick Maroney <Pmaroney@Specere.org> wrote:

re:."This is the start of the 2 day review process prior to the initial motion, so please provide any comments now so we can adjust. We can discuss comments on the Tuesday working call and, assuming we’re able to resolve them, make the motions to approve these sections on Wednesday morning."

I have added notional changes to Identifier to add previously requested support for Deterministic UUID Identifiers. The current language arbitrarily constrains the RFC 1422 UUID identifier to the UUIDv4 pseudo-random generation method.

Format and language are notional and will need corrections by the editors.

Basis:

If certain Vendor Factions, Communities of Trust,etc. want to use psuedo-random UUIDv4 generation

-- They can!!!

if Intelligence Communities, Licensed Content Providers, Aggregators, etc. want/need to establish provenance, detect leakages and license infringement, Generate and track UUIDs specific to distribution channels, provide anonymous path traceability back through a dissemination chain the the Source Analyst/Organization, , etc.


-- They can To!!!


The method of generation of any UUID can of course be determined from the generated UUID.
-- Win-Win for both camps and with no cost that I can discern





4.5.​ Identifier
Type Name: identifier
Status: Review
MVP: Yes

An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUIDv4], where [object-type] is the exact value from the type field of the object being identified or referenced and [uuid] is an RFC 4122 compliant Version 4 UUID. The uuid field MUST be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID).

4.5.1 Version 4 UUID. The uuid field MAY be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID).

4.5.1.​2 Examples
{
"type": "indicator",
"id": "indicator--e2e1a340-4415-4ba8-9671-f7343fbf0836",
...
}

4.5.2 ​Version 5 UUID. A Deterministic Object ID MAY be generated by the original Source making the assertion or reporting a sighting.

Object ID = uuid5(NameSpace[.subNameSpace] + ObjectClass + ObjectType + ObjectValue)

PublIcNameSpace = NameSpace[.subNameSpace]

ObfuscatedNameSpace = uuid5(NameSpace.Key), where Key = uuid5(subNameSpace)


(4.5.2.1) Examples

NameSpace = 'specere.org'
subNameSpace = 'isac-2'
ObjectClass = 'AddressObject'
ObjectType = 'ipv4-addr'
ObjectValue = '1.2.3.5'

AddressObject.ipv4-addr::622fca15-8e5f-5503-80af-bdbd7a69e9f7

Demonstrate deterministic generation of CTI Object IDs
Generates Namespace Prefix deterministic generation of CTI Object IDs

Type:AddressObjectType.ipv4-addr

Value:1.2.3.4

Namespace: development.specere.org
Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a
ID Suffix: 524d9aa0-2dc5-55b9-b0bc-312314f19a69

Namespace: production.specere.org
Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb
ID Suffix: 377d9ac3-3dd6-594d-a1d6-9250bb268b69

Namespace: operational-testing.specere.org
Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765
ID Suffix: e3c285c9-676d-53e8-beaa-abca2fbd0b10

Namespace: isac-1.specere.org
Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910
ID Suffix: 432d8296-a2e7-5280-9ea9-ca5ca0c9bb26

Namespace: isac-2.specere.org
Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a
ID Suffix: ed8e845d-2eb9-50e2-a5df-aa35df074c30


Type:AddressObjectType.ipv4-addr

Value:1.2.3.5

Namespace: development.specere.org
Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a
ID Suffix: 6f596396-4e4f-501e-8631-58b7ccadca83

Namespace: production.specere.org
Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb
ID Suffix: 4fb2a115-30f9-5c84-8d85-bc40c6a00ff0

Namespace: operational-testing.specere.org
Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765
ID Suffix: bad7a975-e1ba-5908-bd46-e88ec61a4d2f

Namespace: isac-1.specere.org
Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910
ID Suffix: 69c3ebaa-db59-5195-a391-0dd1b5a43c48

Namespace: isac-2.specere.org
Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a
ID Suffix: 622fca15-8e5f-5503-80af-bdbd7a69e9f7


Type:DomainNameType.TLD

Value:badguys.com

Namespace: development.specere.org
Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a
ID Suffix: d73294e6-502f-5bf5-8df8-ec7992cfb6da

Namespace: production.specere.org
Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb
ID Suffix: b6e7b0f4-a655-5330-a2a2-be0066a221bc

Namespace: operational-testing.specere.org
Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765
ID Suffix: 991a169f-b1c1-5b75-bc9c-fbaafe57a3d3

Namespace: isac-1.specere.org
Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910
ID Suffix: 0980a506-235e-5822-a2a2-a0af0d64c17f

Namespace: isac-2.specere.org
Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a
ID Suffix: 04791921-09be-53dc-b676-a0dbbf82b6ac


Type:URIObjectType.URL

Value:http://www.badguys.com/kickme?Hard

Namespace: development.specere.org
Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a
ID Suffix: 2d066889-d2d5-58a6-b6d2-1c3b848af84e

Namespace: production.specere.org
Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb
ID Suffix: fceb13b5-f605-524c-a2fc-90d57027c0fe

Namespace: operational-testing.specere.org
Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765
ID Suffix: 93b612bd-222f-5338-afc1-463b8674144a

Namespace: isac-1.specere.org
Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910
ID Suffix: 1b4881bf-04b5-5668-81ba-6191fbe30e03

Namespace: isac-2.specere.org
Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a
ID Suffix: 4f0cb8cd-cf10-59ee-98a8-90cefd2386b2


Type:pattern

Value:
"pattern": { "type": "twigs", "properties": [
],
"prop1": {
}, "prop2": {
}, "prop3": {
"key":"FileObject:hashes/hash/simple_hash_value", "operator":"equals", "value":"c38862b4835729d979e7940d72a48172"
"key":"FileObject:file_name", "operator":"contains", "value":"abcd.dll"
"key":"WinRegistryKeyObject:key",
"operator":"equals", "value":".DEFAULT\Software\Microsoft\Windows\CurrentVersion\Explorer\{19127AD2­394B­70F5 ­C650­B97867BAA1F7"
}, "prop4": {
} "prop5": {
}
"key":"WinRegistryKeyObject:hive", "operator":"equals", "value":"HKEY_USERS"
"key":"IPv4AddressObject:hive", "operator":"equals", "value":"1.2.3.4"
"condition": "(prop1 AND prop2) OR (prop3 AND prop4) FOLLOWED_BY prop5 WITHIN 15 MINS" }


Namespace: development.specere.org
Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a
ID Suffix: f7634169-2a18-5971-a919-585a4a913b72

Namespace: production.specere.org
Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb
ID Suffix: 10a3b6ea-2f93-55f7-a232-916c834e0e56

Namespace: operational-testing.specere.org
Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765
ID Suffix: 7428f623-54cb-5922-aeac-60b0bdfc025a

Namespace: isac-1.specere.org
Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910
ID Suffix: c782e0fe-4066-5d3b-9aba-7ee50d704bcc

Namespace: isac-2.specere.org
Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a
ID Suffix: 4df00be2-38cd-5a86-8b71-2bec5d3e2ef8







4.1.3. Version

The version number is in the most significant 4 bits of the time
stamp (bits 4 through 7 of the time_hi_and_version field).

The following table lists the currently-defined versions for this
UUID variant.

Msb0 Msb1 Msb2 Msb3 Version Description

0 0 0 1 1 The time-based version
specified in this document.

0 0 1 0 2 DCE Security version, with
embedded POSIX UIDs.


0 0 1 1 3 The name-based version
specified in this document
that uses MD5 hashing.

0 1 0 0 4 The randomly or pseudo-
randomly generated version
specified in this document.

0 1 0 1 5 The name-based version
specified in this document
that uses SHA-1 hashing.

The version is more accurately a sub-type; again, we retain the term
for compatibility.
Patrick Maroney
Office: (856)983-0001
Cell: (609)841-5104

<C690F973-64C5-4C00-889B-C1A5BB4A2A0B[2].png>

President
Integrated Networking Technologies, Inc.
PO Box 569
Marlton, NJ 08053

From: <cti-stix@lists.oasis-open.org> on behalf of John Wunder <jwunder@mitre.org>
Date: Friday, April 29, 2016 at 4:23 PM
To: "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org>
Subject: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties

All,

As we make progress on these specifications, it’s important to make sure that we have consensus on specification text and can document that consensus. To that end, the STIX co-chairs and editors would like to start a development cadence where we move content in the specifications through informal consensus, to review, to motions to approve the text.

In order to balance this desire with the desire to avoid hundreds of votes, we’d like to try the following process:

1. Content is developed by the SC and achieves some consensus, potentially in a mini-group (Status = Concept to Development)
2. Normative text is developed, iterated, and settles down (Status = Development)
3. We send a notice out to the cti-stix list saying that text is ready for review and formal acceptance (Status = Review)
4. After waiting 2 business days without hearing comments, we make a motion on the cti-stix list to accept the text as-is.
5. We’ll wait 5 business days to hear objections. If there are no objections, we’ll consider it accepted without a formal vote via unanimous consent (this will be made clear in the motions). If there are objections, depending on the type of objection and the exact circumstances we’ll either move back to the development/review phase or hold a ballot to approve the text via a majority vote. Once the motion is passed either via unanimous consent or via a ballot we’ll move it to the draft status (Status = Draft)
Draft status doesn’t mean that the text cannot change. We can make editorial changes through out the process without going back to earlier phases, but if we make any material changes we would move the concept back to the “Development” phase and start again. This is also not a replacement for the formal approval of the complete specification text when STIX 2.0 is done, it’s just a way to ensure that we have consensus at a more granular level as we move forward.

We hope this process gives you time to both have input prior to the official review phase and see what we’re moving to the review phase while at the same time avoiding votes on every single topic.

For this first round, please review the following sections in the STIX 2.0 specification:

Identifier: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.ko24ggw4eq0q
Timestamp: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.de8ah59mobqf
Timestamp Precision: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.sp8ake5xbk8j
Custom Properties: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.8072zpptza86

This is the start of the 2 day review process prior to the initial motion, so please provide any comments now so we can adjust. We can discuss comments on the Tuesday working call and, assuming we’re able to resolve them, make the motions to approve these sections on Wednesday morning.

Thanks everyone! I realize this may sound overly formal to some of you but in practice I’d expect that it just means you have more defined things to be reviewing at any given time.

John