OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

cti-stix message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]


Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties


I agree with this. Those wanting to have deterministic IDs can still do so as a custom field, and if it ends up being useful they could potentially even make it into the specification in future versions. But, for simplicity’s sake, it seems to me like we should be very clear and limit it to UUID4.

Based on this e-mail conversation and the conversation on the call we seem to have fairly strong, but not unanimous, consensus that UUIDv4, as-is, would be the best solution. Given that and the lengthy previous discussions on this topic, I’d like to make a motion open a ballot declaring that the following text, contained in the STIX specification for identifier, be accepted and moved to “draft” status:

An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUIDv4], where [object-type] is the exact value from the type field of the object being identified or referenced and [UUIDv4] is an RFC 4122 compliant Version 4 UUID. The uuid field MUST be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID).

John

From: <cti-stix@lists.oasis-open.org> on behalf of Terry MacDonald <terry.macdonald@cosive.com>
Date: Tuesday, May 3, 2016 at 6:36 PM
To: Paul Patrick <ppatrick@isightpartners.com>
Cc: Bret Jordan <bret.jordan@bluecoat.com>, "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org>, "Eric.Burger@georgetown.edu" <Eric.Burger@georgetown.edu>
Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties

I personally believe we need to specify just one way to do it unless there is an important reason that would affect a significant part of the community. I don't see a significant population being affected by the specifying of uuidv4 based object IDs, yet the potential for problems when the object ID creation process isn't common across all objects is concerning.

I believe we should stay with just uuidv4 based object IDs.

Cheers
Terry MacDonald

On 4/05/2016 06:14, "Paul Patrick" <ppatrick@isightpartners.com> wrote:
You’d have to ask Pat if that is the primary/sole issue he was proposing deterministic identifiers to address.  As for duplication detection, I don’t think comparing identifiers and revision alone is the best was to accomplish that, but I’ll leave that as an implementation detail for a provider.

But given RFC 4122, I think the best we could do is state that the uuid portion of the identifier MUST be compliant with RFC 4122 and that the specification RECOMMENDS the of the uuid4 algorithm in the generation of uuid values.  This statement would be able to be enforced and verified and yet allow folks like Pat and others to use the uuid5 algorithm if they so desire.  I suspect that might be enough to get Pat to agree and us avoid further discussion on this topic.


Paul P.

From: <cti-stix@lists.oasis-open.org> on behalf of "Jordan, Bret" <bret.jordan@bluecoat.com>
Date: Tuesday, May 3, 2016 at 3:09 PM
To: "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org>, "Eric.Burger@georgetown.edu" <Eric.Burger@georgetown.edu>
Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties
Resent-From: <Paul.Patrick@FireEye.com>

I thought one of the primary goals of using deterministic IDs was de-duplication?  The only way you can do de-duplication is if you are hashing all of the relevant fields, of which versioning would be part of.  

We have debated this issue almost as long as we have debated timestamps. So, as with timestamps, there is a difference between "I just do not like it" and "this is the wrong way to go and here is why".  If either the timestamps or IDs that we have proposed are wrong, or are going to make life miserable, please speak up and explain why. 

Maybe our resident academic can chime in here?  

Thanks,

Bret



Bret Jordan CISSP
Director of Security Architecture and Standards | Office of the CTO
Blue Coat Systems
PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 

On May 3, 2016, at 11:04, Jason Keirstead <Jason.Keirstead@ca.ibm.com> wrote:

What would be the proposed third value used in the tuple that would not break versioning?

-
Jason Keirstead
STSM, Product Architect, Security Intelligence, IBM Security Systems
www.ibm.com/security | www.securityintelligence.com

Without data, all you are is just another person with an opinion - Unknown


<graycol.gif>Paul Patrick ---05/03/2016 01:37:13 PM---Folks, I don’t mean to cause a stir or re-open topics, but on today’s call during the discussion abo

From: Paul Patrick <ppatrick@isightpartners.com>
To: "Jordan, Bret" <bret.jordan@bluecoat.com>, Patrick Maroney <Pmaroney@Specere.org>
Cc: "Wunder, John A." <jwunder@mitre.org>, "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org>
Date: 05/03/2016 01:37 PM
Subject: Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties
Sent by: <cti-stix@lists.oasis-open.org>





Folks,

I don’t mean to cause a stir or re-open topics, but on today’s call during the discussion about identifiers there appeared to be some confusion about how Patrick had proposed having deterministic identifiers. If I heard the comments correctly, there seemed to be the opinion that you had to hash the entire object, which would include the revision information, in order to generate the identifier. But as I read Pat’s proposal, that isn’t what he proposed if you look back below at his email.

In Pat’s proposal, he proposed using a tuple of organization namespace (aka domain name), type of object, and one or more value(s) as input to the uuid5 algorithm. The organization namespace and type of object was used to significantly the possibility of collisions that could occur with the key value(s) (e.g., IP address value) when the tuple contents were hashed using the SHA1 per the uuid5 algorithm. Because the entire object, including revision information, isn’t used to determine the hash there isn’t the issue of breaking relationships.

From an implementors point of view, I've been doing a very similar approach for a while without any issues. It avoids the need to maintain a list of identifiers that we’ve used before and thus simplifies the effort to support. In addition, the generated identifiers are valid UUIDs with the same number of bytes, etc. Per RFC 4122, there is no way, beyond making sure that the timestamp portion of the UUID is in the future, to validate if a UUID is valid much less enforce that the UUID had been generated using the uuid4 algorithm or not. So attempting to enforce that the uuid portion of identifiers are generated by the uuid4 algorithm is pointless.

So I submit that if someone wants to use the uuid3 or uuid5 algorithm, they should not be prohibited to do so but our text should clearly state that it is RECOMMENDED that the uuid4 algorithm be used.

I’m happy to tweak the text in the document accordingly, but wanted to raise what I believe was confusion on this topic to the group before doing any text changes.


Paul Patrick

From: <cti-stix@lists.oasis-open.org> on behalf of "Jordan, Bret" <bret.jordan@bluecoat.com>
Date:
Monday, May 2, 2016 at 6:01 PM
To:
Patrick Maroney <Pmaroney@Specere.org>
Cc:
"Wunder, John A." <jwunder@mitre.org>, "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org>
Subject:
Re: [cti-stix] Identifier: [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties
Resent-From:
<Paul.Patrick@FireEye.com>
      Pat,

      Are you planning then to build your deterministic IDs without the use of the versioning information? Because, if you do use versioning information for your IDs, to help with all the things you outline below (which you would need to actually find real duplicates), your deterministic IDs will change with ever revision of the object. And thus all of your relationships will break.

      Thanks,

      Bret



      Bret Jordan CISSP
      Director of Security Architecture and Standards | Office of the CTO
      Blue Coat Systems
      PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050
      "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."
          On Apr 30, 2016, at 13:27, Patrick Maroney <Pmaroney@Specere.org> wrote:

          re:."This is the start of the 2 day review process prior to the initial motion, so please provide any comments now so we can adjust. We can discuss comments on the Tuesday working call and, assuming we’re able to resolve them, make the motions to approve these sections on Wednesday morning."
              I have added notional changes to Identifier to add previously requested support for Deterministic UUID Identifiers. The current language arbitrarily constrains the RFC 1422 UUID identifier to the UUIDv4 pseudo-random generation method.

              Format and language are notional and will need corrections by the editors.

              Basis:
                If certain Vendor Factions, Communities of Trust,etc. want to use psuedo-random UUIDv4 generation
                    -- They can!!!

                if Intelligence Communities, Licensed Content Providers, Aggregators, etc. want/need to establish provenance, detect leakages and license infringement, Generate and track UUIDs specific to distribution channels, provide anonymous path traceability back through a dissemination chain the the Source Analyst/Organization, , etc.

                    -- They can To!!!

                The method of generation of any UUID can of course be determined from the generated UUID.
                    -- Win-Win for both camps and with no cost that I can discern




                4.5.​ Identifier
                Type Name: identifier
                Status: Review
                MVP: Yes

                An identifier uniquely identifies a STIX top-level object. Identifiers MUST follow the form [object-type]--[UUIDv4], where [object-type] is the exact value from the type field of the object being identified or referenced and [uuid] is an RFC 4122 compliant Version 4 UUID. The uuid field MUST be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID).

                4.5.1 Version 4 UUID. The uuid field MAY be generated according to the algorithm(s) defined in RFC 4122, Section 4.4 (Version 4 UUID).

                4.5.1.​2 Examples
                {
                "type": "indicator",
                "id": "indicator--e2e1a340-4415-4ba8-9671-f7343fbf0836",
                ...
                }

                4.5.2 ​Version 5 UUID. A Deterministic Object ID MAY be generated by the original Source making the assertion or reporting a sighting.

                Object ID = uuid5(NameSpace[.subNameSpace] + ObjectClass + ObjectType + ObjectValue)

                PublIcNameSpace = NameSpace[.subNameSpace]

                ObfuscatedNameSpace = uuid5(NameSpace.Key), where Key = uuid5(subNameSpace)


                (4.5.2.1) Examples

                NameSpace = 'specere.org'
                subNameSpace = 'isac-2'
                ObjectClass = 'AddressObject'
                ObjectType = 'ipv4-addr'
                ObjectValue = '1.2.3.5'

                AddressObject.ipv4-addr::622fca15-8e5f-5503-80af-bdbd7a69e9f7

                Demonstrate deterministic generation of CTI Object IDs
                Generates Namespace Prefix deterministic generation of CTI Object IDs

                Type:AddressObjectType.ipv4-addr

                Value:1.2.3.4

                Namespace: development.specere.org
                Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a
                ID Suffix: 524d9aa0-2dc5-55b9-b0bc-312314f19a69

                Namespace: production.specere.org
                Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb
                ID Suffix: 377d9ac3-3dd6-594d-a1d6-9250bb268b69

                Namespace: operational-testing.specere.org
                Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765
                ID Suffix: e3c285c9-676d-53e8-beaa-abca2fbd0b10

                Namespace: isac-1.specere.org
                Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910
                ID Suffix: 432d8296-a2e7-5280-9ea9-ca5ca0c9bb26

                Namespace: isac-2.specere.org
                Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a
                ID Suffix: ed8e845d-2eb9-50e2-a5df-aa35df074c30


                Type:AddressObjectType.ipv4-addr

                Value:1.2.3.5

                Namespace: development.specere.org
                Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a
                ID Suffix: 6f596396-4e4f-501e-8631-58b7ccadca83

                Namespace: production.specere.org
                Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb
                ID Suffix: 4fb2a115-30f9-5c84-8d85-bc40c6a00ff0

                Namespace: operational-testing.specere.org
                Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765
                ID Suffix: bad7a975-e1ba-5908-bd46-e88ec61a4d2f

                Namespace: isac-1.specere.org
                Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910
                ID Suffix: 69c3ebaa-db59-5195-a391-0dd1b5a43c48

                Namespace: isac-2.specere.org
                Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a
                ID Suffix: 622fca15-8e5f-5503-80af-bdbd7a69e9f7


                Type:DomainNameType.TLD

                Value:badguys.com

                Namespace: development.specere.org
                Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a
                ID Suffix: d73294e6-502f-5bf5-8df8-ec7992cfb6da

                Namespace: production.specere.org
                Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb
                ID Suffix: b6e7b0f4-a655-5330-a2a2-be0066a221bc

                Namespace: operational-testing.specere.org
                Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765
                ID Suffix: 991a169f-b1c1-5b75-bc9c-fbaafe57a3d3

                Namespace: isac-1.specere.org
                Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910
                ID Suffix: 0980a506-235e-5822-a2a2-a0af0d64c17f

                Namespace: isac-2.specere.org
                Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a
                ID Suffix: 04791921-09be-53dc-b676-a0dbbf82b6ac


                Type:URIObjectType.URL

                Value:http://www.badguys.com/kickme?Hard

                Namespace: development.specere.org
                Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a
                ID Suffix: 2d066889-d2d5-58a6-b6d2-1c3b848af84e

                Namespace: production.specere.org
                Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb
                ID Suffix: fceb13b5-f605-524c-a2fc-90d57027c0fe

                Namespace: operational-testing.specere.org
                Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765
                ID Suffix: 93b612bd-222f-5338-afc1-463b8674144a

                Namespace: isac-1.specere.org
                Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910
                ID Suffix: 1b4881bf-04b5-5668-81ba-6191fbe30e03

                Namespace: isac-2.specere.org
                Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a
                ID Suffix: 4f0cb8cd-cf10-59ee-98a8-90cefd2386b2


                Type:pattern

                Value:
                "pattern": { "type": "twigs", "properties": [
                ],
                "prop1": {
                }, "prop2": {
                }, "prop3": {
                "key":"FileObject:hashes/hash/simple_hash_value", "operator":"equals", "value":"c38862b4835729d979e7940d72a48172"
                "key":"FileObject:file_name", "operator":"contains", "value":"abcd.dll"
                "key":"WinRegistryKeyObject:key",
                "operator":"equals", "value":".DEFAULT\Software\Microsoft\Windows\CurrentVersion\Explorer\{19127AD2­394B­70F5 ­C650­B97867BAA1F7"
                }, "prop4": {
                } "prop5": {
                }
                "key":"WinRegistryKeyObject:hive", "operator":"equals", "value":"HKEY_USERS"
                "key":"IPv4AddressObject:hive", "operator":"equals", "value":"1.2.3.4"
                "condition": "(prop1 AND prop2) OR (prop3 AND prop4) FOLLOWED_BY prop5 WITHIN 15 MINS" }


                Namespace: development.specere.org
                Object ID Prefix-Obfuscated d4b75663-a072-5bee-b9e6-a3846bdb393a
                ID Suffix: f7634169-2a18-5971-a919-585a4a913b72

                Namespace: production.specere.org
                Object ID Prefix-Obfuscated 40f364c5-8811-590d-8015-00252b64ddbb
                ID Suffix: 10a3b6ea-2f93-55f7-a232-916c834e0e56

                Namespace: operational-testing.specere.org
                Object ID Prefix-Obfuscated 6fb06dda-a8b7-5ec0-b6ad-1e264d172765
                ID Suffix: 7428f623-54cb-5922-aeac-60b0bdfc025a

                Namespace: isac-1.specere.org
                Object ID Prefix-Obfuscated 948cbebf-81f8-5a2d-8a29-fba063335910
                ID Suffix: c782e0fe-4066-5d3b-9aba-7ee50d704bcc

                Namespace: isac-2.specere.org
                Object ID Prefix-Obfuscated 083df4c3-869b-50d3-91ba-7de554c19d0a
                ID Suffix: 4df00be2-38cd-5a86-8b71-2bec5d3e2ef8





                4.1.3. Version

                The version number is in the most significant 4 bits of the time
                stamp (bits 4 through 7 of the time_hi_and_version field).

                The following table lists the currently-defined versions for this
                UUID variant.

                Msb0 Msb1 Msb2 Msb3 Version Description

                0 0 0 1 1 The time-based version
                specified in this document.

                0 0 1 0 2 DCE Security version, with
                embedded POSIX UIDs.


                0 0 1 1 3 The name-based version
                specified in this document
                that uses MD5 hashing.

                0 1 0 0 4 The randomly or pseudo-
                randomly generated version
                specified in this document.

                0 1 0 1 5 The name-based version
                specified in this document
                that uses SHA-1 hashing.

                The version is more accurately a sub-type; again, we retain the term
                for compatibility.
          Patrick Maroney
          Office: (856)983-0001
          Cell: (609)841-5104

          <C690F973-64C5-4C00-889B-C1A5BB4A2A0B[2].png>

          President
          Integrated Networking Technologies, Inc.
          PO Box 569
          Marlton, NJ 08053

          From: <cti-stix@lists.oasis-open.org> on behalf of John Wunder <jwunder@mitre.org>
          Date:
          Friday, April 29, 2016 at 4:23 PM
          To:
          "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org>
          Subject:
          [cti-stix] Formalizing Consensus - Identifier, Timestamp, Timestamp Precision, and Custom Properties

          All,

          As we make progress on these specifications, it’s important to make sure that we have consensus on specification text and can document that consensus. To that end, the STIX co-chairs and editors would like to start a development cadence where we move content in the specifications through informal consensus, to review, to motions to approve the text.

          In order to balance this desire with the desire to avoid hundreds of votes, we’d like to try the following process:
            1. Content is developed by the SC and achieves some consensus, potentially in a mini-group (Status = Concept to Development)
            2. Normative text is developed, iterated, and settles down (Status = Development)
            3. We send a notice out to the cti-stix list saying that text is ready for review and formal acceptance (Status = Review)
            4. After waiting 2 business days without hearing comments, we make a motion on the cti-stix list to accept the text as-is.
            5. We’ll wait 5 business days to hear objections. If there are no objections, we’ll consider it accepted without a formal vote via unanimous consent (this will be made clear in the motions). If there are objections, depending on the type of objection and the exact circumstances we’ll either move back to the development/review phase or hold a ballot to approve the text via a majority vote. Once the motion is passed either via unanimous consent or via a ballot we’ll move it to the draft status (Status = Draft)
          Draft status doesn’t mean that the text cannot change. We can make editorial changes through out the process without going back to earlier phases, but if we make any material changes we would move the concept back to the “Development” phase and start again. This is also not a replacement for the formal approval of the complete specification text when STIX 2.0 is done, it’s just a way to ensure that we have consensus at a more granular level as we move forward.

          We hope this process gives you time to both have input prior to the official review phase and see what we’re moving to the review phase while at the same time avoiding votes on every single topic.

          For this first round, please review the following sections in the STIX 2.0 specification:

          Identifier: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.ko24ggw4eq0q
          Timestamp: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.de8ah59mobqf
          Timestamp Precision: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.sp8ake5xbk8j
          Custom Properties: https://docs.google.com/document/d/1HJqhvzO35h62gQGPvghVRIAtQrZn3_J__0UcDAj-NXY/edit#heading=h.8072zpptza86

          This is the start of the 2 day review process prior to the initial motion, so please provide any comments now so we can adjust. We can discuss comments on the Tuesday working call and, assuming we’re able to resolve them, make the motions to approve these sections on Wednesday morning.

          Thanks everyone! I realize this may sound overly formal to some of you but in practice I’d expect that it just means you have more defined things to be reviewing at any given time.

          John






[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]