OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

cti message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]


Subject: RE: [cti] CybOX Datatype Refactoring/Deprecation


Yeah. Upon reading Alex’s post I’d have to agree. It should really be up to the UI to do any defanging of links as necessary. Defanging is just a result of not using the correct tools in the first place…

 

Cheers

 

Terry MacDonald

Senior STIX Subject Matter Expert

SOLTRA | An FS-ISAC and DTCC Company

+61 (407) 203 206 | terry@soltra.com

 

 

From: Jordan, Bret [mailto:bret.jordan@bluecoat.com]
Sent: Tuesday, 23 February 2016 4:58 PM
To: Foley, Alexander - GIS <alexander.foley@bankofamerica.com>
Cc: Coderre, Robert <rcoderre@verisign.com>; Terry MacDonald <terry@soltra.com>; Barnum, Sean D. <sbarnum@mitre.org>; Kirillov, Ivan A. <ikirillov@mitre.org>; cti@lists.oasis-open.org
Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation

 

Really good points Alex, as always.  

 

Thanks,

 

Bret

 

 

 

Bret Jordan CISSP

Director of Security Architecture and Standards | Office of the CTO

Blue Coat Systems

PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050

"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 

 

On Feb 22, 2016, at 19:38, Foley, Alexander - GIS <alexander.foley@bankofamerica.com> wrote:

 

I’m on the other side of the fence here.  I don’t think we should be defanging at all.  It’s a flipping mess.

 

While I totally agree that the standard is what Terry and Rob say (with slight variations, as Outlook doesn’t require hxxp or brackets around every [.] to defang a link), we’ve spent the last six months figuring out the caveats to the standard.  For example, take our domain and URI regexes:

 

·        Domains (did you know .homedepot is a gTLD?  Neither did I until we had to write a regex to find it)

·        URIs containing domains with protocols

·        URIs containing domains without protocols

·        URIs containing IP addresses with protocols

·        URIs containing IP addresses without protocols

 

I promise you that the moment you tell a developer we need to start parsing URLs that start with poop://, woof://, meow:// or purr:// you will stop seeing the value in enabling defanging.

 

I seriously think we should finally decide that our core value is to have a machine-to-machine language and stop prioritizing human readable use cases, or to decide that we want a human readable language and stop prioritizing machine-to-machine use cases.  If we would have decided this during the timestamp discussions we’d currently have UNIX timestamps set to GMT format instead of using ISO timestamps to indicate a timezone that will never be changed.

 

Alex

 

From: cti@lists.oasis-open.org [mailto:cti@lists.oasis-open.org] On Behalf Of Coderre, Robert
Sent: Monday, February 22, 2016 7:51 PM
To: Terry MacDonald
Cc: Barnum, Sean D.; Kirillov, Ivan A.; cti@lists.oasis-open.org
Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation

 

This is pretty standard for our community. 

--
Rob


On Feb 22, 2016, at 4:49 PM, Terry MacDonald <terry@soltra.com> wrote:

Hi Sean,

 

I think this is an area where the benefits of the STIX/TAXII/CybOX ecosystem will outweigh the small chance that an organization will be majorly impacted by the selection of a single ‘most common’ way of defanging. This is DEFINITELY a case of the 80/20 rule applying.

 

From the various trusted lists I’m on, the most common defanging standard is:

 

 

Cheers

 

Terry MacDonald

Senior STIX Subject Matter Expert

SOLTRA | An FS-ISAC and DTCC Company

+61 (407) 203 206 | terry@soltra.com

 

 

From: cti@lists.oasis-open.org [mailto:cti@lists.oasis-open.org] On Behalf Of Barnum, Sean D.
Sent: Tuesday, 23 February 2016 4:29 AM
To: Kirillov, Ivan A. <ikirillov@mitre.org>; cti@lists.oasis-open.org
Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation

 

If we attempted to force such a decision then yes it would require less fields.

I would worry about taking such an approach however given that the real-world community of practitioners do NOT have such a "ONE standard way of performing defanging/refanging on all fields” and it is not for lack of trying. My understanding is that many different parties over the years have tried to drive towards a single algorithm and have not been successful. Different communities/players have strong feelings that certain approaches offer necessary capabilities that others do not.

I know that Pat Maroney has been part of these sorts of discussion within DSIE and elsewhere so I would encourage him to comment here.

 

The danger of us thinking we can decide and enforce the ONE way of doing something even though the communities that we are attempting to support clearly do not yet agree is that we could be making STIX/CybOX untenable for whole swaths of players.  I would think this issue potentially very much akin to the whole XML vs JSON issue.

I think it makes total sense to work to identify ONE default approach but don’t think we can then simply disallow all other approaches. I think 2 or 3 fields is an acceptable price to pay for something like this.

 

sean

 

From: Steve Cell <ikirillov@mitre.org>
Date: Monday, February 22, 2016 at 12:11 PM
To: "Barnum, Sean D." <sbarnum@mitre.org>, "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation

 

Ah, I see – that makes sense. So I suppose the corollary to this is, what if we specified ONE standard way of performing defanging/refanging on all fields, such as by base64-encoding the data? Seems like this would make the specification simpler, and consumers would only need to understand/implement one refanging method.

 

Regards,

Ivan

 

From: Sean Barnum <sbarnum@mitre.org>
Date: Monday, February 22, 2016 at 10:02 AM
To: Ivan Kirillov <ikirillov@mitre.org>, "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation

 

>[Ivan] That makes sense, though I do wonder if in such a case Party A will indeed provide refanging information if they do not have certainty about the defanging method used. >Also, the refanging field as designed before was meant to be a machine-parseable transform (e.g, a regex); is there still the expectation that it is actually used in this way (i.e., >that consumers would actually machine parse it and apply it to the data) in the wild?

 

I apologize if my comment on refanging was confusing. It was not intended to be tied directly to the case of no defanging algorithm provided.

It was just meant in general having the ability to specify a refanging algorithm. I would certainly think that having an explicit way to automatically refang content (within a GUI for example) would still be very useful. If everyone was using the same defanging algorithm then the refanging could likely be hardcoded and would not need a separate field but I do not believe that this is the reality we live in or are likely to live in for a while. Some communities or organizations use diverse defanging approaches and the only way to refang such content automatically would require explicit specification of a refanging algorithm.

 

</my-two-cents>

 

sean

 

From: Steve Cell <ikirillov@mitre.org>
Date: Monday, February 22, 2016 at 11:54 AM
To: "Barnum, Sean D." <sbarnum@mitre.org>, "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation

 

 

  1.  
    1. Could we limit the properties associated with obfuscation and defanging/refanging to the following?
      1. obfuscation_algorithm_ref (URI): a reference to a description of the algorithm used to obfuscate the object property
      2. defanging_algorithm_ref (URI): a reference to a description of the algorithm used to defang the object property

 

[sean]So for these I assume you are suggesting that the presence of these fields implicitly means that the field is obfuscated or defanged thereby negating the need for the explicit field. The only potential issue I see with that is when the field is obfuscated or defanged but no algorithm is provided or available. 

 

[Ivan] Yes, that’s correct – John and I thought that we could likely remove the is_obfuscated/is_defanged fields, since the presence of the _ref fields is enough to state that a field is obfuscated and/or defanged. Do we as the community feel that not specifying a reference to the obfuscation/defanging algorithm used would be a common occurrence? 

 

My understanding of a common use case is that Party A passes along data to Party B that was not originally produced by Party A where they have determined the data is obfuscated or defanged and wish to convey that to Party B but do not have certainty on the exact algorithm used.

Either way, I would think you will also need to include refanging

 

[Ivan] That makes sense, though I do wonder if in such a case Party A will indeed provide refanging information if they do not have certainty about the defanging method used. Also, the refanging field as designed before was meant to be a machine-parseable transform (e.g, a regex); is there still the expectation that it is actually used in this way (i.e., that consumers would actually machine parse it and apply it to the data) in the wild?

 

Regards,

Ivan

 

From: Sean Barnum <sbarnum@mitre.org>
Date: Tuesday, February 16, 2016 at 7:43 AM
To: Ivan Kirillov <ikirillov@mitre.org>, "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation

 

Comments inline

 

From: <cti@lists.oasis-open.org> on behalf of Steve Cell <ikirillov@mitre.org>
Date: Monday, February 15, 2016 at 10:45 AM
To: "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation

 

Are there any additional thoughts on this topic? Just trying to see if have any consensus on:

  1. This overall approach, i.e., supporting specific metadata properties as part of CybOX Object fields (implemented via JSON Objects)
  2. The specific set of metadata properties that we should support
    1. Does it make sense to include an “is_random” property for assertions of randomness in the data of a particular field?

 

[sean]I have no dog in this hunt but know that this field came from request came from digital forensics use cases where they were fully aware that this was an observation and not a pattern. We should ask Eoghan for his thoughts on this.

  1.  
    1. Could we limit the properties associated with obfuscation and defanging/refanging to the following?
      1. obfuscation_algorithm_ref (URI): a reference to a description of the algorithm used to obfuscate the object property
      2. defanging_algorithm_ref (URI): a reference to a description of the algorithm used to defang the object property

 

[sean]So for these I assume you are suggesting that the presence of these fields implicitly means that the field is obfuscated or defanged thereby negating the need for the explicit field. The only potential issue I see with that is when the field is obfuscated or defanged but no algorithm is provided or available. My understanding of a common use case is that Party A passes along data to Party B that was not originally produced by Party A where they have determined the data is obfuscated or defanged and wish to convey that to Party B but do not have certainty on the exact algorithm used.

Either way, I would think you will also need to include refanging

 

Regards,

Ivan

 

From: Terry MacDonald <terry@soltra.com>
Date: Tuesday, February 9, 2016 at 4:49 PM
To: Ivan Kirillov <ikirillov@mitre.org>, "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: RE: [cti] CybOX Datatype Refactoring/Deprecation

 

What about using the relationships to handle the levels of obfuscation? That would allow description of obfuscated and encrypted objects within other obfuscated objects.

 

In other words to describe a PDF containing obfuscated _javascript_ that decodes and extracts a windows executable from within the PDF we could do something like this:

 

 

<image001.jpg>

 

Would that sort of thing work? BTW the relationship thing seems pretty darn powerful when used like this….

 

Cheers

 

Terry MacDonald

Senior STIX Subject Matter Expert

SOLTRA | An FS-ISAC and DTCC Company

+61 (407) 203 206 | terry@soltra.com

 

 

From:cti@lists.oasis-open.org [mailto:cti@lists.oasis-open.org] On Behalf Of Kirillov, Ivan A.
Sent: Wednesday, 10 February 2016 8:16 AM
To: cti@lists.oasis-open.org
Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation

 

>> As much as I would like fields to just be the value, I agree that we probably need this layer of indirection to support these extra properties.

 

I was also hoping for Observations/Objects to just hold the value, but some of these fields do seem to be necessary. I think the only other way to do this would be via relationships, but then you’d have to assign an ID to each field, and that would just get messy quickly without any real benefit.

 

>> We’ll need to decide what that means for patterning, if anything, by the way.

 

Yup, agreed. My initial thinking is that these properties don’t (or shouldn’t) have any significance for patterning. 

 

>>Given that we’re talking about actual observations here (instances), is “appears_random” appropriate? That seems to be a statement about a pattern of behavior over several observations rather than a single observation.

 

I’ve wondered the same – this does seem to be an assertion that one wouldn’t typically make based on a single data point. Also, it seems like you’d want to be able to specify some level of confidence and perhaps other properties with this as well, so I’m also wondering if this belongs here or perhaps in some separate structure (which could also be used for related observations such as entropy, etc.)?

 

>>Can we combine the “obfuscation” fields into some single “obfuscation_algorithm”, representing a URI or something? Then if the field is there, it’s obfuscated using that. If it’s not, then it’s not obfuscated.

 

Yeah, that makes good sense to me. I think having the “is_defanged” boolean isn’t strictly necessary, as you can infer that something is obfuscated if the corresponding “obfuscation_algorithm” field is populated. Having a URI in there seems reasonable.

 

>> Given that we’re now talking about machine-to-machine transmissions, is defanging still important? Can’t the raw data be exchanged as it is and then let tools to present it to users defanged however they want? It seems weird to me to transmit it defanged if it’s M2M (vs. a PDF that people might accidentally click).

 

I had the same question when we were first implementing defanging/fanging; it does seem like something that should be done at the data ingest/GUI layer rather than at the data specification layer. It’s definitely odd to support it if there is no expectation that humans will actually look at the data.

 

>> If we do need to include it, can we similarly collapse these attributes into one “defanging_algorithm” URI? They seem very redundant right now.

 

I’m also onboard with this. So we’d get something like:

  • obfuscation_algorithm_ref (URI): a reference to a description of the algorithm used to obfuscate the object property
  • defanging_algorithm_ref (URI): a reference to a description of the algorithm used to defang the object property
  • refanging_transform (string): an automated transform that can be applied to the object property in order to refang it to its original format

Regards,

Ivan

 

From: <cti@lists.oasis-open.org> on behalf of John Wunder <jwunder@mitre.org>
Date: Tuesday, February 9, 2016 at 1:56 PM
To: "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation

 

As much as I would like fields to just be the value, I agree that we probably need this layer of indirection to support these extra properties. We’ll need to decide what that means for patterning, if anything, by the way.

 

On the attributes you listed, I just have a few questions:

  1. Given that we’re talking about actual observations here (instances), is “appears_random” appropriate? That seems to be a statement about a pattern of behavior over several observations rather than a single observation.
  2. Can we combine the “obfuscation” fields into some single “obfuscation_algorithm”, representing a URI or something? Then if the field is there, it’s obfuscated using that. If it’s not, then it’s not obfuscated.
  3. For defanging, I have a two part question:
    1. Given that we’re now talking about machine-to-machine transmissions, is defanging still important? Can’t the raw data be exchanged as it is and then let tools to present it to users defanged however they want? It seems weird to me to transmit it defanged if it’s M2M (vs. a PDF that people might accidentally click).
    2. If we do need to include it, can we similarly collapse these attributes into one “defanging_algorithm” URI? They seem very redundant right now.

John

 

From: <cti@lists.oasis-open.org> on behalf of Ivan Kirillov <ikirillov@mitre.org>
Date: Tuesday, February 9, 2016 at 12:21 PM
To: "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: [cti] CybOX Datatype Refactoring/Deprecation

 

Sending this to the broader CTI since it’s part of the STIX/CybOX Indicator tranche. 

 

Given that we’ll be splitting out embedded patterning from CybOX, there are changes that we need to make to the base datatypes used currently for specifying properties in CybOX Objects. At a high level, this involves the following:

  • Deprecating any types/enums associated with patterning
  • Refactoring the base BaseObjectPropertyType (the base type for ALL Object properties) to remove patterning and permit more accurate datatype based field specification in Objects (e.g., an integer is represented as an integer and not a string). Accordingly, this would entail creating a new BaseStringObjectPropertyType and BaseNumberObjectPropertyType that extend from the refactored BaseObjectPropertyType.
  • Updating the existing ObjectPropertyTypes (e.g., StringObjectPropertyType, DoubleObjectPropertyType, etc.) to extend from the new BaseString/Number ObjectPropertyTypes

More details on these proposed changes can be found here: https://github.com/CybOXProject/schemas/issues/416#issuecomment-181916815

 

Accordingly, this would result in CybOX instance content that looks like the following; each property is an Object (so that it can specify metadata about refanging/defanging/etc) and thus the “value” field MUST be included in every property to capture its actual value.

 

{
   "size": {"value": 23134},
   "file-system-properties" : {"file_name": {"value":"test.dll",
                                             "observed_encoding":"utf-8"}}
}

 

Some open questions:

  • Does this approach seem reasonable?
  • Do we need all of the properties defined on the BaseObjectPropertyType (see below), or are there other properties that we should consider adding?
    • appears_random (boolean): whether the property appears random in nature
    • is_obfuscated (boolean): whether the property has been obfuscated
    • obfuscation_algorithm_ref (string): a reference to a description of the algorithm used to obfuscate the object property
    • is_defanged (boolean): whether the property has been defanged
    • defanging_algorithm_ref (string): a reference to a description of the algorithm used to defang the object property
    • refanging_transform_type (string): the type of refanging transform specified in the refanging_transform property
    • refanging_transform (string): an automated transform that can be applied to the object property in order to refang it to its original format
    • observed_encoding (string): the encoding of the string as observed
  • For the extensions of the BaseStringObjectPropertyType and BaseNumberObjectPropertyType, such as PositiveIntegerPropertyType, should we define additional patterns (i.e. regular expressions) for field-level validation of these types? E.g., in this example, to make sure that the number is a valid positive integer?

Regards,

Ivan


This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message.

 



[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]