OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

cti message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]


Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation


I’m not sold on the need for capturing obfuscation (seems like it falls in a bucket similar to defanging), but supporting the capture of observed encoding is something that makes more sense. 

If we were to use a flattened representation for this, it would mean that every field that has a string as its base type would also need a corresponding field for the capture of observed encoding. Accordingly, this would entail MANY extra fields in the Object data models, which doesn’t seem ideal.

Ultimately, though, this question boils down to what would be easiest for content producers/consumers to work with – a flattened representation:

{
  “email_body”: “blah blah”,
  “email_body_observed_encoding”: “utf8”
}

Or a nested one?

{
  “email_body”: {“value”:“blah blah”, “observed_encoding”:”utf8”}
}


Regards,
Ivan

From: Sean Barnum <sbarnum@mitre.org>
Date: Thursday, February 25, 2016 at 10:20 AM
To: John Wunder <jwunder@mitre.org>, Ivan Kirillov <ikirillov@mitre.org>, "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation

Well, yeah. You could flatten anything. The question is how much is appropriate.
Though if flattening is the desired approach then it kind of negates Ivan’s issue below as well.

sean

From: John Wunder <jwunder@mitre.org>
Date: Thursday, February 25, 2016 at 12:14 PM
To: "Barnum, Sean D." <sbarnum@mitre.org>, Steve Cell <ikirillov@mitre.org>, "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation

You could always flatten that as well, right?

{
  “email_body”: “blah blah”,
  “email_body_observed_encoding”: “utf8”
}

From: <cti@lists.oasis-open.org> on behalf of Sean Barnum <sbarnum@mitre.org>
Date: Thursday, February 25, 2016 at 12:04 PM
To: Ivan Kirillov <ikirillov@mitre.org>, "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation

>My issue with this approach is that it would impose itself on how CybOX Object fields are specified in an instance document (making all CybOX Object values a JSON Object), >resulting in some unnecessary complications even in cases where defanging is not used:

I don’t think this necessarily goes away no matter what is decided on defanging.
Obfuscation and observed_encoding are still field level issues.

sean

From: <cti@lists.oasis-open.org> on behalf of Steve Cell <ikirillov@mitre.org>
Date: Thursday, February 25, 2016 at 10:49 AM
To: "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation

I’m still of the belief that defanging is a presentation-layer problem. However, to go with Pat’s first example: 

       {
            "id": "object--2",
            "type": "IPAddress",
            "IPv4": "MTI3LjAuMC4x"
        },
        {
            "id": "object--3",
            "type": "IPAddress",
            "IPv4": "MTkyLjE2OC4xLjM0"
        }

This is also along the lines of what I was envisioning for a single defanging method (option 2 in my email below). I’m not sure if the other approach of Base64 encoding an entire CybOX Object is tenable, as this could create some serious burdens for consumers trying to parse the data (imagine a document with mixed plaintext JSON and Base64 encoded blobs). However, there are a few open issues and questions around the first approach:
  • How would you specify which fields on a CybOX Object are defanged (without resorting to the existing field-basd approach)? For simple Objects such as IPv4 Address, a single “is_defanged” field at the root Object level would suffice, but this wouldn’t work for more complex Objects. 
  • As a corollary, do we need to support defanging on ALL Objects or just a limited subset? If the latter, perhaps we could agree on which fields we care about defanging, and create a defanged sibling field for this purpose in their corresponding models, e.g.,
       {
            "id": "object--2",
            "type": “ipv4addr-object",
            “ip_address": "192.168.1.1"
        },
        {
            "id": "object--3",
            "type": “ipv4addr-object",
            "ip_address_defanged": "MTkyLjE2OC4xLjM0"
        }

>I’m not suggesting multiple methods of defanging but rather the STIX spec should not mandate one or the other but rather just have STIX communicate what is in the content and let the implementers decide what works best for their use cases.

My issue with this approach is that it would impose itself on how CybOX Object fields are specified in an instance document (making all CybOX Object values a JSON Object), resulting in some unnecessary complications even in cases where defanging is not used:

        {
            "id": "object--2",
            "type": “ipv4addr-object",
            “ip_address": {“value”:"192.168.1.1"}
        },
        {
            "id": "object--3",
            "type": “ipv4addr-object",
            "ip_address": {“value”:"MTkyLjE2OC4xLjM0, “defanging_method”:base64”}
        }

Regards,
Ivan

From: <cti@lists.oasis-open.org> on behalf of Allan Thomson <athomson@lgscout.com>
Date: Thursday, February 25, 2016 at 2:13 AM
To: Alexander Foley <alexander.foley@BANKOFAMERICA.COM>
Cc: Patrick Maroney <Pmaroney@Specere.org>, Terry MacDonald <terry@soltra.com>, Bret Jordan <bret.jordan@bluecoat.com>, Jason Keirstead <Jason.Keirstead@ca.ibm.com>, Paul Patrick <ppatrick@isightpartners.com>, John Wunder <jwunder@mitre.org>, "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation

Hi Alex - 

In response to your specific "your suggestions seemed to indicate that we should support multiple methods of defanging for the same indicator / content types.

The answer is not really.

I’m not suggesting multiple methods of defanging but rather the STIX spec should not mandate one or the other but rather just have STIX communicate what is in the content and let the implementers decide what works best for their use cases.

I believe there a valid use cases for both original and defanged content.

allan

On Feb 24, 2016, at 3:59 PM, Foley, Alexander - GIS <alexander.foley@BANKOFAMERICA.COM> wrote:

OK so I must admit I’m getting confused.  Pat, below you’re describing “truth on the wire”, which I think means that content is fanged on the wire.  Paul, I think your message from yesterday seemed to indicate that you’d be supportive of one way to defang content, accounting for the differences between indicator / content types.  Allan, your suggestions seemed to indicate that we should support multiple methods of defanging for the same indicator / content types.
 
To further complicate the mess, I’d like to focus on a specific example.  Today, the sharing communities I am a part of send executables almost exclusively as password protected compressed archives, with the password being generally accepted amongst the community.  Am I wrong to assume that binaries shared as STIX / CybOX content would not be executable because they would be shared as part of a larger blob of JSON?  Is there a need to “defang” these JSON-encoded blobs so they’re not blocked by perimeter controls?  Does it go to the level that we need to support multiple kinds of enclosures and user-selected passwords?  Translating this into the URI use case, is there anyone encouraging the ability to enable multiple methods of defanging a URI?
 
As far as I can tell, defanging exists almost exclusively so that email clients do not turn URLs into hyperlinks or so that users don’t accidentally execute a binary or that perimeter controls don’t block a binary.  These seem to be application level or user error problems that could be solved by defanging all content but would not be solved by optionally allowing for defanged content.
 
Let me put this another way.  We use threat intelligence APIs to retrieve IOCs from many sources today.  Almost none of these sources defang their content.  I am hoping that we can continue this trend, and leave defanging to the applications that process / store / retrieve / view STIX-CybOX content.
 
Again, if we want one and only one way to defang content, I can get along with that.  I would prefer that we do not encourage or allow defanging.  I just want everyone to be aware of the burden that comes with defanged content, especially when it allows for multiple methods of defanging.
 
Thanks,
 
Alex Foley
 
From: Patrick Maroney [mailto:Pmaroney@Specere.org] 
Sent: Wednesday, February 24, 2016 6:37 PM
To: Terry MacDonald; Allan Thomson; Foley, Alexander - GIS
Cc: Jordan, Bret; Jason Keirstead; Paul Patrick; Wunder, John A.; cti@lists.oasis-open.org
Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation
 
I suspect that semantics may be getting in our way (once again ;-).   As stated previously, I fully support the concept of passing "the truth" on the wire.  However,  I'm also arguing that we need a normative standard for how we package this "Live Ammo" so we can and safely handle it as it is processed along potentially diverse paths through the CTI Ecosystem on it's way to the actual endpoints that need to understand "the truth".  Our "CTI Ecosystem Vision" ultimately wants all of these security/operational infrastructure elements to produce and consume STIX.  Therefore, STIX content does not independently exist outside of "The System", it is holistically integrated within.
 
Along with malicious Binaries, there are a number of real world 'problematic' scenarios for text encoded "Live Ammo" that exist outside of the context of the "Human UI".  As with the examples provided earlier in this thread, the following all reflect directly observed scenarios* (with some literary embellishment to protect the innocent and inject humor):
 
* Note - For those who will rightfully argue that "Geez Pat, the TAXII Traffic will be SSL/TLS 'Encrypted on the Wire', so these aren't valid scenarios": You are absolutely correct...unless the traffic passes through transparent Secure Proxies (e.g., HTTPS,  SMTP/TLS ) that decrypt all traffic passing through one's secure infrastructure for deep content inspection.  
 
Even if decryption and deep inspection of all content by proxies is not part of your use cases, STIX packages ultimately need to be decrypted and decomposed.  The malicious content then needs to be processed and potentially transferred to a diverse set of security/operational infrastructure elements that can understand and process that specific content type.  
 
 
(1) Automated Process scanning external incoming mail (using a "Secret-Squirrel" Yara pattern for detecting malicious APT payloads) detects a high probability threat in an attachment in an email to 3,000 employees. The mail gateway receives instructions from the Yara Server to Quarantine the message for 24 Hours** and to send a copy to the Automated Analysis Server/Sandbox.  
 
** Note: Over the strenuous objections of Mail Operations Team, the APT Security Operations Team was authorized to take the Draconian measures of indefinitely Quarantining all suspicious emails received at the external perimeters  (including the audacity to actually delay Aunt Millie's communiques to her nephew 'Ralphie' , the CEO's Executive Assistant).  However, after the prior disruption of Aunt Millie's email:  "You've just got to see this video of 'Fluffy the Cat 'playing poker!!!!!!!" , the APT Security Operations Team was forced to compromise and allow automatic delivery of all messages within 24 Hours.  
 
A mail-gateway in the path (independently managed by the Mail Operations Team) has a public domain signature that detects that the forwarded message header/body contains "BadGuysRUs@BadGuys.Com" (which it's latest subscription update from 5 minutes ago says is "bad") and quietly blocks the transmission of the message with the malicious payload to the automated analysis server.  The automated workflow designed to ensure all suspicious emails are analyzed within 24 hours is never triggered.  
 
After 24 Hours,  3,000 Employees receive the new 0Day Exploit. Yada..Yada..Yada...send check to NameYourFavoriteSecurityCompanyLikeMandiant.
 
 
(2) CSIRT Analyst in TimeZone "X" submits a STIX incident report to an external agency (via Web) at 4:00 PM.  NameYourFavoriteSecurityCompanyLikeMandiant provides a 7x24 APT Detection Deep Content inspection perimeter monitoring service and detects "BadGuys.Com" in the body of the report and triggers a "Danger Will Robinson!!! Confirmed APT Threat  Detected!!!" Alert Notification to the Security Operations Team.   Security Operations Team in TimeZone "Y" receives the "All Hands to Battle Stations" alert at 3AM.  Eventually it is discovered that a false positive was triggered...but relations between the CSIRT, the tired and cranky Security Operations team who were rousted out of bed, and NameYourFavoriteSecurityCompanyLikeMandiant are even further degraded.
 
 
(3) Malicious APT artifact is submitted to SharePoint Server responsible for managing CSIRT and Security Operations Team Workflows. In a 1 in a Million fluke, the requisite COTS AntiVirus Engine deployed by policy on every operational server actually detects the APT Malware and quarantines it (deleting it from the SharePoint Server document library that holds "To Be Processed" items that in turn triggers the automated submission to the Malware Analysis Sandbox Work Flow.   Yada..Yada..Yada...3,000 Employees receive and execute a new 0Day Exploit. ...send check to NameYourFavoriteSecurityCompanyLikeMandiant.
 
 
(4) CSIRT Analyst creates/updates a ticket with new actionable IOCs and COAs. The Ticketing system sends a STIX package with the updated IOC information via Email to multiple distributed Operational Mitigation systems/processes.  Malicious artifacts are detected and delivery is blocked.  One or more COAs are not delivered 
Yada..Yada..Yada......send check to NameYourFavoriteSecurityCompanyLikeMandiant.
 
 
Note:
 
I'm reaching out to some of the communities who have developed "de-fanging" conventions to get permission to incorporate same into a draft CTI TC proposal.  Again, I argue that even a standalone CTI TC Normative Specification for "de-fanging" provides better outcomes for all,  independent of any decisions on what we do about "Truth on the Wire".  I'll do the work put a proposal in front of the CTI TC.
 
 
Patrick Maroney
Office:  (856)983-0001
Cell:      (609)841-5104
 
<image001.png>
 
President
Integrated Networking Technologies, Inc.
PO Box 569
Marlton, NJ 08053
 
From: "cti@lists.oasis-open.org" <cti@lists.oasis-open.org> on behalf of Terry MacDonald <terry@soltra.com>
Date: Wednesday, February 24, 2016 at 4:31 PM
To: Allan Thomson <athomson@lgscout.com>, Alexander Foley <alexander.foley@BANKOFAMERICA.COM>
Cc: Bret Jordan <bret.jordan@bluecoat.com>, Jason Keirstead <Jason.Keirstead@ca.ibm.com>, Paul Patrick <ppatrick@isightpartners.com>, John Wunder <jwunder@mitre.org>, "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: RE: [cti] CybOX Datatype Refactoring/Deprecation
 
Hi Allan,
 
Your log viewer example is exactly (as you note) where it is up to the tool to do the defanging. It should be the responsibility of the tool doing the syslog export to allow you to defanging at the point of export. 
 
My concern with this is that we then need to document which fields can be defanged and how they are marked and under what circumstances that it valid. We then also push this testing requirement onto vendors, and onto certification/testing bodies when ensuring compliance.
 
I’m not sure we should building all these defanging requirements into the specification when STIX is designed to get the intelligence from one Threat Intel location replicated in another Threat Intel location. Defanging only comes into play when the information is displayed to the user – and that is not the responsibility of the STIX standard. Most Threat intel toolswill be using STIX as one of the many ways they can ingest or export data. It is going to be a single module in amongst the other modules they can use. Those Threat Intel solutions will store the data they receive in their own proprietary format, and then reverse the process when exporting the data. Forcing defanging only on the STIX components of that does not make sense.
 
Instead I would propose that we provide some implementation guidance about defanging as part of our CTI Interoperability recommendations. This would be one of the guides that the CTI would provide to vendors to aid them in producing a ‘standard’ implementation.
 
But I don’t believe it makes sense to build in human protections into a specification developed specifically for machine-2-machine specification.
 
Cheers
 
Terry MacDonald
Senior STIX Subject Matter Expert
SOLTRA | An FS-ISAC and DTCC Company
+61 (407) 203 206 | terry@soltra.com
 
 
From: cti@lists.oasis-open.org [mailto:cti@lists.oasis-open.org] On Behalf Of Allan Thomson
Sent: Thursday, 25 February 2016 6:38 AM
To: Foley, Alexander - GIS <alexander.foley@BANKOFAMERICA.COM>
Cc: Jordan, Bret <bret.jordan@bluecoat.com>; Jason Keirstead <Jason.Keirstead@ca.ibm.com>; Paul Patrick <ppatrick@isightpartners.com>; Wunder, John A. <jwunder@mitre.org>; cti@lists.oasis-open.org
Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation
 
Alex - My preference is to have STIX specification focus on describing what the content includes and allow providers/consumers decide whether they want defanged or not. I don’t believe the STIX spec should mandate this one way or another.
 
So my suggestion was to facilitate the following:
 
1) Provide an options field in STIX content that describes whether information is defanged or not. Could be communicated at a package level. 
 
- This provides vendors and systems to decide whether they are providing defanged only or both in their intel.
- This might also fit into other optional content conversation that may take place such as language options that may or may not be supported by the provider.
- Machines can be programmed to do both easily and this will provide greater flexibility to implementers on what they prefer for different use cases
 
2) Provide a mechanism in TAXII discovery/sharing exchange to a) determine if both options are supported or not b) request what is preferred
 
I know this might seem more complicated than just having one format but having flexibility in the content seems useful.
 
Example:
 
One common thing that occurs in our products is log files that are capturing ingested or exported information. Typically the code that writes those log files is very simplistic, high-speed is important and does not convert most of the content if it does not have to. Generating syslog messages is an example where you may want the original content so that the receiver can act on the content but if a user is looking at the content in a generic syslog server viewer then they absolutely don’t want the url to be active. There have been occasions where log files get examined by users or developers (or in a user interface) and those users/developers accidentally click or copy that information without defanging it. The result is not ideal.
 
You could argue that this is user error and the log file or syslog viewer should have created the content defanged but its often not done.
 
Therefore a pragmatic approach is to distribute content defanged and if original content is required to act on it then the “machine” can easily convert it back to the original content.
 
My 10cents.
 
Allan
 
On Feb 24, 2016, at 9:49 AM, Foley, Alexander - GIS <alexander.foley@BANKOFAMERICA.COM> wrote:
 
Paul / Allan,
 
Perhaps you could provide a more specific justification of what could go wrong if we’re moving fanged content around machine-to-machine?  I’m sure I might be glossing over some danger, but given that STIX / CybOX content is shared as fanged today, it’s harder for me to explain the benefit of having the option of de-fanged content when systems processing this information have to assume at least some of it will be fanged.
 
Thanks,
 
Alex Foley
 
From: Jordan, Bret [mailto:bret.jordan@bluecoat.com] 
Sent: Wednesday, February 24, 2016 12:38 PM
To: Jason Keirstead
Cc: Paul Patrick; Foley, Alexander - GIS; Wunder, John A.; cti@lists.oasis-open.org
Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation
 
I think we all agree that defanging is important.  But I am with Jason and others, I do not believe the "STIX / CybOX" content should be defanged.  I believe it should be done in the tools that present information to humans. 

 

Thanks,
 
Bret
 
 
 
Bret Jordan CISSP
Director of Security Architecture and Standards | Office of the CTO
Blue Coat Systems
PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 
 
On Feb 24, 2016, at 09:07, Jason Keirstead <Jason.Keirstead@ca.ibm.com> wrote:
 
This whole conversation to me seems to again go against our base principle discussed at F2F that STIX is going to be targetted at machines. We don't know what those machines want to do with the data. IMO it should be raw as possible. Any defanging would be done at the tool end before showing  a URL to the user or having them download the binary.

Sent from IBM Verse




Paul Patrick --- Re: [cti] CybOX Datatype Refactoring/Deprecation --- 
 
From:
"Paul Patrick" <ppatrick@isightpartners.com>
To:
"Foley, Alexander - GIS" <alexander.foley@bankofamerica.com>, "Wunder, John A." <jwunder@mitre.org>, cti@lists.oasis-open.org
Date:
Tue, Feb 23, 2016 6:35 PM
Subject:
Re: [cti] CybOX Datatype Refactoring/Deprecation

 
Alex,
 
I don’t believe we need multiple different approaches but I do think we need to be able to support the ability to have defanged data and thus wouldn’t we need a means to tell the receiving end whether the value is or isn’t defanged? 
 
Also don’t different types of data need different defanging approaches.  For example,  defanging a binary and a malicious web page content could be done in probably a common approach.  But you probably wouldn’t want to use that same approach for say malicious URLs.  So wouldn’t we need to defined the ‘standard’ defanging mechanism for each different type of data?
 
Perhaps I’m missing something and if so perhaps a concrete example might be helpful so that we can see what is specifically being proposed.
 
So I’m on-board with one approach for each of the different types of data.  I also able to get on-board with having only one property, which is the algorithm that indicates the mechanism and that the value is defanged as proposed in Ivan’s proposal.
 
Like you Alex, I don’t want to be a roadblock either and definitely open to figuring out a solution that works for us all.
 
 
Paul
 
 
From: "Foley, Alexander - GIS" <alexander.foley@bankofamerica.com>
Date: Tuesday, February 23, 2016 at 6:59 PM
To: Paul Patrick <ppatrick@isightpartners.com>, "Wunder, John A." <jwunder@mitre.org>, "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: RE: [cti] CybOX Datatype Refactoring/Deprecation
 
Paul,
 
I’m certainly sympathetic to what you’re talking about.  We too don’t want to enable people to accidentally detonate malware or click links.  However, I do think there’s a distinction between malware samples (which might be contained in a STIX package but are best thought of as an “attachment” in my mind) and the metadata captured from a piece of malware.  For the purposes of our discussion here, I think we’re talking about metadata rather than the actual malware.
 
In this case, I could get along with the #2 approach where we have one and absolutely one approach to defanging.  On the other hand, I am also sympathetic to the performance concerns associating with refanging in real-time analysis systems.  Furthermore, I fear that if we even allow the option of defanging, people will do it in different ways which will introduce downstream problems.  If you’ll forgive me for being simple, I also worry about how effective edge protections are if content is defanged and it can’t be effectively refanged for blocking.
 
But again, if this is a showstopper for members of the community, I don’t mean to be a roadblock.  I just want to provide some real world examples of how defanging is both a blessing and a curse.
 
Thanks,
 
Alex Foley
 
From:cti@lists.oasis-open.org [mailto:cti@lists.oasis-open.org] On Behalf Of Paul Patrick
Sent: Tuesday, February 23, 2016 4:43 PM
To: Wunder, John A.; cti@lists.oasis-open.org
Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation
 
We tend to protect things as often things go on a bus and arrive a different systems including storage systems where samples are kept.  We don’t care to keep live “weapons” sitting around in these types of systems.  So we protect at the edge instead of making assumptions about where it might go.
 
From: <cti@lists.oasis-open.org> on behalf of "Wunder, John A." <jwunder@mitre.org>
Date: Tuesday, February 23, 2016 at 2:26 PM
To: "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation
 
Can you explain more about why it’s necessary for the machine-to-machine use case?
 
From: <cti@lists.oasis-open.org> on behalf of Paul Patrick <ppatrick@isightpartners.com>
Date: Tuesday, February 23, 2016 at 2:13 PM
To: Ivan Kirillov <ikirillov@mitre.org>, "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation
 
As someone that has to actually exchange malware in a machine-to-machine scenario, I can tell you we do need to support defanging/refanging.  With that in mind, I don’t think that #1 is really a option.   What I’m trying to decide is whether option #2 would be acceptable.
 
 
Paul
 
From: <cti@lists.oasis-open.org> on behalf of Ivan Kirillov <ikirillov@mitre.org>
Date: Tuesday, February 23, 2016 at 10:29 AM
To: "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation
 
Great discussion here! Personally, I’m with Alex and others in the school of thought that UI/presentation layer fields should be kept out of the standards. 
 
So I think we have a few options:
1.       Remove ALL defanging/refanging and metadata from of CybOX Object fields. The main impact of this is the simplification of Object field specification in instance data, e.g., {“file_name”:”xyz.dll”} instead of {“file_name”:{“value”:”xyz.dll”}}. The downside is that we lose the ability to specify these properties on Object fields.
2.       Orient around defining a common defanging/refanging process and use this as the “one way” of doing it in the CTI standards. We may be able to do this without having to directly specify this on the Object fields.
3.       Leave defanging/refanging open for users/implementers to specify at the Object field level; this is essentially the same as the existing (CybOX 2.1) approach.
My own preference is option 1, as this type of simplification (i.e. removing extra layers between object fields and their values) is what I had in mind when doing some of the initial thinking around CybOX 3.0.
 
Regards,
Ivan
 
From: <cti@lists.oasis-open.org> on behalf of "Coderre, Robert" <rcoderre@verisign.com>
Date: Tuesday, February 23, 2016 at 7:06 AM
To: John Wunder <jwunder@mitre.org>
Cc: "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation
 
Very good points are raised by all. Thanks Alex for showing me the light.  Defanging is important for data presentation to humans, but not so much for machine to machine.  I went back and realized that we defang in narrative form only (i.e., descriptions, analysis, etc.).  For data level fundamentals, it's stored in native form (IPv4, URL, etc.).
 
It's on the vendor to defang for human consumption and presentation but not in a machine to machine context.

-- 
Rob


On Feb 23, 2016, at 3:01 PM, Wunder, John A. <jwunder@mitre.org> wrote:

Yeah Alex makes some really good points and I agree. Defanging is a human-facing concern and our machine-to-machine languages shouldn’t need to deal with it.
 
Whatever gets presented to the user can be defanged by the tool, and the backend languages can just pass the actual values because humans won't be seeing them.
 
John
 
On Feb 23, 2016, at 8:38 AM, Mark Davidson <mdavidson@soltra.com> wrote:
 
I agree with Alex and Terry completely.
 
At the end of the day, what people currently do on e.g., mailing lists is a good indication of the use cases or requirements that need to be supported. IMO they are not a good indication of how those use cases or requirements should be implemented. Current practice highlights problems that need to be solved; but it is up to us to come up with the solutions.
 
I do not see how a single defanging solution would make STIX/CybOX untenable for whole swaths of players. IMO, the opposite is true - a single defanging solution lowers the effort required to create interoperable software and therefore helps increase the number of solutions that are STIX/TAXII complaint. The biggest reason people aren’t using STIX/TAXII today is because their tools don’t support it. Their tools support it because it's complicated to implement. Simplification is paramount if we care to increase adoption.
 
Personally, I am in favor of having only one (or zero!) defanging solutions for the reason stated above. IMO the big win will be when two things that say “STIX/TAXII” on the label just work together. If we get that, analysts will not care what defanging algorithm STIX uses – that will become a technical detail they no longer care about.
 
Thank you.
-Mark
 
From: <cti@lists.oasis-open.org> on behalf of Terry MacDonald <terry@soltra.com>
Date: Tuesday, February 23, 2016 at 4:17 AM
To: "Jordan, Bret" <bret.jordan@bluecoat.com>, "Foley, Alexander - GIS" <alexander.foley@bankofamerica.com>
Cc: "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: RE: [cti] CybOX Datatype Refactoring/Deprecation
 
Yeah. Upon reading Alex’s post I’d have to agree. It should really be up to the UI to do any defanging of links as necessary. Defanging is just a result of not using the correct tools in the first place…
 
Cheers
 
Terry MacDonald
Senior STIX Subject Matter Expert
SOLTRA | An FS-ISAC and DTCC Company
+61 (407) 203 206 | terry@soltra.com
 
 
From: Jordan, Bret [mailto:bret.jordan@bluecoat.com] 
Sent: Tuesday, 23 February 2016 4:58 PM
To: Foley, Alexander - GIS <alexander.foley@bankofamerica.com>
Cc: Coderre, Robert <rcoderre@verisign.com>; Terry MacDonald <terry@soltra.com>; Barnum, Sean D. <sbarnum@mitre.org>; Kirillov, Ivan A. <ikirillov@mitre.org>; cti@lists.oasis-open.org
Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation
 
Really good points Alex, as always.  

 

Thanks,
 
Bret
 
 
 
Bret Jordan CISSP
Director of Security Architecture and Standards | Office of the CTO
Blue Coat Systems
PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 
 
On Feb 22, 2016, at 19:38, Foley, Alexander - GIS <alexander.foley@bankofamerica.com> wrote:
 
I’m on the other side of the fence here.  I don’t think we should be defanging at all.  It’s a flipping mess.
 
While I totally agree that the standard is what Terry and Rob say (with slight variations, as Outlook doesn’t require hxxp or brackets around every [.] to defang a link), we’ve spent the last six months figuring out the caveats to the standard.  For example, take our domain and URI regexes:
 
·        Domains (did you know .homedepot is a gTLD?  Neither did I until we had to write a regex to find it)
·        URIs containing domains with protocols
·        URIs containing domains without protocols
·        URIs containing IP addresses with protocols
·        URIs containing IP addresses without protocols
 
I promise you that the moment you tell a developer we need to start parsing URLs that start with poop://, woof://, meow:// or purr:// you will stop seeing the value in enabling defanging.
 
I seriously think we should finally decide that our core value is to have a machine-to-machine language and stop prioritizing human readable use cases, or to decide that we want a human readable language and stop prioritizing machine-to-machine use cases.  If we would have decided this during the timestamp discussions we’d currently have UNIX timestamps set to GMT format instead of using ISO timestamps to indicate a timezone that will never be changed.
 
Alex
 
From: cti@lists.oasis-open.org [mailto:cti@lists.oasis-open.org] On Behalf Of Coderre, Robert
Sent: Monday, February 22, 2016 7:51 PM
To: Terry MacDonald
Cc: Barnum, Sean D.; Kirillov, Ivan A.; cti@lists.oasis-open.org
Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation
 
This is pretty standard for our community. 

--
Rob


On Feb 22, 2016, at 4:49 PM, Terry MacDonald <terry@soltra.com> wrote:

Hi Sean,
 
I think this is an area where the benefits of the STIX/TAXII/CybOX ecosystem will outweigh the small chance that an organization will be majorly impacted by the selection of a single ‘most common’ way of defanging. This is DEFINITELY a case of the 80/20 rule applying.
 
From the various trusted lists I’m on, the most common defanging standard is:
 
 
Cheers
 
Terry MacDonald
Senior STIX Subject Matter Expert
SOLTRA | An FS-ISAC and DTCC Company
+61 (407) 203 206 | terry@soltra.com
 
 
From: cti@lists.oasis-open.org [mailto:cti@lists.oasis-open.org] On Behalf Of Barnum, Sean D.
Sent: Tuesday, 23 February 2016 4:29 AM
To: Kirillov, Ivan A. <ikirillov@mitre.org>; cti@lists.oasis-open.org
Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation
 
If we attempted to force such a decision then yes it would require less fields.
I would worry about taking such an approach however given that the real-world community of practitioners do NOT have such a "ONE standard way of performing defanging/refanging on all fields” and it is not for lack of trying. My understanding is that many different parties over the years have tried to drive towards a single algorithm and have not been successful. Different communities/players have strong feelings that certain approaches offer necessary capabilities that others do not.
I know that Pat Maroney has been part of these sorts of discussion within DSIE and elsewhere so I would encourage him to comment here.
 
The danger of us thinking we can decide and enforce the ONE way of doing something even though the communities that we are attempting to support clearly do not yet agree is that we could be making STIX/CybOX untenable for whole swaths of players.  I would think this issue potentially very much akin to the whole XML vs JSON issue.
I think it makes total sense to work to identify ONE default approach but don’t think we can then simply disallow all other approaches. I think 2 or 3 fields is an acceptable price to pay for something like this.
 
sean
 
From: Steve Cell <ikirillov@mitre.org>
Date: Monday, February 22, 2016 at 12:11 PM
To: "Barnum, Sean D." <sbarnum@mitre.org>, "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation
 
Ah, I see – that makes sense. So I suppose the corollary to this is, what if we specified ONE standard way of performing defanging/refanging on all fields, such as by base64-encoding the data? Seems like this would make the specification simpler, and consumers would only need to understand/implement one refanging method.
 
Regards,
Ivan
 
From: Sean Barnum <sbarnum@mitre.org>
Date: Monday, February 22, 2016 at 10:02 AM
To: Ivan Kirillov <ikirillov@mitre.org>, "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation
 
>[Ivan] That makes sense, though I do wonder if in such a case Party A will indeed provide refanging information if they do not have certainty about the defanging method used. >Also, the refanging field as designed before was meant to be a machine-parseable transform (e.g, a regex); is there still the expectation that it is actually used in this way (i.e., >that consumers would actually machine parse it and apply it to the data) in the wild?
 
I apologize if my comment on refanging was confusing. It was not intended to be tied directly to the case of no defanging algorithm provided.
It was just meant in general having the ability to specify a refanging algorithm. I would certainly think that having an explicit way to automatically refang content (within a GUI for example) would still be very useful. If everyone was using the same defanging algorithm then the refanging could likely be hardcoded and would not need a separate field but I do not believe that this is the reality we live in or are likely to live in for a while. Some communities or organizations use diverse defanging approaches and the only way to refang such content automatically would require explicit specification of a refanging algorithm.
 
</my-two-cents>
 
sean
 
From: Steve Cell <ikirillov@mitre.org>
Date: Monday, February 22, 2016 at 11:54 AM
To: "Barnum, Sean D." <sbarnum@mitre.org>, "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation
 
 
  1.  
    1. Could we limit the properties associated with obfuscation and defanging/refanging to the following?
      1. obfuscation_algorithm_ref (URI): a reference to a description of the algorithm used to obfuscate the object property
      2. defanging_algorithm_ref (URI): a reference to a description of the algorithm used to defang the object property
 
[sean]So for these I assume you are suggesting that the presence of these fields implicitly means that the field is obfuscated or defanged thereby negating the need for the explicit field. The only potential issue I see with that is when the field is obfuscated or defanged but no algorithm is provided or available. 
 
[Ivan] Yes, that’s correct – John and I thought that we could likely remove the is_obfuscated/is_defanged fields, since the presence of the _ref fields is enough to state that a field is obfuscated and/or defanged. Do we as the community feel that not specifying a reference to the obfuscation/defanging algorithm used would be a common occurrence? 
 
My understanding of a common use case is that Party A passes along data to Party B that was not originally produced by Party A where they have determined the data is obfuscated or defanged and wish to convey that to Party B but do not have certainty on the exact algorithm used.
Either way, I would think you will also need to include refanging
 
[Ivan] That makes sense, though I do wonder if in such a case Party A will indeed provide refanging information if they do not have certainty about the defanging method used. Also, the refanging field as designed before was meant to be a machine-parseable transform (e.g, a regex); is there still the expectation that it is actually used in this way (i.e., that consumers would actually machine parse it and apply it to the data) in the wild?
 
Regards,
Ivan
 
From: Sean Barnum <sbarnum@mitre.org>
Date: Tuesday, February 16, 2016 at 7:43 AM
To: Ivan Kirillov <ikirillov@mitre.org>, "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation
 
Comments inline
 
From: <cti@lists.oasis-open.org> on behalf of Steve Cell <ikirillov@mitre.org>
Date: Monday, February 15, 2016 at 10:45 AM
To: "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation
 
Are there any additional thoughts on this topic? Just trying to see if have any consensus on:
  1. This overall approach, i.e., supporting specific metadata properties as part of CybOX Object fields (implemented via JSON Objects)
  2. The specific set of metadata properties that we should support
    1. Does it make sense to include an “is_random” property for assertions of randomness in the data of a particular field?
 
[sean]I have no dog in this hunt but know that this field came from request came from digital forensics use cases where they were fully aware that this was an observation and not a pattern. We should ask Eoghan for his thoughts on this.
  1.  
    1. Could we limit the properties associated with obfuscation and defanging/refanging to the following?
      1. obfuscation_algorithm_ref (URI): a reference to a description of the algorithm used to obfuscate the object property
      2. defanging_algorithm_ref (URI): a reference to a description of the algorithm used to defang the object property
 
[sean]So for these I assume you are suggesting that the presence of these fields implicitly means that the field is obfuscated or defanged thereby negating the need for the explicit field. The only potential issue I see with that is when the field is obfuscated or defanged but no algorithm is provided or available. My understanding of a common use case is that Party A passes along data to Party B that was not originally produced by Party A where they have determined the data is obfuscated or defanged and wish to convey that to Party B but do not have certainty on the exact algorithm used.
Either way, I would think you will also need to include refanging
 
Regards,
Ivan
 
From: Terry MacDonald <terry@soltra.com>
Date: Tuesday, February 9, 2016 at 4:49 PM
To: Ivan Kirillov <ikirillov@mitre.org>, "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: RE: [cti] CybOX Datatype Refactoring/Deprecation
 
What about using the relationships to handle the levels of obfuscation? That would allow description of obfuscated and encrypted objects within other obfuscated objects.
 
In other words to describe a PDF containing obfuscated _javascript_ that decodes and extracts a windows executable from within the PDF we could do something like this:
 
 
<image001.jpg>
 
Would that sort of thing work? BTW the relationship thing seems pretty darn powerful when used like this….
 
Cheers
 
Terry MacDonald
Senior STIX Subject Matter Expert
SOLTRA | An FS-ISAC and DTCC Company
+61 (407) 203 206 | terry@soltra.com
 
 
From:cti@lists.oasis-open.org [mailto:cti@lists.oasis-open.org] On Behalf Of Kirillov, Ivan A.
Sent: Wednesday, 10 February 2016 8:16 AM
To: cti@lists.oasis-open.org
Subject: Re: [cti] CybOX Datatype Refactoring/Deprecation
 
>> As much as I would like fields to just be the value, I agree that we probably need this layer of indirection to support these extra properties.
 
I was also hoping for Observations/Objects to just hold the value, but some of these fields do seem to be necessary. I think the only other way to do this would be via relationships, but then you’d have to assign an ID to each field, and that would just get messy quickly without any real benefit.
 
>> We’ll need to decide what that means for patterning, if anything, by the way.
 
Yup, agreed. My initial thinking is that these properties don’t (or shouldn’t) have any significance for patterning. 
 
>>Given that we’re talking about actual observations here (instances), is “appears_random” appropriate? That seems to be a statement about a pattern of behavior over several observations rather than a single observation.
 
I’ve wondered the same – this does seem to be an assertion that one wouldn’t typically make based on a single data point. Also, it seems like you’d want to be able to specify some level of confidence and perhaps other properties with this as well, so I’m also wondering if this belongs here or perhaps in some separate structure (which could also be used for related observations such as entropy, etc.)?
 
>>Can we combine the “obfuscation” fields into some single “obfuscation_algorithm”, representing a URI or something? Then if the field is there, it’s obfuscated using that. If it’s not, then it’s not obfuscated.
 
Yeah, that makes good sense to me. I think having the “is_defanged” boolean isn’t strictly necessary, as you can infer that something is obfuscated if the corresponding “obfuscation_algorithm” field is populated. Having a URI in there seems reasonable.
 
>> Given that we’re now talking about machine-to-machine transmissions, is defanging still important? Can’t the raw data be exchanged as it is and then let tools to present it to users defanged however they want? It seems weird to me to transmit it defanged if it’s M2M (vs. a PDF that people might accidentally click).
 
I had the same question when we were first implementing defanging/fanging; it does seem like something that should be done at the data ingest/GUI layer rather than at the data specification layer. It’s definitely odd to support it if there is no expectation that humans will actually look at the data.
 
>> If we do need to include it, can we similarly collapse these attributes into one “defanging_algorithm” URI? They seem very redundant right now.
 
I’m also onboard with this. So we’d get something like:
  • obfuscation_algorithm_ref (URI): a reference to a description of the algorithm used to obfuscate the object property
  • defanging_algorithm_ref (URI): a reference to a description of the algorithm used to defang the object property
  • refanging_transform (string): an automated transform that can be applied to the object property in order to refang it to its original format
Regards,
Ivan
 
<-Your data has been truncated.
 

This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message.
 

This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message.



[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]