RE: Proposal for simple CybOX objects

["Kemp, David P" <dpkemp@nsa.gov>]

I'm not sure I understand the distinction.   In order to read any JSON document, every byte in the document must be "fully scanned" down to the closing brace, and the time that it takes to do the scan depends on the size of the document.  An application that counts IP addresses can build a flat symbol table of IP addresses (or of all 10 CybOX object types) in linear time either as the file is being read or after it is in memory as, e.g., a Javascript or Python datastructure.

I understand CybOX and STIX to be interface standards - serialization formats that allow data to be exchanged between applications.  They are not intended to be API (in-memory) specifications, are they?  An API for counting IP addresses would want them all collected into one data structure.  But an ICD would most naturally encode them within the object where they occur.   Example 9.3.0.2, Complex MIME Message looks like a pointless slicing and dicing that would be more difficult for a receiving application to process than either the raw MIME message or a set of CybOX objects structured as they appear in the raw message.

A graph would show, for example, all of the IP addresses that occurred in all of the email messages that flowed through a sensor, discovering how many addresses a sender used.   Is one CybOX document really intended to transfer such a complete graph between applications, or is it intended to define in a standard way all of the building blocks (observables) from which such a graph could be constructed.   I thought its purpose was the latter.

Dave


-----Original Message-----
From: Mates, Jeffrey CIV DC3\DCCI [mailto:Jeffrey.Mates@dc3.mil]
Sent: Wednesday, August 17, 2016 1:30 PM
To: Kemp, David P <dpkemp@nsa.gov>; 'Kirillov, Ivan A.' <ikirillov@mitre.org>; cti-cybox@lists.oasis-open.org
Subject: RE: Proposal for simple CybOX objects

Allowing embedded objects was one of the biggest problems in the previous version of CybOX and if possible I really want to avoid this. Allowing mixed typing destroys the graph model that supports both STIX and CybOX information while also making relatively simple tasks, such as counting the number of distinct IP addresses that appear in your message, difficult as a full traversal will always be required for all queries.

Consider a simple query against a CybOX document such as: Give me all of the IP addresses that appear in it.

Without nesting all top level objects can be checked and any IP address objects will be returned.

With nesting EVERY object must be fully scanned to see if an IP appears anywhere within any of its substructures.  For example a network connection to an FQDN to an IP.

As such, I am strongly in favor of keeping it flat.

Jeffrey Mates, Civ DC3/DCCI
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Computer Scientist
Defense Cyber Crime Institute
jeffrey.mates@dc3.mil
410-694-4335


-----Original Message-----
From: cti-cybox@lists.oasis-open.org [mailto:cti-cybox@lists.oasis-open.org] On Behalf Of Kemp, David P
Sent: Wednesday, August 17, 2016 12:49 PM
To: 'Kirillov, Ivan A.'; cti-cybox@lists.oasis-open.org
Subject: [Non-DoD Source] [cti-cybox] Proposal for simple CybOX objects

I have added a comment on the Network Objects file requesting that subcommittee adopt a design goal of defining all CybOX objects as simple objects.  In practice, this means that properties that are defined indirectly (as references) have a direct (non-referenced) base property.  If there are use cases that require references, a  _ref property can be defined in addition to the direct property.  But in most cases, _refs should not be needed and the object can be used without having to wrap it in a cybox-container.

For example in ipv4-address-object,  type and value are defined as a direct properties:

{
    "type": "ipv4-address-object",
    "value": "1.2.3.4"
}

but resolves_to_refs and belongs_to_refs are defined indirectly.

The proposal is to define equivalent direct properties that can be used in simple objects, e.g.:

{
    "type": "ipv4-address-object",
    "value": "1.2.3.4",
    "resolves_to": [{
        "type": "mac-address-object",
        "value": "01:23:45:67:89:ab"
    }]
}

Fortunately mac-address-object is defined as a simple object so ipv4-address-object can use it without having to wrap it.   For simplicity and consistency, all other CybOX objects (including ipv4-address-object) should also be usable without wrapping them in cybox-containers.

Thoughts?

Dave Kemp



-----Original Message-----
From: cti-cybox@lists.oasis-open.org [mailto:cti-cybox@lists.oasis-open.org] On Behalf Of Kirillov, Ivan A.
Sent: Wednesday, August 17, 2016 10:49 AM
To: cti-cybox@lists.oasis-open.org
Subject: Re: [cti-cybox] Recent CybOX Changes

I just wanted to let everyone know where we stand on these issues. Trey and I had a discussion on this topic and decided to make the following changes:



·         file_path in File Object/file-system-properties-type was changed to a string. As others had pointed out, there was no benefit to the delimited approach and it had the side effect of making patterns more complex.

·         We’ve removed the string-with-encoding-type and reverted all uses back to string. We will similarly go back to our old approach of capturing observed encoding using the flattened “_enc” and “_b64” fields – for an example of this please see the file_name and corresponding fields [1]. We’ve also reverted our old section in CybOX Core on this topic [2] – please review and comment. Also, let us know which other fields in the CybOX Objects we should be adding this for.



[1] https://docs.google.com/document/d/1DdS-NrVTjGJ3wvCJ7dbSlhYeiaWS6G6dOXu2F3POpUs/edit#heading=h.8qkuomuewogh <https://docs.google.com/document/d/1DdS-NrVTjGJ3wvCJ7dbSlhYeiaWS6G6dOXu2F3POpUs/edit#heading=h.8qkuomuewogh>

[2] https://docs.google.com/document/d/1PSGv6Uvo3YyrK354cH0cvdn7gGedbhYJkgNVzwW9E6A/edit?pref=2&pli=1#heading=h.mqdyq1mo042h <https://docs.google.com/document/d/1PSGv6Uvo3YyrK354cH0cvdn7gGedbhYJkgNVzwW9E6A/edit?pref=2&pli=1#heading=h.mqdyq1mo042h>



Regards,

Ivan



From: Jason Keirstead <Jason.Keirstead@ca.ibm.com>
Date: Tuesday, August 2, 2016 at 3:28 PM
To: Allan Thomson <athomson@lookingglasscyber.com>
Cc: John-Mark Gurney <jmg@newcontext.com>, "Mates, Jeffrey CIV DC3\\DCCI" <Jeffrey.Mates@dc3.mil>, Ivan Kirillov <ikirillov@mitre.org>, John Wunder <jwunder@mitre.org>, Greg Back <gback@mitre.org>, "cti-cybox@lists.oasis-open.org" <cti-cybox@lists.oasis-open.org>
Subject: Re: [cti-cybox] Recent CybOX Changes



I agree, I am also unsure what the purpose of this field is... are there actual threat analytics use cases where knowing the encoding of a piece of text conveys useful information? Was this requested by a specific user group?

-
Jason Keirstead
STSM, Product Architect, Security Intelligence, IBM Security Systems www.ibm.com/security | www.securityintelligence.com

Without data, all you are is just another person with an opinion - Unknown


nactive hide details for Allan Thomson ---08/02/2016 03:47:31 PM---That’sAllan Thomson ---08/02/2016 03:47:31 PM---That’s a good clarification and it makes me wonder why this attribute needs to be conveyed at all. W

From: Allan Thomson <athomson@lookingglasscyber.com>
To: John-Mark Gurney <jmg@newcontext.com>
Cc: "Mates, Jeffrey CIV DC3\\DCCI" <Jeffrey.Mates@dc3.mil>, "'Kirillov, Ivan A.'" <ikirillov@mitre.org>, Jason Keirstead/CanEast/IBM@IBMCA, "Wunder, John A." <jwunder@mitre.org>, "Back, Greg" <gback@mitre.org>, "cti-cybox@lists.oasis-open.org" <cti-cybox@lists.oasis-open.org>
Date: 08/02/2016 03:47 PM
Subject: Re: [cti-cybox] Recent CybOX Changes Sent by: <cti-cybox@lists.oasis-open.org>

________________________________




That’s a good clarification and it makes me wonder why this attribute needs to be conveyed at all. What is a recipient expected to do with this information?

Certainly combining it into a single field makes no sense now.

Thanks for sharing this perspective. It changes my opinion of what this encoding field is attempting to convey and how it should be done.

allan

On 8/2/16, 10:47 AM, "John-Mark Gurney" <jmg@newcontext.com> wrote:

   Allan Thomson wrote this message on Tue, Aug 02, 2016 at 14:02 +0000:
   > Finally, *if* encoding must be conveyed when a field that may have different encodings possible is required. Then what is the most efficient and effective way to allow two systems to a) generate that package b) read that package efficiently and effectively.

   I want to clarify this statement..  Fields only ever contain Unicode
   data.  They never contain non-Unicode, or encoded data..  The encoding
   is a way to convey what encoding the original data had before converting
   to CybOX.

   --
   John-Mark