Allan â I donât disagree that we should be flexible enough to meet the various use cases needed by the market. However, I think the issues weâre running into here are based on our attempt to rearchitect Observed Data around an entirely different set of use cases than originally envisioned, which is why I want to make sure that whatever we end up with is coherent from both a specification perspective and for implementers/end users as well.
Regards,
Ivan
Ivan â I believe that the primary use cases where systems do capture the time window and number of instances, they *will* provide that information. There is a good intelligence use of that information being provided and more mature use of that data will naturally select providers that have it to share.
But there are also certain use cases where itâs not known and those use cases should not be precluded by the schema itself.
Regarding the objects property definition. Iâm not exactly sure how we were going to test that all observables are related to each other anyway. That is a complicated analysis if you have systems reporting both network and host based observations in the same observed data object. I doubt most systems would be testing for the connected-ness of the data to begin with.
From my perspective I suggest we let the market/products dictate how complete their data sets are rather than excluding use cases solely based on a spec.
Allan
Iâm OK with #1 as long as we understand the implications of doing this and that weâre changing the definition of Observed Data that weâve long held. E.g., are we comfortable with the fact that users can now create Observed Data instances without specifying the time window during which the data was observed and how many times it as observed during that window?
We also still havenât changed the definition of the objects property which stipulates that any Observables captured must be part of the same graph (i.e., related to each other). This is particularly problematic for use cases such as the capture of malware sandbox analysis data where you may end up with a large collection of unrelated Observables.
Regards,
Ivan
Having been the one who pushed to release 2.1 in the Spring as is, I guess I have to also agree with #1.
However, I feel #3 is ultimately (2.2?) the right answer. Remember in STIX 1.x when we conflated observed data and patterns because they âkindaâ looked the same?
I support #1.
I do not support #2, simply because of the extremely large & cascading ramifications this has for software implementors, for something so minor as the name of a JSON field not meant to be primarily consumed by humans.
-
Jason Keirstead
Lead Architect - IBM.Security
www.ibm.com/security
"Things may come to those who wait, but only the things left by those who hustle." - Unknown
From: Allan Thomson <athomson@lookingglasscyber.com>
To: "Wunder, John A." <jwunder@mitre.org>, "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Date: 08/14/2018 06:42 PM
Subject: Re: [cti] Possible Changes to Observed Data
Sent by: <cti@lists.oasis-open.org>
I support #1 -> #2 in that order.
Although I donât really see a need for #2 and therefore am primarily supportive on #1.
I agree with Johnâs perspective on getting 2.1 done but that needs to include the new malware definition that I believe was leveraging this new ârelaxedâ observed data object.
That is why I think #1 is also the best course of action.
Regards
allan
From: "cti@lists.oasis-open.org" <cti@lists.oasis-open.org> on behalf of "Wunder, John" <jwunder@mitre.org>
Date: Tuesday, August 14, 2018 at 2:38 PM
To: "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: Re: [cti] Possible Changes to Observed Data
I would say that, philosophically, we should do whichever option lets us finish STIX 2.1 as soon as possible while ensuring that itâs usable and complete (note that I didnât say âperfectâ). Weâre waiting on 2.1 in order to do internationalization, confidence, malware, and a whole pile of other things. These are critical capabilities that we canât share in a standardized way right now, and thatâs preventing people from using STIX 2. If we spend another year trying to get observed data perfect I think we risk losing support and getting the perception that we wonât finish things.
To that end, IMO the best options are #1 or #2 because they had the most support on the working calls (personally I prefer #5 but I think #1 and #2 are workable). So my suggestion would be that we do #1 or #2 and the people that prefer it should address the comments from JMG and others (see working call notes from today) to ensure we donât lose the important aspects of observed-data when used to capture operational data observed by sensors. If we can get that done by the end of the review period for WD02 letâs include it in CSD01, otherwise letâs defer to CSD02.
John
From: <cti@lists.oasis-open.org> on behalf of "Bret Jordan (CS)" <Bret_Jordan@symantec.com>
Date: Tuesday, August 14, 2018 at 5:04 PM
To: "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: [cti] Possible Changes to Observed Data
All,
There has been several discussions over the past few months about relaxing and making slight changes to Observed Data to make it useable for other use cases (Malware, Infrastructure, Incident, Intrusion Sets, etc). We have talked about it a few times on working calls and over slack and we now need to make a decision about what to do. As such we are bringing this issue to the email list for review and comment by the full TC.
Historically, the Observed Data object came to be after the great Arglebargle debate from the DC3 F2F meeting. The question at that time was, "should Cyber Observables (CybOX) be first order citizens or should they be in a wrapper". At that time, most people felt like it should be in a wrapper, even though this would mean a graph inside of a graph. So the TC created the Observed Data Object to address one specific use case, that is relating cyber observables to another STIX object, specifically an indicator, via the Sighting Relationship object. This would allow you to say, I saw this indicator, and what I saw is located in this Observed Data object. So all of the descriptions, normative text, and properties were designed to address this one specific use case.
Fast forward 24 months, some TC members are now asking for a way to capture cyber observables that are not just "observations" (in the definition that was defined for Observed Data). They are looking for a more generic container or solution so cyber observables can be documented, captured, and shared. This data may come from an analyst that reads some report, this may come from a sandbox, this may come from a URL lookup service like URLVoid, etc, but the key point is that it is not an "observation" it is just cyber observable data.
I have heard a few different options that we as a TC could take, if you have other ideas, please let us know. This is really a critical decision that the TC needs to address. The options I have heard are:
1) Relax Observed Data and some of the properties to make it useable by more use cases
2) Same as 1, but rename Observed Data to be something more appropriate
3) Created a different cyber observable container (this would mean we could have more than one, and that might cause confusion)
4) Revisit the ArgleBargle decision and look to make cyber observables first order citizens (this would get rid of the graph inside a graph design that some people have since complained about)
5) Due nothing and try and create some other cyber observable property on objects that need cyber observables that are not "observations". This could represent multiple ways of doing the same thing, something we have tried hard to avoid.
## Personal Opinion ##
From my personal perspective, it would be nice if we could address this in this first CSD, since it may represent breaking changes. This does not mean we have to get it 100% correct for CSD01, as we can tweak it and refine it more in CSD02 and CSD03. But it would be nice to at least get the major changes done for CSD01.
## END ##
Please comment here on the email list and help us understand what you as a TC member would like to have happen here.
Bret