cti message
[Date Prev]
| [Thread Prev]
| [Thread Next]
| [Date Next]
--
[Date Index]
| [Thread Index]
| [List Home]
Subject: Re: Re: [cti] Re: [EXT] Re: [cti] Working Call Recap
- From: "Jason Keirstead" <Jason.Keirstead@ca.ibm.com>
- To: Drew Varner <drew.varner@ninefx.com>
- Date: Tue, 21 May 2019 13:00:37 +0000
This problem exists
with all SCOs in STIX and has always existed.
You can always
have "orphan" relationships and "orphan" identity objects
outside your bundle. There is no requirement that everything inside a bundle
or page has to be de-referenced inside itself. To impose such a requirement
would make bundles unnecessarily large.
-
Jason Keirstead
Lead Architect - IBM Security Connect
www.ibm.com/security
"Would you like me to give you a formula for success? It's quite simple,
really. Double your rate of failure."
- Thomas J. Watson
From:
Drew
Varner <drew.varner@ninefx.com>
To:
Bret
Jordan <Bret_Jordan@symantec.com>
Cc:
Jason
Keirstead <Jason.Keirstead@ca.ibm.com>, Allan Thomson <athomson@lookingglasscyber.com>,
"cti@lists.oasis-open.org" <cti@lists.oasis-open.org>,
"Piazza, Rich" <rpiazza@mitre.org>, Sean Barnum <sean.barnum@FireEye.com>,
"Kelley, Sarah E." <skelley@mitre.org>
Date:
05/17/2019
02:45 PM
Subject:
[EXTERNAL]
Re: [cti] Re: [EXT] Re: [cti] Working Call Recap
Sent
by: <cti@lists.oasis-open.org>
Bret,
Your email of Friday, May 17, 2019 at 7:41 AM
> We no longer look at "id_method" and "id_method_details"
as common properties. We do, however, add them to the STIX Bundle
and we may consider adding them to the TAXII envelope and or TAXII http
headers. But these properties would no longer be an option on SCOs themselves.
I would also be open to adding these properties to an identity object as
well. This way if someone wanted to say that they "ALWAYS"
generate their IDs in this method, they can just do it via the identity
object.
I was referencing the pagination of a Get Objects response with heterogeneous
ID types when the id_* is in the Envelope. What would that look like?
If we add it to the Identity object, what happens when I receive SCOs where
I have not received the Identity object yet? There is no requirement that
theyâre even in the same Bundle that I know of.
Thanks,
Drew
> On May 17, 2019, at 1:23 PM, Bret Jordan <Bret_Jordan@symantec.com>
wrote:
>
> Drew,
>
> You do not paginate based on content in a bundle. You paginate
based on objects in an envelope. So you would never paginate on the
ID method, just like you would never paginate on a name property or labels
property.
>
> Bret
>
> From: Drew Varner <drew.varner@ninefx.com>
> Sent: Friday, May 17, 2019 11:14:51 AM
> To: Jason Keirstead
> Cc: Allan Thomson; Bret Jordan; cti@lists.oasis-open.org; Piazza,
Rich; Sean Barnum; Kelley, Sarah E.
> Subject: Re: Re: [cti] Re: [EXT] Re: [cti] Working Call Recap
>
> Agreed, it must be available at the object level.
>
> Imagine the TAXII client and server implementations for Get Objects
that needs to paginate across multiple bundles partitioned by ID method.
"Bundle 2 of 3, Rows 30-59â. Brutal.
>
> Also, are people using large Bundles in practice, such as a Bundle
that is not parsed entirely in RAM as a whole? I get large XML files when
SAX is a standard. Most JSON parsers Iâve found serialize and deserialize
them in memory. Iâve worked with XMPP where you have a massive collection
of elements inside a <stream/> element. I find it unwieldy. Iâd
expect truly large collections of STIX to be individual files.
>
> Adding spec_version back to Bundle with a new meaning will likely
cause confusion.
>
> 2.0 -> Bundle.spec_version is the version of all objects inside
> 2.1 -> Bundle.id_methods is the id_methods of all SCOs inside
but Bundle.spec_version just applies to the Bundle
>
>
> > On May 17, 2019, at 12:39 PM, Jason Keirstead <Jason.Keirstead@ca.ibm.com>
wrote:
> >
> > I prefer adding it to the identity object.
> >
> > A bundle will very often - I would argue almost normally - contain
many objects from many different tools and producers.
> >
> > If I have 5000 objects from one tool in the portfolio, and 5000
from another tool in the portfolio, and these use two different ID methods,
I would prefer to define those methods twice (one in each producer identity)
- not 5000 times each. Tacking it on a bundle doesn't solve for this problem
- identity does.
> >
> > -
> > Jason Keirstead
> > Lead Architect - IBM Security Connect
> > https://clicktime.symantec.com/3BDij1fDK1W8yDwUi8hQWsM7Vc?u=www.ibm.com%2Fsecurity
> >
> > "Would you like me to give you a formula for success? It's
quite simple, really. Double your rate of failure."
> >
> > - Thomas J. Watson
> >
> >
> >
> > From: Allan Thomson <athomson@lookingglasscyber.com>
> > To: Bret Jordan <Bret_Jordan@symantec.com>,
Sean Barnum <sean.barnum@FireEye.com>, "drew.varner@ninefx.com"
<drew.varner@ninefx.com>
> > Cc: "Piazza, Rich" <rpiazza@mitre.org>,
"Kelley, Sarah E." <skelley@mitre.org>, "cti@lists.oasis-open.org"
<cti@lists.oasis-open.org>
> > Date: 05/17/2019 12:01 PM
> > Subject: [EXTERNAL] Re: [cti] Re:
[EXT] Re: [cti] Working Call Recap
> > Sent by: <cti@lists.oasis-open.org>
> >
> >
> > Thanks Bret.
> >
> > I would be fine with any of the suggested options a) bundle changes
b) taxii envelope changes c) identity properties where that identity is
associated with creation of the object.
> >
> > My preference would lie primarily with either a) or b). With
a slight preference for a) because a bundle is and can be used to provide
a wrapper of STIX content outside of TAXII transport whereas making the
change for a TAXII transport only fixes TAXII not all cases on how STIX
is exchanged.
> >
> > The challenge with c) is that it would still require a property
on every SCO to point to the identity creating the SCO and that somewhat
defeats the purposes of a SCO (given that they are meant for re-use across
any vendor that uses the same deterministic id).
> >
> > allan
> >
> > From: Bret Jordan <Bret_Jordan@symantec.com>
> > Date: Friday, May 17, 2019 at 7:41 AM
> > To: Sean Barnum <sean.barnum@FireEye.com>, "drew.varner@ninefx.com"
<drew.varner@ninefx.com>, Allan Thomson <athomson@lookingglasscyber.com>
> > Cc: "Piazza, Rich" <rpiazza@mitre.org>, "Kelley,
Sarah E." <skelley@mitre.org>, "cti@lists.oasis-open.org"
<cti@lists.oasis-open.org>
> > Subject: Re: [cti] Re: [EXT] Re: [cti] Working Call Recap
> >
> >
> > All,
> >
> > I would like to step back and focus this discussion a bit, since
we seem to be all over the place. First a bit of background to bring everyone
up to speed.
> >
> > 1) Back at the DC3 F2F the main objection to having SCOs be TLOs
was the explosion of objects with the same data (e.g.. ip address 1.2.3.4).
At that point, many said we could use deterministic IDs, but also
at that point NO ONE has produced a solid proposal for how that could or
should be done. Thanks to Allan's works, we now have a solution that
solves that problem.
> >
> > 2) Some people will want to use deterministic IDs to try and
figure out of content is the same or not. On the flip side, some
people will not care at all, and will just treat every IDs as unique. We
can not force people to pay attention to the IDs.
> >
> > 3) It is super critical that we do not have un-intended collisions
on IDs. The importances of this is carved in stone. Our whole
versioning model and graph design hinges on this.
> >
> > 4) Drew is correct, we removed the spec_version from Bundle because
we realized that using a spec_version on bundle to describe the spec version
of the content in side the bundle was wrong.
> >
> > 5) A bundle in STIX or an envelope in TAXII can contain meta
data about the data it transmits. This is perfectly valid.
> >
> > 6) There has been significant concerned raised many times over
about bloat on SCOs. We also need to remember as Allan reminds us,
that STIX is about "TRANSPORT" it is not about how your local
system stores or uses data locally. It is an "interchange"
format. As such, we should constantly remind ourselves of that. Yes,
even I need to be reminded of this from time to time.
> >
> > My _personal_ proposal:
> >
> > We no longer look at "id_method" and "id_method_details"
as common properties. We do, however, add them to the STIX Bundle
and we may consider adding them to the TAXII envelope and or TAXII http
headers. But these properties would no longer be an option on SCOs themselves.
I would also be open to adding these properties to an identity object as
well. This way if someone wanted to say that they "ALWAYS"
generate their IDs in this method, they can just do it via the identity
object.
> >
> > Producers SHOULD tell consumers about how their IDs were created.
But they are not required to do so. What they are required
to do is guarantee that non-SCO ID's are globally unique and do not un-intentionally
collide with someone else's IDs. Meaning, they SHOULD be UUIDv4 or
the likes. If they want to use UUIDv5 then it is up to them to figure
out a namespace and such that makes sure they do not collide.
> >
> > For consumers that want to know and track the information about
how the IDs were created, they can. If they do not care, they do
not care. For organizations that want to track how the IDs were created
so they can re-share that information back out, then they can do so, how
ever they want in their datastore. But these fields will not be used
as part of any future digital signatures as they are purely meta-data that
may be learned outside of STIX.
> >
> > I would also suggest that we add the spec_version back to Bundle
and say that it represents the spec version of the Bundle container. This
way we can add extra meta data "things" to the bundle at a later
time if needed.
> >
> >
> > Bret
> >
> >
> > From:Sean Barnum <sean.barnum@FireEye.com>
> > Sent: Thursday, May 16, 2019 9:50:28 AM
> > To: drew.varner@ninefx.com; Allan Thomson
> > Cc: Piazza, Rich; Kelley, Sarah E.; Bret Jordan; cti@lists.oasis-open.org
> > Subject: Re: [cti] Re: [EXT] Re: [cti] Working Call Recap
> >
> > > Bundles are a way to package STIX objects. They are not
âtheâ way to package them. TAXII 2.1 packages them in an Envelope without
a Bundle.
> >
> > Sure, but this proposal does not require anyone to use Bundle.
> > If a producer/sharer wanted to exchange objects via TAXII without
a Bundle (which I do not necessarily recommend) they would need to assert
the properties on each object.
> >
> > This proposal is simply providing a mechanism for producers/sharers
who choose to use Bundle to convey their content in a vastly more efficient
way that makes it practical/possible to use.
> >
> > This does not preclude anyoneâs choices on how they wish to
share. It simply provides a viable option for a portion of the community
to be able to use STIX to share.
> >
> > The alternative of forcing the properties onto ALL SCOs would
basically double the size of most commonly shared high volume SCOs (IP
addresses, URLs, DomainNames, etc). Bloat like that would make STIX impractical
for many users and the Bundle approach proposal at hand offers a simple
and effective solution.
> >
> > Sean Barnum
> > Principal Architect
> > FireEye
> > M: 703.473.8262
> > E: sean.barnum@fireeye.com
> >
> > From: "drew.varner@ninefx.com" <drew.varner@ninefx.com>
> > Date: Thursday, May 16, 2019 at 10:23 AM
> > To: Allan Thomson <athomson@lookingglasscyber.com>
> > Cc: "Piazza, Rich" <rpiazza@mitre.org>, "Kelley,
Sarah E." <skelley@mitre.org>, Sean Barnum <sean.barnum@FireEye.com>,
Bret Jordan <Bret_Jordan@symantec.com>, "cti@lists.oasis-open.org"
<cti@lists.oasis-open.org>
> > Subject: Re: [cti] Re: [EXT] Re: [cti] Working Call Recap
> >
> > We removed it in 2.1 because we realized that a bundle could
contain objects of multiple versions. STIX objects have a spec_version
property so they are now-self-describing.
> >
> > Bundles are a way to package STIX objects. They are not âtheâ
way to package them. TAXII 2.1 packages them in an Envelope without a Bundle.
> >
> > On May 16, 2019, at 10:07 AM, Allan Thomson <athomson@lookingglasscyber.com>
wrote:
> >
> > It was one example. There are other properties on bundle and
my point remains.
> >
> > If I remember rightly the reason we moved it (?) was because
TAXII request/response took the version instead to make sure the bundle
contained the right set of objects for the exchange. So instead of the
bundle the TAXII req/response contained the version information.
> >
> > Thatâs just another way of doing the same thing.
> >
> > That is -> a parameter applied to a group of objects within
a wrapper (in this case it would be TAXII req/resp).
> >
> > I still think itâs a valid thing to do.
> >
> > Allan
> >
> > From: "Piazza, Rich" <rpiazza@mitre.org>
> > Date: Thursday, May 16, 2019 at 6:48 AM
> > To: Allan Thomson <athomson@lookingglasscyber.com>, "Kelley,
Sarah E." <skelley@mitre.org>, Sean Barnum <sean.barnum@FireEye.com>,
Bret Jordan <Bret_Jordan@symantec.com>
> > Cc: "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
> > Subject: Re: [cti] Re: [EXT] Re: [cti] Working Call Recap
> >
> > The spec_version property has been removed from Bundle in 2.1.
> >
> > Whether or not this was the right decisionâ
> >
> > From: <cti@lists.oasis-open.org> on behalf of Allan Thomson
<athomson@lookingglasscyber.com>
> > Date: Thursday, May 16, 2019 at 9:30 AM
> > To: "Kelley, Sarah E." <skelley@mitre.org>, Sean
Barnum <sean.barnum@FireEye.com>, Rich Piazza <rpiazza@mitre.org>,
Bret Jordan <Bret_Jordan@symantec.com>
> > Cc: "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
> > Subject: Re: [cti] Re: [EXT] Re: [cti] Working Call Recap
> >
> > Just because it is non-persistent does *not* mean that it can
not convey meaning and value to the recipient on how to model/capture the
objects within the bundle once they ingest those objects.
> >
> > Just as the bundle conveys version information (stating that
this bundle has objects with version X) then an additional property on
the bundle for other aspects that apply to the objects in the bundle is
really not that different.
> >
> > STIX2 should focus on how to efficient and effectively transmit
information between 2 or more systems.
> >
> > It does not represent a database.
> >
> > So I think the proposal does not break or undermine the current
definition of Bundle and does allow consumers to read those properties
and apply any logic on conversion to their internal database schemaâetc.
appropriately quite easily just as much as the version property on a bundle
is used.
> >
> > Allan
> >
> > From: "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
on behalf of "Kelley, Sarah E." <skelley@mitre.org>
> > Date: Thursday, May 16, 2019 at 4:40 AM
> > To: Sean Barnum <sean.barnum@FireEye.com>, "Piazza,
Rich" <rpiazza@mitre.org>, Bret Jordan <Bret_Jordan@symantec.com>
> > Cc: "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
> > Subject: RE: [cti] Re: [EXT] Re: [cti] Working Call Recap
> >
> > Arenât bundles designed to be thrown away? So if the information
is only kept at the bundle level for how the identifier is generated, you
would lose that information as soon as the bundle drops on the floor. You
could never retransmit with that information (in the use case that you
were passing along data you didnât create), and the consumer wouldnât
be able to go back and reference it in the future.
> >
> > Am I missing something?
> >
> > Sarah Kelley
> > Lead Cybersecurity Engineer, T8B2
> > Defensive Operations
> > The MITRE Corporation
> > 703-983-6242
> > skelley@mitre.org
> > <image001.jpg>
> >
> > From:cti@lists.oasis-open.org<cti@lists.oasis-open.org>
On Behalf Of Sean Barnum
> > Sent: Wednesday, May 15, 2019 5:15 PM
> > To: Piazza, Rich <rpiazza@mitre.org>; Bret Jordan <Bret_Jordan@symantec.com>
> > Cc: cti@lists.oasis-open.org
> > Subject: Re: [cti] Re: [EXT] Re: [cti] Working Call Recap
> >
> > I see two options.
> >
> > â Make Bundles contain objects of
only one id method approach. If you have objects of different approaches
you would need to break them into separate bundles. This would address
the question but could be kind of complex in some situations so would not
be my preferred approach.
> > â Add id_method and id_method_details
to Bundle but also leave them as optional on objects.
> > â This
way the 99.9X% case (a producer sharing their own homogenous content) would
simply assert the values at the Bundle level and it would apply to all
content in the bundle.
> > â For
the 00.0X% case where a Bundle could contain objects using multiple id
approaches you would define the primary method on the Bundle and then the
different method on the objects where it applies. So, if an object contained
those properties it would override the assertion at the Bundle level. I
think this is the best of all worlds. It is highly efficient. It very simply
covers the large majority of cases and still provides a slightly more complex
but very simple way of dealing with the minority case.
> >
> >
> > Sean Barnum
> > Principal Architect
> > FireEye
> > M: 703.473.8262
> > E: sean.barnum@fireeye.com
> >
> > From: <cti@lists.oasis-open.org> on behalf of "Piazza,
Rich" <rpiazza@mitre.org>
> > Date: Wednesday, May 15, 2019 at 3:56 PM
> > To: Bret Jordan <Bret_Jordan@symantec.com>, Sean Barnum
<sean.barnum@FireEye.com>
> > Cc: "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
> > Subject: Re: [cti] Re: [EXT] Re: [cti] Working Call Recap
> >
> > How do we handle bundles that contain SCOs with different methods
of creating the UUID?
> >
> > For instance - Iâm just passing on various SCOs I received from
different producers, who used different methodsâ
> >
> > From: <cti@lists.oasis-open.org> on behalf of Bret Jordan
<Bret_Jordan@symantec.com>
> > Date: Wednesday, May 15, 2019 at 3:50 PM
> > To: Sean Barnum <sean.barnum@fireeye.com>
> > Cc: "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
> > Subject: [cti] Re: [EXT] Re: [cti] Working Call Recap
> >
> > After talking with Sean on the phone about this, even though
this is very late in the game, I think this really makes a lot of sense.
I think if we would have thought about this during the Mini-Group we probably
would have done this from the get go.
> >
> > Basically, to be really clear, what he is proposing is that we
move the id_method and id_method_details from individual SCOs to the Bundle.
> >
> > The pros I see are:
> > 1) less bloat on the wire
> > 2) simplify some of the indicator text
> >
> > We can talk about this in next weekâs working call.
> >
> > Bret
> >
> >
> > Sent from my Commodore 128D
> >
> >
> >
> >
> >
> > PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE
7415 0050
> >
> > On May 15, 2019, at 8:33 PM, Sean Barnum <sean.barnum@fireeye.com>
wrote:
> >
> > I have reviewed the changes highlighted below and added various
comments in the doc.
> > I see that Bret has already accepted many of them.
> >
> > There is one very major issue we have with a proposed change
that would require id_method and id_method_details on EVERY SCO for a producer
using a non-default approach for deterministic id generation.
> > This proposed change would require MASSIVE unnecessary bloat
in STIX content as the same exact ~7 lines of JSON would need to be added
to every SCO produced. Given the fact that we (and I am sure others) deal
with SCOs on the scale of billions this proposed change would make STIX
untenable as an option for us.
> >
> > I would like to propose an alternate approach that I think achieves
the intended purpose in a MUCH more efficient and unbloated fashion.
> > I propose that the id_method and id_method_details properties
be added to the Bundle object and would assert the deterministic id method
used for content within the bundle.
> > â It would support the intended purpose
under options 1-3
> > â It would be far more efficient
in that it would only be necessary to specify it a single time for a Bundle
that could contain thousands of SCOs.
> > â It would be clear and easy to understand.
> > â It would allow us to simplify the
language under the identifier section of the spec
> >
> >
> >
> > Sean Barnum
> > Principal Architect
> > FireEye
> > M: 703.473.8262
> > E: sean.barnum@fireeye.com
> >
> > From: <cti@lists.oasis-open.org> on behalf of Bret Jordan
<Bret_Jordan@symantec.com>
> > Date: Tuesday, May 14, 2019 at 5:56 PM
> > To: "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
> > Subject: [cti] Working Call Recap
> >
> >
> > All,
> >
> > We had a good call today and were able to resolve some of the
last few issues that were holding up the release of Working Draft 04.
> >
> > Summary:
> >
> > 1) After editorial review we noticed that the language for identifiers
would not allow two organizations to write code and produce the same deterministic
ID. So we needed to tighten down the language and define what an
OASIS STIX 2 deterministic ID actually is. Prior to the call I worked with
Allan, Trey, Rich P, and the MISP folks on an acceptable solution. We reviewed
the changes on the working call and the changes are now in section 2.9.
Please review.
> >
> > 2) On the Malware Analysis object we had a property called "analysis_environment"
that was a dictionary that used a three value vocabulary. This design
no longer works since we do not have the cyber observable container anymore.
So we had to make a change. The consensus on the call was to use
properties on the malware analysis object itself and remove the vocabulary.
Those changes have been made in suggestion mode in section 4.11.
Please review.
> >
> > 3) We also accepted small changes to the following sections;
> >
> > a) 2.7 Hashes
> > b) 3.2 Spec Version
> > c) 10.13 Infrastructure Type Vocabulary
> > d) 4.3 Course of Action os_execution_envs
> > e) 4.13 Observed Data
> > f) 4.18 Vulnerability Relationships
> >
> >
> >
> > Action Items:
> >
> > i) Please make a final in depth review of COA and provide any
description text changes you feel appropriate.
> > ii) Review the changes to Malware Analysis
> > iii) Review the changes to the identifier
> > iv) Start making your top down full review of the documents
> >
> >
> > Schedule:
> > On the call we said we would give the TC one more week to review
the changes to Malware Analysis and the identifier. So depending
on how next week's working call goes, we might be able to release Working
Draft 04 next week.
> >
> >
> > Thanks
> > Bret
> >
> >
> >
> >
> >
> >
> > This email and any attachments thereto may contain private, confidential,
and/or privileged material for the sole use of the intended recipient.
Any review, copying, or distribution of this email (or any attachments
thereto) by others is strictly prohibited. If you are not the intended
recipient, please contact the sender immediately and permanently delete
the original and any copies of this email and any attachments thereto.
> > This email and any attachments thereto may contain private, confidential,
and/or privileged material for the sole use of the intended recipient.
Any review, copying, or distribution of this email (or any attachments
thereto) by others is strictly prohibited. If you are not the intended
recipient, please contact the sender immediately and permanently delete
the original and any copies of this email and any attachments thereto.
> > This email and any attachments thereto may contain private, confidential,
and/or privileged material for the sole use of the intended recipient.
Any review, copying, or distribution of this email (or any attachments
thereto) by others is strictly prohibited. If you are not the intended
recipient, please contact the sender immediately and permanently delete
the original and any copies of this email and any attachments thereto.
> >
---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that
generates this mail. Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php
[Date Prev]
| [Thread Prev]
| [Thread Next]
| [Date Next]
--
[Date Index]
| [Thread Index]
| [List Home]