OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

cti-taxii message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]


Subject: Re: [cti-taxii] Re: [EXT] Re: [cti-taxii] TAXII Query / Search


Gary - how would you propose in the future we can extend or expand this approach, without redoing it from scratch?

This is the problem with having query-specific endpoints or query parameters such as "related to" - they can't be extended or added to in any logical way.

It will result in either no way to actually do combinations of parameters, or an unworkable mishmash of normative text quantifying all of the edge cases of which endpoints can be combined with which others to get what behaviour.

Sent from IBM Verse


Gary Jay Katz --- Re: [cti-taxii] Re: [EXT] Re: [cti-taxii] TAXII Query / Search ---

From:"Gary Jay Katz" <gary.katz@FireEye.com>
To:"Jason Keirstead" <Jason.Keirstead@ca.ibm.com>, "Bret Jordan" <Bret_Jordan@symantec.com>
Cc:cti-taxii@lists.oasis-open.org
Date:Mon, Nov 19, 2018 4:55 PM
Subject:Re: [cti-taxii] Re: [EXT] Re: [cti-taxii] TAXII Query / Search


Jason,

    I believe you are correct, there are a number of complex use cases that this would not meet.  For those that remember back to some of the early face-to-face meetings, I pushed for trying to find a solution that met all of these solutions in the first release.  I now believe that this needs to be taken in parts.  There may be certain TAXII implementations that only need to and are able to support basic search and pivoting, while others will be built to perform complex querying such as what you are suggesting.

 

     Having an ability to easily query relationships and pivot off of them which does not support the more complex use cases, does not mean that the capability needs to be rewritten or that those complex use cases are invalid.  We just need ways to do the simple things simply, while eventually put in place the ability to do the complex capabilities. 

 

    Currently we have no ability to query relationships or pivot.  This is a blocker for anyone trying to provide the baseline capabilities to support the STIX model using TAXII.  I would suggest we get the baseline capabilities in place to allow releasing a minimal viable product in the near-term while continuing to work towards more advanced query capabilities in a future release.

 

-Gary

 

From: <cti-taxii@lists.oasis-open.org> on behalf of Jason Keirstead <Jason.Keirstead@ca.ibm.com>
Date: Monday, November 19, 2018 at 3:38 PM
To: Bret Jordan <Bret_Jordan@symantec.com>
Cc: "cti-taxii@lists.oasis-open.org" <cti-taxii@lists.oasis-open.org>
Subject: [cti-taxii] Re: [EXT] Re: [cti-taxii] TAXII Query / Search

 

The problem with this approach of "getting something done in a a few days", is it is highly likely it will have to be re-done in the future because it isn't considering the other use cases.

For example - In the future how can I find all sightings related to Object Foo that match the indicator 1.2.3.4, with your proposal? Seeing how any future "match the indicator" portion would be an entirely different endpoint than the "related-to" endpoint, I now can't combine them - and therefore would have to make multiple - perhaps dozens, hundreds, or even thousands of round-trips to the server to collect this information.

-
Jason Keirstead
Lead Architect - IBM Security Connect
www.ibm.com/security

"Things may come to those who wait, but only the things left by those who hustle." - Unknown




From:        Bret Jordan <Bret_Jordan@symantec.com>
To:        Jason Keirstead <Jason.Keirstead@ca.ibm.com>
Cc:        "cti-taxii@lists.oasis-open.org" <cti-taxii@lists.oasis-open.org>
Date:        11/19/2018 04:27 PM
Subject:        Re: [EXT] Re: [cti-taxii] TAXII Query / Search





Jason, the proposal you have submitted is valid, and one potential solution.  However, it represents a fair bit of complexity on the specification and on implementations. There are also a lot of unanswered questions about how it would actually be done. To support this, we would also need to create several conformance levels for it, which is non-trivial.

Per your own statements, over the past 12 months I have seen very little to no public support for it. As such, I am trying to find a simpler solution that we can do now and then look at the more complicated solution later.  I fear if we try to do a more complicated solution now, then TAXII 2.1 will still be many months away from being done.

The proposal I have worked on with Drew and Gary is something we can do very easily in a matter of days, which could enable us to finish TAXII 2.1 within the next month.

But all of this is up to the TC to decide.  We can see what kind of feedback we get, and see where the majority of TC members sit. But at this stage, I would personally (not as a chair or editor) prefer a more simple solution that can be used now.  

Bret



From: Jason Keirstead <Jason.Keirstead@ca.ibm.com>
Sent:
Monday, November 19, 2018 1:06:11 PM
To:
Bret Jordan
Cc:
cti-taxii@lists.oasis-open.org
Subject:
[EXT] Re: [cti-taxii] TAXII Query / Search

 
Hi Brett - Terry and I have an alternative proposal we've been asking the TC to have a look at here

https://docs.google.com/document/d/1Cy_9Bh5tKEkDHGg2iv5c3AwriqVr7ygbKXWOv4-uHxs/edit#heading=h.1jcqb6vc5y7z

As we lay out in the document, there are a number of issues TAXII has with query, and I think the proposal you have below is insufficient to solve the use cases we have in Github:


We first started working on this last Christmas - so it's been almost an entire year. We've been struggling to get feedback on the document and would like the TC to comment on it. It is in my opinion a very simple proposal - yet it allows query to be extensible for many future use cases.


We need to avoid endpoint pollution as one tries to bolt-on new endpoints and arguments for every kind of query parameter under the sun without any plan for the future use cases we have to meet.


       
Issue 15 - As a User, I want to traverse the STIX graph over TAXII in an efficient manner...
Issue 7 - Need ability to request related objects in one request to a distance of 1(?)
Issue 5 - Add support to get embedded objects in a single request, dereference...
Issue 6 - Add ability to find all objects related to a particular STIX object ID
Issue 4 - TAXII Observed Data Query  

Also, I think "the biggest element holding people back" is very much open to interpretation and what persona you fill. From the perspective of most any SOAR-based security tool integrating with a TAXII server, the biggest element holding me back is ability to find indicators that match an observation, or find observation that match an indicator.


Finding out who is related to a STIX object is very much a TIP-centric use case (although I will add that that use case is also addressed in our above proposal as well).


My point is that we should not be trying to come out with a lot of piecemeal-endpoints without thinking of the bigger picture of query.


-
Jason Keirstead
Lead Architect - IBM Security Connect

www.ibm.com/security

"Things may come to those who wait, but only the things left by those who hustle." - Unknown





From:        
Bret Jordan <Bret_Jordan@symantec.com>
To:        
"cti-taxii@lists.oasis-open.org" <cti-taxii@lists.oasis-open.org>
Date:        
11/19/2018 03:26 PM
Subject:        
[cti-taxii] TAXII Query / Search
Sent by:        
<cti-taxii@lists.oasis-open.org>





All,

Over the past year I have heard from several people that they are unable to implement TAXII because there is no way to search for relationships. Yes, there are lots of other things that people would like to do with a query request, but the ability to search for a relationship seems to be the single biggest missing element holding people back from implementing TAXII.

I have talked with Drew and Gary about how we could solve this in the very short term and would like to propose the following solution to the TC. If the TC agrees with this, Drew and I can add this to Working Draft 05 in the next week and submit Working Draft 05 to the TC for Review and CSD ballot the first of December.

Proposal

1) We create an endpoint called:
<api-root>/collections/<id>/relationships/related-to/<stix-id>/

This endpoint would return any SROs where the source_ref or target_ref matched the supplied STIX ID. This would be a very simple database query, and would not require a lot of computation work.  This endpoint would be mandatory to implement for all TAXII Servers as defined in the conformance section.


2) We add an optional URL parameter for the relationships endpoint called:
?deref=true

This optional URL parameter would tell the server to automatically send the objects that are referenced in the SROs that are being returned. From a conformance standpoint, this URL parameter would be optional to implement.

If a client makes a request with that URL parameter and the server has not implemented it, the server would respond with an HTTP 501 Not Implemented error code and a TAXII error message that says the URL parameter is not implemented.  

From a database performance standpoint, this URL parameter would require that the server perform multiple database queries for each request and would require the server to do some book keeping to ensure that it does not send the same object multiple times. However, this feature would eliminate the overhead of parsing multiple RESTful requests from the client as it comes back and asks for each of the objects one at a time.

Conclusion

We have some support for doing this already as this would be easy to implement in the specification and in code, and would solve a major blocker that has been identified. I would be curious to know if anyone else would support this or be against this.  This does not mean that we will not look at a more elaborate pattern based / property based query solution in the future.

Bret










This email and any attachments thereto may contain private, confidential, and/or privileged material for the sole use of the intended recipient. Any review, copying, or distribution of this email (or any attachments thereto) by others is strictly prohibited. If you are not the intended recipient, please contact the sender immediately and permanently delete the original and any copies of this email and any attachments thereto.


[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]