Another thing that is called out is that Filtering in JSON-API is more robust. Can you clarify EclecticIQ's view here? The JSON API says the following about filtering:
The filter query parameter is reserved for filtering data. Servers and clients SHOULD use this key for filtering operations.
Note: JSON API is agnostic about the strategies supported by a server. The filter query parameter can be used as the basis for any number of filtering strategies.
So it seems like JSON-API is "agnostic about the strategies" around filtering. And thus gives not additional context. So how is it better than what we are doing? In fact, our filtering currently mimics this design. We just call it "match" instead of
"filter". But that could easily be changed if the community decided on it.
From: Joep Gommers <email@example.com>
Sent: Sunday, March 12, 2017 2:10 PM
To: Bret Jordan; firstname.lastname@example.org
Subject: Re: On current TAXII discussions
My apologies if I’ve misspoken, it was not the intent to call anything broken. I make two remarks on opportunities for improvement and de-risking.
The first is about using RC1 (roots, collections, objects) and using an existing API to de-risk the path from unproven specification to robust implementation (pagination, filtering, syntax) etc. JSON-API as
an example as shown in our draft, but there might be others.
The other point is about the separation of STIX and TAXII. TAXII query is a good example (and an awesome idea). I’ll defer to any technical commentary and specifics to your discussions with the team, that’s
beyond my expertise.
From: Bret Jordan <Bret_Jordan@symantec.com>
Date: Sunday, 12 March 2017 at 16:24
To: Joep Gommers <email@example.com>, "firstname.lastname@example.org" <email@example.com>
Subject: Re: On current TAXII discussions
After reading your email and comparing it to what has been said by Sergey and Wouter of your team it sounds like the real issue at play is the fact that you do not want TAXII to have TAXII Query functionality.
Aka, the fact that RC1 of TAXII 2.0 has the beginnings of TAXII Query and allows a TAXII client to query for STIX data. Yes this means that the TAXII server needs to know how to look in to the STIX data and find the right records. This seems to be the real
As I have said all along, if there are things that RC1 does not do or things that it does that are broken, please bring them up. But saying it is broken without concrete examples or illustrating where it is
broken, is hard to follow.
From: firstname.lastname@example.org <email@example.com> on behalf of Joep Gommers <firstname.lastname@example.org>
Sent: Sunday, March 12, 2017 6:13:07 AM
Subject: [cti] On current TAXII discussions
We’re in the business of exchanging CTI and being able to use it per the use-cases of analysts and subsequent intelligence-led security efforts. Over the past three years we’ve had to be business of many other things to get there; protocols, work-arounds to
bad implementations, extending specs, ignoring specs – all in the pursuit of robustness, stability and moving on to what really mattered. We’ve only recently had the luxury of time in being able to look holistically at RC1 in the backdrop of only know being
able to reflect on our real-world experiences the past 18-months. I apologize if this cadence is not ideal for the community. This is part on us and part of the reality of where the market stands today in being able to give it real-world insights.
Our company’s big bet for scalability is STIX and TAXII and we have and will continue to invest millions in its advancement. Open-source implementations, storage, distribution, gateways, uni-directional data diodes, workflows, exchange and so forth. I can assure
the community that we’re both invested in the effort and grounded in the reality of its many gaps, use-cases and non-functional requirements. I’m sure there are many technical pro’s and con’s to our proposal and spec draft and I assume the community with give
it the consideration and discussion it deserves. The underlying message though is that the journey from unproven specification to proven and robust implementation is long and costly. This does not seem to be a key consideration and from a business perspective
I see disproportional risk to reward not choosing for existing APIs with mature existing communities of developers; JSON-API is an example, so is Odata. And if I’m completely honest, I’m not too keen on having to pick up that slack as one of the few that will
invest in fully implementing and supporting the specification if it can be avoided. We will if the community decides so though.
Reviewing the substance of discussion, there is allot of brain allocated to technical nuances that many others have solved before where I think it would be better spend on CTI and a wider set of use-cases. I realize that, not having had the luxury of participating
much more than we did, this hypocrisy is forgiven and seen in the light in which it is intended; positive criticism/reality check from the outside.
Lastly, it’s worthwhile to point out that in the current spec we require TAXII servers/clients to understand STIX. STIX is non-trivial and manipulating it (considering its many features) in this way puts a disproportioned burden on TAXII servers and their implementation.
Although less complicated for IOC-only data, that’s not what TAXII is all about. We would highly recommend making a much cleaner cut.
Combined we believe there is a notable risk that 2.1 or 3.0 will already require significant protocol changes. Our stance is:
- There exists significant avoidable risk in the current specification and recommend to seriously consider APIs with existing and mature communities (our short-cut with the JSON-API proposal and draft or another);
- Our proposed draft is not equal to ‘a suggestion’ or ‘brought up before’ and comes with a draft, takes into account RC1 functionality and adds some. We hoped this would have been enough to separate this discussions from some of the previous. If by its own
merit is not seen as sufficiently clear in terms of added value, that’s fine – but is not equal to an opening to put everything back on the table.
- Putting a burden of parsing STIX on TAXII as per RC1 is strongly advised to re-consider. Considerably impacts barrier to entry in implementing and for those implementing, might lead to less open code (due to size investment).
Hope this helps in the discussions around our proposal, but perhaps even more so for the future.
Founder & CEO
From: <email@example.com> on behalf of Jason Keirstead <Jason.Keirstead@ca.ibm.com>
Date: Saturday, 11 March 2017 at 01:29
To: "firstname.lastname@example.org" <email@example.com>
Cc: "Bret_Jordan@symantec.com" <Bret_Jordan@symantec.com>, "firstname.lastname@example.org" <email@example.com>
Subject: Re: [cti] TAXII
Hello all - I have been making numerous arguments against this JSON-API proposal on Slack, but figured I had better send them to the mailing list as the whole TC does not monitor Slack.
- First, let me preface this by saying personally I do not agree at all with how this proposal has come about. We have been working on TAXII 2 as a community for over a year, and many, many TC members have contributed to it. To come in at the RC stage with
an "out-of-the-blue" ground-up re-proposal is far from ideal. It is in my opinion very hard to consider a proposal solely on its merits with timing such as this.
- In my opinion, in order to make an argument to re-visit all of TAXII at this late stage, one should have a compelling set of reasons for this re-visit - ie a list of problems that exist in the RC proposal that either do not exist or are solved in the new
proposal. There is no such set of reasons presented here - it is simply a ground-up re-think with no compelling arguments presented as to why it is superior to the current RC TAXII proposal. As such, to me this JSON-API proposal to me is very much a solution
in search of a problem... what are the specific problems it is solving?
- The new JSON-API based proposal is **extremely verbose**. It is difficult to put an exact number on the "data bloat" occurring here, but I would conservatively place it at around 10x vs. the RC1 proposal. We must keep in mind that most vendors are potentially
dealing with extremely large data set sizes when it comes to CTI production and consumption, and size matters - both on the wire, and in memory, and at parse time. The argument "CPU/Memory/Disk is cheap" does not hold water in the world of big data. Data size
still matters. In return for this data bloat, I would hope that this proposal came with a large set of concrete benefits, but I don't see them.
- Finally - If we were actually going to re-open everything and spend another 3/6/9/12 months on TAXII 2, then I would humbly also request time to submit a proposal for OData as well - http://www.odata.org/ -
as this is an existing OASIS standard developed over a long period of time, and backs large scale enterprise services already, so I have a lot more confidence in it than JSON-API for our use cases, and it would also give us TAXII Query as well as many other
capabilities out of the box.
STSM, Product Architect, Security Intelligence, IBM Security Systems
Without data, all you are is just another person with an opinion - Unknown
----- Original message -----
From: Patrick Maroney <firstname.lastname@example.org>
Sent by: <email@example.com>
To: Bret Jordan <Bret_Jordan@symantec.com>
Cc: "firstname.lastname@example.org" <email@example.com>
Subject: Re: [cti] TAXII
Date: Wed, Mar 8, 2017 11:22 AM
The Eclectic-IQ slide on Objects (slide 8) is a compelling argument (as are arguments for json-api, etc.). As is the general argument for looking to and fully vetting mature transport mechanisms like XMPP.
My .02: With all due respect to the efforts of those folks who've expended significant effort to get us to where we are, I don't think one week is adequate for the community to fully assess and understand these counter-proposals.
Principle Engineer - Data Science & Analytics
On Mar 7, 2017, at 12:24 PM, Bret Jordan <Bret_Jordan@symantec.com> wrote:
On the working call today, we had two presentations about possible additions / changes / modifications to the current TAXII RC 1 specification. The first presentation was from EclecticIQ on the possible use of JSON-API. The second presentation was from Cisco
on the use of XMPP-Grid. At a minimum, I would like this community to use these presentations as a sanity check for what we have done.
Call to action: As a member of this open community, if you are in support of either of these technologies, or would like the TAXII Community to focus more time on either of them, please speak up. If you have questions about what these technologies would mean
for TAXII, please ask here on the email list and we will defer to either EclecticIQ or Cisco to answer.
With all suggestions, it is up to the sponsor of that idea to gather support, drive the discussion, and help write any and all normative text that would support it.
As a reminder, the current TAXII specification is nearing completing. So if we need to adopt or change anything based on these presentations, we really need this community to identify that within the next week.
--------------------------------------------------------------------- To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail. Follow this link to all your TCs in OASIS at: