RE: [cti-taxii] Goals for TAXII 2.0

I’m right there with Jason in terms of the conceptual simplification that such an arrangement would offer – we need fewer ways to do the same things.

That said, there are some interesting aspects to this discussion that I’d like to highlight.

In terms of HTTP, Polling is basically a busy wait (aka an anti-pattern). The client makes a connection to determine if there’s information available. Then reconnects. Then reconnects. As this scales up – more clients and smaller polling intervals – the overall system’s computing power will be used primarily to determine there’s nothing to do (Are we there yet? Are we there yet?). In terms of HTTP, the only real way around the busy wait (AFAIK) is to require that all information recipients have addressable TAXII systems so that pushing is available.

However, requiring that everyone has a web server in order to participate just seems wrong. Plus, that arrangement will force odd/inefficient architectures – you’ll have to have a “TAXII Server” on the perimeter (e.g., DMZ) to receive info and that TAXII Server will have to act as a gateway into your organization. This adds cost to the overall solution (both in terms of computing power, architecture, and mental overhead).

So what do we do?

If you feel like this is an unanswerable question, you might be stuck in the mental box of client/server architectures that I was stuck in for a while: you can’t push to a client – it’s dumb, and that’s not a supported pattern anyway! The subtle issue (for me anyway) was a conflation of the networking definition of client/server – the client connects, the server listens – and traditional client/server roles – the client is dumb, the server is smart. HTTP and web programming strongly reinforces this co-mingling of definitions – servers do the hard things and clients are just dumb, transient boxes that do what you tell them when they ask to be told something. For me, it was easy to work from an HTTP/Web Programming perspective for a while and lose sight of other perspectives.

In all practical senses, we must work within the client/server bounds of TCP/UDP. However, we should only work in terms of the client/server paradigm if we decide that it suits our use. If we want smart TCP clients, we should have them!

At the risk of going slightly long (I guess the horse has been out of the barn for a few days there…), for TAXII 2.0 we should think of the things we want to do and then figure out the best way of doing them. Do not frame your thinking around client/server architectures, specific protocols, or specific message formats. Bump up a level of abstraction and think about the roles and goals of information sharing participants, information sharing use cases, and the design goals of the system. I think I’m hearing support for Tenet 1, which I would notionally cast as a design goal – “Only One Way To Do It”. I also put my support behind this tenet.

Thank you.

- Mark

PS – My apologies for accidentally addressing my previous response to Terry and not Jason.

From: Jason Keirstead [mailto:Jason.Keirstead@ca.ibm.com]
Sent: Wednesday, July 15, 2015 9:02 AM
To: Davidson II, Mark S
Cc: Trey Darley; Terry MacDonald; cti-taxii@lists.oasis-open.org
Subject: RE: [cti-taxii] Goals for TAXII 2.0

OK now things are starting to be more clear... essentially

- PUSH messages with subscriptions would require TAXII services to be open to both parties so one can subscribe then the other can push. So, this model does not really make sense for hub-and-spoke

- However, PUSH messages could also be used for an INBOX service without any subcription information. This model aligns with hub-and-spoke, where the spoke can push to the hub but not vice-versa.

So, harkening back to "Tenant 1" - I would propose that the publish / subscribe model in TAXII 2.0 should be adapted to be POLL only. Then it would work with hub-and-spoke.

-
Jason Keirstead
Product Architect, Security Intelligence, IBM Security Systems
www.ibm.com/security | www.securityintelligence.com

Without data, all you are is just another person with an opinion - Unknown

"Davidson II, Mark S" ---2015/07/15 09:50:41 AM---Subscription Information is optional in an Inbox Message. FWIW, I see how this would be confusing -

From: "Davidson II, Mark S" <mdavidson@mitre.org>
To: Jason Keirstead/CanEast/IBM@IBMCA, Trey Darley <trey@soltra.com>
Cc: Terry MacDonald <terry.macdonald@threatloop.com>, "cti-taxii@lists.oasis-open.org" <cti-taxii@lists.oasis-open.org>
Date: 2015/07/15 09:50 AM
Subject: RE: [cti-taxii] Goals for TAXII 2.0
Sent by: <cti-taxii@lists.oasis-open.org>

Subscription Information is optional in an Inbox Message. FWIW, I see how this would be confusing – the Subscription ID field itself is required, but it’s in an optional grouping (Subscription Information). Forgive my elite graphical editing skills (read: MS Paint).

cid:image002.jpg@01D0BEDE.8C10E900

So Terry, you have this part right:
> based on this flow, both services require open TAXII ports since the client connects to the server and then later on the server connects to the client.
In order to subscribe to a data collection and get pushed information back, both TAXII endpoints have to be addressable to each other. However, I think the key part you were missing is that the Subscription ID is optional and there is instead a workflow you didn’t envision.

I think this is an area where, moving forward, having a clear architecture, components, and interactions would help significantly. Then you (plural) wouldn’t have to try and infer relationships out of message properties.

Thank you.
-Mark

From: cti-taxii@lists.oasis-open.org [mailto:cti-taxii@lists.oasis-open.org] On Behalf Of Jason Keirstead
Sent: Wednesday, July 15, 2015 8:41 AM
To: Trey Darley
Cc: Terry MacDonald; cti-taxii@lists.oasis-open.org
Subject: Re: [cti-taxii] Goals for TAXII 2.0

( Note - If people want me to take this off the list let me know... )

But if that is the case... what is the "subscription ID" the hubs are using in the push message, and where did it come from? I am referring to the sample document - https://taxii.mitre.org/specifications/version1.0_draft1/TAXII_SampleUsage_November_2012.pdf . In this document, the workflow is the following:

- Client connects to server

2.3.2 Step 2: Feed Information Exchange Having learned of the Source's offered TAXII Services, the Subscriber wishes to learn which Data Feeds are available.

- Client requests subscription to feed from server

2.3.4 Step 4: Subscription Management Exchange The Subscriber now seeks to establish a subscription to their chosen TAXII Data Feed. T

- On interval, server connects to client, and issues PUSH messages with the subscription ID in them

"2.3.5 Step 5a: Data Push Exchange When the Source wishes to provide content associated with a TAXII Data Feed, it uses a TAXII Client to send a STIX Message (see Figure 10) to each Subscriber's Inbox Service."

Based on this flow, I presume the Client is the Spoke and the Server is the Hub. Regardless... based on this flow, both services require open TAXII ports since the client connects to the server and then later on the server connects to the client.

-
Jason Keirstead
Product Architect, Security Intelligence, IBM Security Systems
www.ibm.com/security | www.securityintelligence.com

Without data, all you are is just another person with an opinion - Unknown

Trey Darley ---2015/07/15 09:23:49 AM---Hey, Jason - In this configuration the spokes pull from the hub and push back to it. The hub itself

From: Trey Darley <trey@soltra.com>
To: Jason Keirstead/CanEast/IBM@IBMCA
Cc: Terry MacDonald <terry.macdonald@threatloop.com>, "cti-taxii@lists.oasis-open.org" <cti-taxii@lists.oasis-open.org>
Date: 2015/07/15 09:23 AM
Subject: Re: [cti-taxii] Goals for TAXII 2.0
Sent by: <cti-taxii@lists.oasis-open.org>

Hey, Jason -

In this configuration the spokes pull from the hub and push back to it. The hub itself doesn't make outbound connections.

Cheers,
Trey
--
Trey Darley
Senior Security Engineer
Soltra | An FS-ISAC & DTCC Company
www.soltra.com

From: cti-taxii@lists.oasis-open.org <cti-taxii@lists.oasis-open.org> on behalf of Jason Keirstead <Jason.Keirstead@ca.ibm.com>
Sent: Wednesday, July 15, 2015 14:20
To: Trey Darley
Cc: Terry MacDonald; cti-taxii@lists.oasis-open.org
Subject: Re: [cti-taxii] Goals for TAXII 2.0

Sorry maybe I am misunderstanding... maybe I have always misunderstood this! What I am referring to is the TAXII push to INBOX which contains a subscription ID... this implies that the client hosting that INBOX service, previously subscribed to something on that TAXII hub... which by nature (since it is subscriber based) implies it is a PUSH from the hub to the spoke.

If you had a hub and spoke configuration.. the hub would not be the subscriber.. the spokes would be subscribers... no?

I am having a hard time envisioning a model where there is a hub and spoke configuration where the hub subscribes to feeds on the spokes when the spokes don't allow inbond connections... how does that work?

-
Jason Keirstead
Product Architect, Security Intelligence, IBM Security Systems
www.ibm.com/security | www.securityintelligence.com

Without data, all you are is just another person with an opinion - Unknown

Trey Darley ---2015/07/15 09:11:21 AM---Hey, Jason - While it might not be immediately apparent, there are institutions which use both the p

From: Trey Darley <trey@soltra.com>
To: Jason Keirstead/CanEast/IBM@IBMCA, Terry MacDonald <terry.macdonald@threatloop.com>
Cc: "cti-taxii@lists.oasis-open.org" <cti-taxii@lists.oasis-open.org>
Date: 2015/07/15 09:11 AM
Subject: Re: [cti-taxii] Goals for TAXII 2.0
Sent by: <cti-taxii@lists.oasis-open.org>

Hey, Jason -

While it might not be immediately apparent, there are institutions which use both the poll and push due to site security policy. Basically, where you'd normally have a hub and spoke with peer-to-peer polling, in these cases no inbound connections are allowed on the edge node. Instead, these nodes are configured to a) poll a centrally accessible hub and b) push outbound to that same node.

I generally agree with the notion of stripping away "needless" flexibility but I don't think that applies in this particular case.

Cheers,
Trey
--
Trey Darley
Senior Security Engineer
Soltra | An FS-ISAC & DTCC Company
www.soltra.com

Hi All,

As per earlier discussions I really think it would be beneficial to discuss some key goals that we want TAXII 2.0 to adhere to. As such I've tried to incorporate the discussions we've had over the past year or so, and distill them into the following list.

Please note, this is a 'starter for 10', just to prompt further discussion. Please do not consider it official, final, or authoritative in any way :) (I should be a lawyer).

Goals for TAXII v2.0:

· Tenet 1 - Only One Way To Do It
TAXII v1.x allows for a lot of flexibility in how you implement it. It was designed to allow people to implement parts of the standard across multiple different nodes in multiple different locations. This has turned out to not be necessary, and in fact potentially a hinderance to the development of implementations (e.g. taxii discovery).

I would like us to have a core tenet that there will be only one way to perform a particular task. This should make it easier for implementers to generate and process TAXII, and will make it easier to test for conformance in the future.
· Tenet 2 - Minimise Size
I know of at least one implementer who is looking to transmit 1TB of data using STIX each day. XML is not a good transport mechanism for that volume of data. We need to ensure that the transmission of TAXII data is done as efficiently as possible. We need to make TAXII use the least amount of space on the wire as is physically possible, so that we fit the most amount of data in the least amount of space.

This naturally leads us towards a binary protocol implementation, as this is what all the major cloud based service providers who deal with voluminous amounts of data have moved to (Google, Microsoft, etc)
· Tenet 3 - Be Easily Extensible
This alludes to the fact that as hard as we might try, we will get things wrong in the initial implementation. Also, over time, the needs of the community will morph and change. We need to ensure that any solutions we devise are easily extensible such that we are able to add new functionality onto the previously released implementations without too much impact.

Please note - This only applies within minor releases i.e. within 2.x. I expect major releases to break stuff as per the TAXII versioning policy.
· Tenet 4 - Only Transmit Whats Necessary
In a way this is a by product of Tenet 2 - Minimise Size, but I thought it was important enough to describe separately. By enabling the ability to do things such as send partial STIX updates, keeping connections open (pipelining), defining 'hardcoded' locations to eliminate discovery phase, we can eliminate parts of the transmission, to help minimise the size of the data that needs to flow from point A to point B.
· Tenet 5 - Minimise Resource Usage
This is also fairly similar to Tenet 2, but is focused on the processing that the producer and consumer need to do at the each end of the communication. We need to ensure that we also make the data as easy to consume/translate/serialize etc as possible. This will allows implementations to scale to much greater heights than they can at present.

Comments and improvements welcomed (nay, required :) ).

Cheers

Terry MacDonald | STIX, TAXII, CybOX Consultant

M: +61-407-203-026
E: terry.macdonald@threatloop.com
W: www.threatloop.com

Disclaimer: The opinions expressed within this email do not represent the sentiment of any other party except my own. My views do not necessarily reflect those of my employers.

cti-taxii message