TAXII, Past and Future

All,

As we transition TAXII into OASIS, I feel that sharing some of my technical thoughts about where TAXII has been and where TAXII might be going could be a useful input for this community.

For those of you that don’t know, I’ve been with TAXII since it started in 2012. During 2012, myself and my co-author (Charles Schmidt) had a notional set of requirements (which we never really wrote down), a name (TAXII), some stakeholders, and a goal (automated CTI transport). Charles and I spent the next few months intensively learning about the threat sharing landscape as it existed then, and attempted to create a solution that would map equally well into each of the models we observed. While I cannot recall all of the concepts and iterations we tried and scrapped, I do recall some key factors that will hopefully be useful to this group.

As best I can recall them, these are the factors that led to the design of TAXII 1.0 (released in April 2013):

# Three Sharing Models

We observed three distinct types of information exchange across security practitioners: Hub and Spoke, Peer to Peer, and Source/Subscriber. I feel compelled to note that these were not typically automated sharing groups; rather, they were typically isolated communities of security practitioners, and group membership was largely influenced by interpersonal relationships. This works well for the “stars” of the cybersecurity community, since they have lots of friends. It works less well for others.

During the design of TAXII 1.0, we weighed each of these models equally. That is, no sharing model’s requirements were given priority over another’s. This caused us to distill each of these models down to the essential automatable capabilities and attempt to find the overlap among them. These capabilities (Discovery, message push, message pull) were termed TAXII Services, and thus TAXII Services were born.

Over time, I think I have learned a few things about this arrangement. First among them is that the distinction of a TAXII Service might not be necessary. As you may know, a TAXII Service is hosted at a particular URL (and different TAXII Services can share a URL). I have personally observed implementers hosting all TAXII Services at the same URL and calling it their “TAXII URL”. To me this seems to be a useful simplification.

Second, I have learned that multiple architectures is more conceptually difficult to discuss than a single architecture. While in hindsight this might seem self-evident, TAXII did land on the requirement to support multiple architectures for a good reason. In my experience, people have tended to focus on the Hub and Spoke architecture as their mental starting point, and the other two (Source/Subscriber and Peer-to-Peer) can interestingly enough be described as variations of the Hub and Spoke architecture. Source/Subscriber is a Hub and Spoke architecture where the spokes are receive-only; Peer-to-Peer is a Hub and Spoke architecture where each peer operates their own “Hub”, and the “Hubs” federate information among each other. I consider this to be one of the more important lessons learned, because when discussions use a single architecture (even if that architecture has variants and/or options) discussion can focus on the roles of components within that architecture and how they interoperate, which is a higher level and more productive discussion than “What TAXII Services do you implement?”.

# Protocol Agnostic, Format Agnostic

During the learning and discovery phase predating TAXII 1.0, the security practitioners and developers we talked to had varying viewpoints on how they wanted to exchange information. Some wanted to use SMTP because email infrastructure exists everywhere. Others wanted to use HTTP because everyone runs a website. Others wanted to use Web Services / SOAP because that’s what they were most experienced with and would have the easiest time building. We saw the same thing with data formats – some wanted JSON, some wanted XML, and there were some other opinions in there that I can no longer recall.

TAXII 1.0 solved this by defining the abstract representation of message concepts and transport concepts in the TAXII Services Specification, and leaving protocol and format specific implementations to “binding” specifications that bind TAXII concepts to particular technologies. While this is a good idea in theory, it has some limitations in practice. Chief among them, in my opinion, is that it prevents you from taking advantage of native aspects of a particular protocol.

For instance, some of you know that a TAXII Message has headers fields – TAXII’s requirement to be protocol agnostic meant that we couldn’t rely on the headers of a particular protocol, and that in order to guarantee there would be some usable header concept, we had to design them into the TAXII Messages. We couldn’t, for instance, say something along the lines of “HTTP has extensible headers, so just use that”.

It’s been a little over three years since TAXII 1.0 was released, and I personally haven’t seen anyone use anything other than HTTP for TAXII. I can rationalize this with a simple argument: Who is going to spend the time to define and implement a binding spec (e.g., SMTP) that basically guarantees non-interoperability with existing implementations? Nobody. I think this is a positive development, since in the future it frees us to consider supporting only one protocol, and taking advantage of that protocol’s features instead of ignoring them because ‘those features are not guaranteed to be available in all protocols’.

Hopefully those thoughts provide some insight into where TAXII is and how it got here. Feel free to get a cup of coffee, take a mental break, and reflect. This is a heavy topic that requires a bit of mental processing. The remainder of the email focuses on where TAXII might go. The remainder of this email reflects my opinions and should not be misinterpreted as facts.

# Message Oriented vs. Resource Oriented

Message vs. Resource is a fundamental tension point within the design of TAXII, and it’s taken me three years to realize it. I’ll begin by offering my perspective on what these two terms mean as a mechanism to get everyone on the same page (well, on my page anyway =) ):

* Message Oriented – A message is a blob of data (a STIX document, file, a JSON message, etc) that will be delivered to some set of recipients, nominally with some set of delivery guarantees (e.g., exactly once, message acknowledgement). Message oriented protocols are things like SMTP, XMPP (chat), AMQP, et al.

* Resource Oriented – A resource is also a blob of data, but it persists at a given location (URL) and delivery is not tracked. Recipients can request a resource multiple times, and that resource might change over time (e.g., google.com). Resources can be cached. HTTP is by far the most prevalent resource oriented protocol.

Conflating these concepts is the fact that quite a few applications do messaging over HTTP. Think about your favorite web mail provider – email is a messaging concept and it’s provided to you, the end user, over a resource oriented protocol (HTTP). Most end users do not care about this distinction, but I think it’s a worthwhile topic for protocol designers (us).

This tension exists most clearly in the way TAXII Data Collections behave. There are messages in Data Collections (nominally identified with a Timestamp Label in a TAXII Data Feed), but some fundamental messaging concepts are not implemented in TAXII: routing algorithms (e.g., direct, fanout), ack/nack concepts, and removal of a message from a message queue once it’s been retrieved. Data Collections are resources in other ways – the same request will often result in the same response (but not always, and is not guaranteed to be), but certain fundamental resource concepts are missing: everything is an HTTP POST (there should probably be some HTTP GETs, and you’ll never see an HTTP 304 (not modified) as a response to a TAXII request. What I mean to say is that TAXII 1.0 and TAXII 1.1 implement both messaging and resource concepts, and that has caused some interesting tradeoffs that weren’t apparent in the beginning.

My opinion is that threat information sharing is fundamentally a messaging application. An indicator creator will want to send that indicator to a set of recipient(s) exactly once, and they will want an acknowledgement when that indicator is accepted by the recipient. When feedback is delivered about an indicator (sightings, etc), that feedback is not best constructed as a web resource to be requested at the whim of a recipient, it is a message to be delivered to a specific party with specific delivery requirements (e.g., exactly once). This is not to say HTTP should not be used – clearly messaging applications can be successful using HTTP. Rather, I encourage the group to recognize the distinction and consider it when thinking about the future of TAXII.

# Interfaces: Machine? Human?

A factor that’s easy to lose sight of is that cyber security, as a field, is not even 30 years old (if you count 1988’s Morris Worm as the starting point). As a result, the cyber security profession is – by the population at large – poorly defined, poorly understood, and in many cases undervalued.

My favorite comparison for maturity from an interoperability perspective is DLNA (http://www.dlna.org/) – DLNA is a group that defines architecture, components, and technology in order to make your whole living room (TV, etc) plug-and-play. DLNA is why you can stream music and video from your computer to your TV. We, cyber security professionals, do not have a widely accepted architecture, component definitions, or plug-and-play technologies for our security products. Let me make this point abundantly clear: Our technology (from a connectedness/plug-and-play perspective) is less mature than the dumb boxes in your living room. When we design technologies, the fact that nothing can be assumed about an organization’s architecture is a critical factor to account for (btw, this is a completely separate area that I am also interested in trying to solve).

What does this mean for TAXII? I think it means that, even for a well-defined interface, we can’t expect there will be a corresponding and well defined machine consumer of that interface. For the near term, this means security teams will be integrating with these interfaces themselves – security professionals will have to glue their systems together until the day comes when our security products become plug and play. Therefore, we need to make sure the systems we build can be glued together by your typical security team, which may or may not include engineers.

# Conclusion

The intent of this email is to stir the collective though process of this subcommittee. I wanted to describe some of the old assumptions and requirements so that they can be challenged and, as necessary, overturned. I wanted to describe some of my thoughts on the future of TAXII, primarily to help broaden the thinking of the group beyond what TAXII “is” into what TAXII “could be”. Hopefully I have achieved that goal, even if that means you vehemently disagree with part of what I said.

Thank you.

Mark Davidson

MDavidson@MITRE.org

(781) 271-3703

Lead Cyber Security Engineer

The MITRE Corporation

cti-taxii message