Re: [cti-users] Questions about TAXII 2.1 Envelopes vs. STIX Bundles

[Adam Pearce <adam.pearce@xorsecurity.com>]

Bret,

Definitely helps! Appreciate the clarity and the response, and it aligns with my expectations. I definitely understand the difficulty of capturing all of this as verbiage and covering all of the edge cases.

Best regards,

Adam

On Tue, May 12, 2020 at 1:23 PM Bret Jordan <bret.jordan@broadcom.com> wrote:

Adam,

I agree with you that a correctly implemented JSON parser should handle this correctly. However, writing allowances for that in normative prose text is really hard and usually, from my experience, causes long-term problems that are unforeseen. If we would have done that, it would have put us right back in to the problems of media-types that we were trying to also solve. This is why we the text was written the way it was.Â

As I wrote the first implementation of this, to make sure it would work, I found that the design for the envelope was super easy. I just reused my code for bundle and then NULL-ed out the two properties before I serialized it.

The other reason we wrote it this way is to ensure implementers do not think they can just use a bundle for TAXIi straight across the board. Since the bundle will break when the server paginates data coming back. If an implementation thinks they can just use the bundle, then they will not be able to paginate data correctly. Now in your solution you could just add support for the extra TAXII properties to the bundle and just have one âthingâ in code. But that is an implementation decision. Meaning, you could have a single âcontainerâ object and then tweak it as part of serialization. But that is an implementation choice.Â

The last thing to be mindful of is, at some point we will enable digital signatures on this content. It is not yet clear how we will do this. But you could run into problems as consumers compute the signature post consumption.

Example: A producer reuses the STIX bundle for TAXII and does not remove the STIX properties. They compute the digital signature for the content based on JCS. The consumer reads the data off the wire and parses it. Now as you said I-JSON says you should ignore what you do not understand, so certain properties get dropped on the floor. Now the consumer tries to recompute the digital signature, if the producer did not remove the extra properties before it did the signature then the signatures will not match.

So just be careful. We wrote the normative prose text the way we did as we were trying to prevent all of these corner cases from causing interoperability problems. We learned from the mistakes of STIX 1.x and TAXII 1.x and are trying to avoid them.

Does this help?Â

Thanks,

Bret

PGP Fingerprint:Â63B4 FC53 680A 6B7D 1447 ÂF2C0 74F8 ACAE 7415 0050

"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."

On May 12, 2020, at 10:49 AM, Adam Pearce <adam.pearce@xorsecurity.com> wrote:

Bret,

Makes total sense and I can see why you took that approach!

The point I am trying to raise is that there are a few reasons why I think sending a Bundle in TAXII 2.1 would still satisfy the requirement that the content "must be an envelope".

Primarily, an Envelope consists of only three optionalÂfields, one of which is the 'objects' field.

Secondarily, the I-JSON specification that TAXII 2.1 references for Serialization has a 'Must Ignore' property (which I've attached at the bottom of this e-mail).

That means the following is an Envelope:
{
 Âobjects: [...]
}

If you add an 'id' and a 'type', it can be interpreted as a STIX Bundle.
{
 Âid: "bundle--eac61de9-4722-4551-b2aa-fc79ac69f1af"

 Âtype: "bundle"
 Âobjects: [...]
}

However, consider for a moment, strictly operating from the TAXII 2.1 point-of-view, and forgetting that TAXII 2.0 exists, or that STIX 2.X exists.

(1) You've implemented the I-JSON must ignore property. (2) You've implemented parsing of incoming request bodies as TAXII 2.1 Envelopes.

You see the following:

{
 Âid: "bundle--eac61de9-4722-4551-b2aa-fc79ac69f1af"

 Âtype: "bundle"
 Âobjects: [...]
}

Based on (1) and (2), you ignore 'id', you ignore 'type', you parse 'objects', all of which are STIX Objects and you have a valid Envelope. From your point-of-view, the "thing" you're seeing isÂan Envelope.

This is the root of my inquiry. Submitting Bundles to the Add Objects API in TAXII 2.1 should be completely acceptable based on the above logic. I don't think this warrants any changes to the specification, I just think according to the spec definitions, if a TAXII 2.1 Server can process Envelopes and implements I-JSON guidance, it should be able to process Bundles.

4.2. Must-Ignore Policy
 ÂIt is frequently the case that changes to protocols are required
 Âafter they have been put in production. Protocols that allow the
 Âintroduction of new protocol elements in a way that does not disrupt
 Âthe operation of existing software have proven advantageous in
 Âpractice.
 ÂThis can be referred to as a "Must-Ignore" policy, meaning that when
 Âan implementation encounters a protocol element that it does not
 Ârecognize, it should treat the rest of the protocol transaction as if
 Âthe new element simply did not appear, and in particular, the
 Âimplementation MUST NOT treat this as an error condition. ÂThe
 Âconverse "Must-Understand" policy does not tolerate the introduction
 Âof new protocol elements, and while this has proven necessary in
 Âcertain protocol designs, in general it has been found to be overly
 Ârestrictive and brittle.
 ÂA good way to support the use of Must-Ignore in I-JSON protocol
 Âdesigns is to require that top-level protocol elements must be JSON
 Âobjects, and to specify that members whose names are unrecognized
 ÂMUST be ignored.

On Tue, May 12, 2020 at 12:33 PM Bret Jordan <bret.jordan@broadcom.com> wrote:

Adam,Â

Thanks for submitting your question. Let me try and answer it. In the TAXII 2.0 release we discovered a few issues when accepting STIX content directly into a TAXII resource. Further, one we fixed pagination we found we needed extra properties on the âcontainerâ. By adding these extra properties we found that TAXII was putting requirements on STIX in a very weird way which would cause all kinds of problems when we try to keep version in sync. This caused problems with content negotiations as well. Further, many in the community asked that we separate the two and make sure the each spec did not put constraints on the other. After doing this, a lot of things got easier.

So yes, the TAXII envelope was designed to mimic the STIX bundle. This was done by design to make it super easy for code to be adapted. But as you will see, the TAXII envelope has extra properties that are required for pagination. This also means that the media types coming in to TAXII are now always TAXII. This prevents a lot of weird code paths that were problematic for early adopters of 2.0. It should now be much easier to code up support for TAXII 2.1. Â

In STIX and TAXII 2.1 the STIX bundle might be used for sending STIX content outside of TAXII. But when using TAXII, the TAXII envelope must be used. This also means that even when getting a single object or sending a single object, the TAXII envelop is used. This will give predictability and interoperability on the client and server and greatly reduces the amount of code needed to process inbound and outbound data.Â

I hope this clarifies things. If not, please let me know.Â

Thanks,

Bret

PGP Fingerprint:Â63B4 FC53 680A 6B7D 1447 ÂF2C0 74F8 ACAE 7415 0050

"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."

On May 12, 2020, at 6:05 AM, Adam Pearce <adam.pearce@xorsecurity.com> wrote:

All,

It appears that TAXII 2.1 Envelopes were introduced in such a way that allows their fields to be backwards compatible with STIX Bundles, so that there is compatibility between TAXII 2.0 and TAXII 2.1 clients.Â

However, there is some ambiguity surrounding the explicitness of the TAXII 2.1 specification. It mentions that

When requesting STIX 2 content, that content will always be delivered in a TAXIIÂenvelopeÂeven if there only one object returned.

And

When adding STIX 2 content, clientsÂMUSTÂdeliver all objects in a TAXIIÂenvelope.Â

Â

If STIX BundlesÂareÂTAXII envelopes, then there is no ambiguity.

However, there is a slight semantic argumentÂto be made around "is" or "is not". My interpretation would be that BundlesÂareÂEnvelopes, but EnvelopesÂare notÂBundles. Envelopes do not include the required 'type' and 'id' fields to be interpreted as Bundles. However, an Envelope has only 3 (optional) properties, one of which is 'objects'. So a Bundle could be interpreted as an Envelope. This is further supported by the 'Must Ignore' property of I-JSON for "unrecognized fields" (https://tools.ietf.org/html/rfc7493, Sec. 4.2).

Is this interpretation correct? Furthermore, is allowing a Bundle for the 'Add Objects' API in TAXII 2.1 acceptable?

Best regards,

Adam Pearce