As an example, our data markings processing rules need requirements for what to do when you can’t understand the marking model (you only do TLP, they send you something different) or marking application (you only do level 1, they send you level 2).
It sounds like people are on the same page here…unlike our STIX 1.2 schemas (which failed validation on extra fields), we can be more open to “extra” content in the documents.
Whoa, whoa, whoa! Major misunderstanding here.
I was not suggesting that we should chose what to do in all cases when confronted with JSON that you do not understand. Terry here gives a perfect example where ignoring fields is plain wrong. In fact, the ‘feature’ of JSON parsers silently dropping
keys it does not understand is a very, very, very serious, potentially fatal, flaw.
If I need you do understand a field to understand my document, and you do not understand that field, in fact, you cannot understand or, worse, will misunderstand, my message. In fact, imagine:
“delightful” : “names”,
“delete” : “*”
You don’t understand ‘delightful’, but you do understand ‘delete’. Wow, you just deleted your whole data base. That sucks - all I wanted you do to was take out the delightful names in your ACL list. You
would have known that if only you knew what my ‘delightful’ message was about.
What I am advocating for is:
- Specify what to do in general when you get a field you do not understand
- Consensus seems to be ignore
- That is a reasonable default action as it is more likely to interoperate into the future
- Have a mechanism for specifying in specific when you need the receiver to understand either the whole message or a particular message element
- RFC 3459 does this for email
I was meaning that if we had this object:
And the object was corrupted in transit and became
Having the illegal fields ignored means that the vocab field is invalid. The whole name_ext object is invalid as all the required fields aren’t present, so that
ultimately the only bit that gets through the parser is:
Which is different to what the producer was meaning to have.
I know it is an edge case and one that only warrants a small amount of thought, but we will need to ensure that the ambiguity is removed in some way in a way
that negates this from happening. It could be as simple as saying exactly what Bret suggested – that the default vocab chosen must be as close approximation as possible to the custom vocab selection.
Senior STIX Subject Matter Expert
SOLTRA | An FS-ISAC and DTCC Company
Not sure I follow. If your tool does not support custom vocabs then you will not have the data in them. I would suggest as a best practice people should always, when possible, pick a fallback option instead of using "other".
Sent from my Commodore 64
This could be problematic for the vocabs extensions if the _ext object is used (custom vocab) and the extension object field is corrupted and ignored. It means that the logic will not see that the extension object exists, and will just use the value selected
within the default vocab...
Maybe we could avert this by specifying a 'custom-vocab' option for all enums to explicitly specify that there is an extension to the custom vocab?
I can see this happening a lot. And the nice thing is most JSON parsers will just silently drop fields it does not understand. I am thinking it might actually be hard to figure out if there is in fact extra data there that it does not know what to do with.
The way I see this working is vendors communications in side a closed system (network / eco-system) may use all kinds of extra stuff. Then at the board TAXII server, it would get stripped out.
Director of Security Architecture and Standards | Office of the CTO
PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."
As someone who is *extremely likely* to have to make use of both provisional portions of future versions of our specifications and also proprietary enhancements for internal use - I would strongly advocate for option (b) (ignore the field). We need to be able
to add fields to the JSON and have them be silently ignored if unsupported by the receiver.
STSM, Product Architect, Security Intelligence, IBM Security Systems
Without data, all you are is just another person with an opinion - Unknown
<graycol.gif>Eric Burger ---01/29/2016 08:28:38 AM---There are only three choices if one is missing or has an odd MTI field: Fail the transaction / rejec
From: Eric Burger <Eric.Burger@georgetown.edu>
Date: 01/29/2016 08:28 AM
Subject: [cti] Missing MTI - what to do?
Sent by: <email@example.com>
There are only three choices if one is missing or has an odd MTI field:
Fail the transaction / reject to document
Ignore the field
Fix the field if you can
I would offer we say what we mean. There are two places to do that. The first is in the spec itself. For example, if “id” is missing, I think we would say in the spec the serialization is malformed and return an error
(or silently go away). The second is to tag the transaction, perhaps at the TAXII level, with what you want to have happen. For example, if I have private extensions, I can tag them as something you can freely ignore or drop. Conversely, if I have extensions
or optional fields that you really need to understand to process the message, I can tag them as something that if you do not recognize them, fail the document so I know you do not know.
For an example of this working in email system gateways, see https://tools.ietf.org/html/rfc3459
On Jan 27, 2016, at 10:57 AM, Paul Patrick <firstname.lastname@example.org>
As a side note, I’ve not seen a statement about what the appropriate behavior should be when a JSON MTI compliant consumer receives a document with a field it doesn’t understand (maybe due to a typo or somebody tried
to make a private extension). But if the specified behavior was to ignore the field rather than fail the processing of the entire document, then under the situation where a JSON-LD language binding document was inadvertently sent to a consumer that only excepts
JSON MTI language binding, the behavior would still allow the document to be processed as a compliant JSON MTI document since the @nodes and @fields would be ignored or skipped.