Subject: Re: [EXT] Re: [cti] Questions about future changes to STIX
Thanks for the comments. I think there are lots of ways we can address this. Some of it is just in advisory notes to do what you suggest like saying their is a default. I am just trying to bring a lot of light to this topics to make sure everyone understands what is at risk and make sure assumptions do not cause us problems.
From a forward compatibility standpoint, you could just drop the properties you do not understand. However, that may break semantic meaning. Given the risks with Patterning and the amount of work we still have to do wit objects,
maybe we just need to be okay with each release being "standalone" for a while.
From: Piazza, Rich <email@example.com>
Sent: Friday, May 19, 2017 8:53:24 AM
To: Bret Jordan; firstname.lastname@example.org
Subject: [EXT] Re: [cti] Questions about future changes to STIX
I think you are a little pessimistic about backward compatibility and required properties. Couldn’t there be a default built into the newer release? Let’s say we make primary_motivation required on Threat Actors in 2.1 (just picking a property – not a suggestion ☺ ). In STIX 2.1, a default could be specified, and when parsing STIX 2.0 content that doesn’t contain that property – we could just use the default. I’m not saying this is always appropriate – but it could work in many cases – even if the default is “unknown”.
At this point, removing the stubs from STIX 2.0 would be a bad idea, IMHO.
For patterning, I think it depends more heavily upon the changes. The new ideas pattern expressions in STIX 2.1 could certainly be specified in ways that STIX 2.0 patterns are still valid.
Forward compatibility is a lot harder, because you kinda need to predict the future when implementing STIX 2.0. We have put some placeholders in the current spec – which is helpful – but not sufficient.
A recent discussion has come up about how to deal with future versions of STIX and what that means for backwards and forward compatibility.
In addition to this, we as a community need to decide how comfortable we are with changes that may break future versions. Just for clarification, OASIS has no rules about when a TC can do breaking changes as they view every release as standalone. So we as a TC need to decide, what does that means for us and when is it okay to make a breaking change and when is it not.
Some definitions to level set everyone.
STIX 2.1 data would be considered backward compatible with STIX 2.0 if every object that is valid under the 2.0 format is still valid in STIX 2.1. Meaning, a well-written STIX 2.1 parser can successfully parse STIX 2.0 content.
On the surface this seems like a good idea. However, this means that we can not add new required fields to objects in a later releases or change a previously defined optional field to now be required. Doing so would break the 2.1 parser if the content was not included in the original STIX 2.0 data (because it used to be optional). This could potentially also impact conformance clauses for future versions, so we would need to be very careful. This is potentially a big issue for the COA and Malware stubs that we put in to STIX 2.0. If we tried to maintain backwards compatibility, that means we could not add any required fields.
If we want to maintain backwards compatibility, we will need to remove the stub objects from STIX 2.0 and re-issue the CSD and public review.
What does this mean for patterning? Can patterning even be backwards compatible?
STIX 2.0 data would be forward compatible with STIX 2.1 if a product that can consume STIX 2.0 can "gracefully" process input designed for STIX 2.1+. Meaning, a well-written STIX 2.0 parser can successfully parse STIX 2.1+ content.
Content marked as required in STIX 2.0 can not be made optional in a later release.
What does this mean for patterning? Can patterning even be forwards compatible?
We will discuss this at the F2F, but I wanted to start the discussion on the email list so that eventually the TC can make a decision (since no decisions are made at the F2F). As I have thought about this from my perspective, I think the only thing we can really hope for is "forward compatibility". I personally do not think "backwards compatibility" is possible given the restriction on not adding additional required fields and what that will mean for Patterning.
But I am curious to hear and know everyone's thoughts on this. Hopefully we can get some discussion here over email, so we can have a better discussion at the F2F. How comfortable are you with future additions and changes? What is your appetite for changes that may be breaking either backward or forward compatibility?