I would contend that having 2 properties with the same name in a single STIX object is a really bad idea, even if they are subcomponents of different extensions.
Unless you know that all of your subscribers properly implement STIX extensions, youâre asking for trouble. And I think we all know thatâs a pipe dream.
For example, my âchicken scratchâ python for parsing custom subcomponent properties that I shared the other day wouldnât work. You would need something much more elaborate.
Iâm struggling to see a use case where you would want to publish content containing a custom property developed by someone else, then turn around and also include your own version of that same custom property with the same property name
in the same object?
The use cases I see (using my custom scoring methodology example) are:
- As Chris-ISAC, I want to use the Chris-ISAC scoring methodology.
- As Chris-ISAC, I want to use the FS-ISAC scoring methodology.
- As Chris-ISAC, I want to have a hybrid scoring methodology, that starts with the FS-ISAC scoring methodology, but extends it with additional scoring properties (example â maybe add a âcriticalityâ
score).
Chris
From: Rich Piazza <rpiazza@mitre.org>
Sent: Tuesday, October 20, 2020 12:10 PM
To: Chris Ricard <cricard@fsisac.com>; cti@lists.oasis-open.org
Subject: Re: [cti] [EXT] [cti] STIX2.1 Extension Example - custom properties
I was more concerned if you wanted to use two top-level extensions in the same object, and they have a common name.
Maybe we should just not support that use case?
Rich
Folks,
Attached is an example of custom property reuse using Option 2 (top-level extensions).
In this example, two organizations (FS-ISAC and Chris-ISAC) both use custom Severity, Urgency, Credibility, and Risk scores. Their scoring methodologies are different.
The attached example demonstrates:
- Custom properties being re-used by an organization in multiple STIX SDO types (vulnerability and incident).
- The same custom property names being reused by multiple organizations.
Iâm not seeing why this doesnât work, or what limitations this incurs.
Chris
Allan,
From a business perspective, I need to add custom properties to an existing STIX object.
The spec, as I read it, provides two mechanisms for accomplishing this:
- Add them as sub-component properties under the âextensionsâ property (shown in
Example - Adding properties to an existing STIX object instance, and what I represented as Option 1)
- Add them as top-level properties (shown in
Example 7 - Adding properties at the top-level to an existing STIX object instance,
and what I represented as Option 2 )
As I see it, the âstix-extensionâ object is what makes the extensions reusable, not the âextensionsâ property. I donât see anything in Option 2 that would prevent or hamper reuse. The âextensionsâ property (in Option 1) DOES theoretically
give you the ability to use the same property name multiple times within a single STIX object instance, but I maintain that this is a really bad practice that shouldnât be encouraged.
Let me work on an example for reusing a custom property across STIX object instances, and across STIX object types. Hopefully Iâll have something to share tonite.
Chris
Chris - I disagree that we are providing 2 ways to do the same thing. The examples being shown might imply this but there is an important distinction that has been defined in the spec between what a) a top level extension is vs b) a sub-component
extension.
With many object modeling activities there is never a wrong or right but just best practices for certain uses.
A helpful reference to consider:
Object modeling often requires programmers to decide whether a construct represents an object or a sub component (another smaller object) of an object that is shared across objects.
A car can be an object where a wheel is an object that is shared across multiple cars. No one would probably model wheels as just a property of a top level car object. But they could if they wanted to. Itâs not a best practice in my mind
though.
The same choices apply to modeling cyber security data models.
The sub-component extension (option 1 in your email) is composition and importantly has many of the advantages of such in terms of reuse, modularity ...etc that are defined in that article above.
Whereas top-level extensions should be reserved for a) cases where backward compatibility is required on previously used custom properties or b) where the properties are adding vital properties that are considered fundamentally vital to
the object theyâre defined in. By definition neither case suggests such an extension should be used with other objects.
Finally the potential naming conflicts introduced by top level extensions are very real. This industry history already proves that multiple vendors have their own scoring systems but similar naming of their properties. If all those vendors
chose to use top level extensions it would be absolute chaos and the organizations worst impacted are the ones that aggregate data from multiple vendors and have the arduous task of normalizing it all. Sub component extension provides a very natural way to
both model this and support multiple vendors at the same time.
Final statement. Weâre very unlikely to get rid of both options defined in the spec for top level and sub component extension.
We will never get consensus on removing one of them so letâs not even try.
We need to move forward and adopt the changes so we can let the industry use the extension methods weâve defined.
Allan,
I we offer multiple options, then weâre back to there being multiple ways to do things. And vendors may or may not support them all.
If we have an option between Option 1 and Option 2, we would pick Option 2, since it much better meets our needs.
However, some vendors may only implement Option 1, since the spec says (paraphrasing) option 1 is preferred. And weâre right back where we were with STIX1.x, where 2 products can be following the standards, yet canât talk.
Specifically on Option 1 being easier for a STIX client to consume . Iâm just not seeing it. Especially if the âextension_propertiesâ was required for Option 2, then the STIX client would instantly know what properties are custom, and
what namespace they belong to. Then if âextension_propertiesâ isnât defined, or any custom property that isnât defined in âextension_propertiesâ, the STIX client could be free to discard, if it so chose.
On name-collisions: If you mean using the same property name multiple times (within separate extension namespaces), I agree that Option 1 would technically support this, while Option 2 would not, but I would maintain that someone doing
this is a really bad idea, and not something the spec should encourage.
Maybe we could get the Mitre folks who developed the python stix2 library to weigh in on implementation challenges for Option 1 vs Option 2, since a significant percent of STIX/TAXII-aware applications use the python libraries they developed?
I do think itâs important to separate out the use case of (1) TAXII servers that send/receive STIX content without actually caring what the STIX content is, from (2) STIX clients that act on the STIX content, and thus have to be able to
understand the STIX content.
Chris
One other thing.
The reason why we have both Option 1 and Option 2 in the standard is *because* we feel that its a choice for orgs to decide which method they prefer. Thereâs also reasons why both should exist.
So the debate over which one is better or not, is academic. The standard will support and we should just provide clear documentation on both and why one over the other should be used.
We (the TC) will never agree that only 1 of them should be done. I also think we *should* have both.
Thanks to Paul Patrick for sending me the attachments directly.
Chris - I reviewed your work and hereâs my feedback.
Option 1 - is sub-component extension and is the suggested best practice. There is no need to list the extension properties in this case as the expectation is that a JSON schema would define the extension including all properties.
To answer Richâs question why extension_properties is only for top-level extension. Its primarily to allow system that already have the schema for the STIX2.1 SO standard built-in to know what properties are added to the schema without
having to parse a 2nd schema. Whereas with a sub-component extension the expectation is that the org defining the extension would publish the complete schema for the extension.
Whether the TC thinks this is a worthwhile reason needs to be agreed. I prefer not duplicating JSON schema if we can. Hence why its currently defined the way it is.
Option 2 - is top-level extension.
I didnât bother reviewing Option 3.
As you assessed both Option 1 and Option 2 could work but Option 1 has multiple benefits over Option 2 particularly when it comes to name-collisions, modularityâetc.
As you state consuming Option 1 or Option 2 are equivalent. I disagree with this assessment. The reason is that in object modeling terms if you are a vendor that defines a table for standard STIX objects then the columns in that table (if
you assume a relational model) will be defined for the standard properties only. Any additional properties need to be modeled as either a) a separate table of objects that are then joined to the primary table b) a separate set of columns that are added for
all extensions that you support on those standard objects.
In my opinion the better approach to modeling this would be an object-model where you have the primary objects being the STIX standard object definitions and any related sub-components be modeled as objects themselves and connected in a
join (if relational db) or via a relationship (in an object db).
Iâm sure there are other ways to solve this but we need to think beyond the simple act of parsing the JSON. Hence the sub-component extension mechanism was designed with ingest to a db/data-store more easily.
Maybe remove me from your ban list? 😉
I never received Chrisâs original email. Checking spam folder also didnât yield it.
Can someone resend or forward me the attachments?
Chris - thank you for sharing the examples. Once I get them I can provide feedback if you would like some.
Thanks Chris. This is great!!
I think that the ability for you to quickly put this together shows that going from custom properties representation to extension properties representation is relatively trivial.
Here is my comments on your experiment, based on my understanding of the Extension proposal.
-
The extension object you have for Option 1 contains the property "extension_properties". The spec currently says that you only use that property for top level property extensions. Iâm not sure why that is. We should probably put in some text into the spec
explaining âwhyâ.
-
Top-level property extensions are there for backward compatibility. The spec indicates that the other type of property extensions is preferred. I know you mention that Option 2 makes your RESTful API more straightforward, but I think you could probably do
the processing you did in the client code as a subroutine that the RESTful API calls â making the use of either extension property method invisible. Of course, I donât know what your API looks like 😊
-
I donât see any advantage to Option 3, but maybe others will
Lead Cyber Security Engineer
On todayâs TC call, Rich asked folks who are using custom STIX extensions to kick the tires on the new extension proposal.
We (FS-ISAC) use custom properties on the STIX2.1 Vulnerability SDO, in order to make some custom vulnerability reporting available via a TAXII2.1 feed.
The intent is for the content to be STIX/TAXII-compliant (since itâs being published to our TAXII server), yet still easy for non-STIX/TAXII applications (such as a vulnerability management system that has no
idea what STIX and TAXII are) to be able to ingest it as a RESTful API.
Iâve attached 4 JSON files:
-
stix21-orig.json: This is an example of what we are currently publishing. Note that all of the âx-ctix-*â properties are custom top-level properties.
-
stix21-option1.json: This is my attempt to convert stix21-orig.json to the âOption 1â proposal (Adding properties to an existing STIX object instance)
-
stix21-option2.json: This is my attempt to convert stix21-orig.json to the âOption 2â proposal for adding custom properties to an existing STIX object (Adding properties at the top-level to an existing STIX object instance).
-
stix21-option3.json: This was just me taking a stab at what it would look like if there was an option to define the extensions in-line, rather than in a separate object. Obviously it would create additional, duplicative data, but I thought I would toss it
out there for consideration, since it would likely be easier for a STIX client to consume.
-
Publishing: It appears that we could publish our current custom vulnerability feed using either Option 1 and Option 2 (or Option 3).
-
Consuming:
-
For STIX/TAXII consumers, Option 1 and Option 2 seem equivalent to me. As long as the STIX client properly understands the spec, either should work. The one advantage I see to option 1 is that it allows you to overload the same custom property name in the
same SDO defined in different extensions (example, it seems that I could have an âFS-ISACâ risk property, and an âIT-ISACâ risk property in the same SDO, both named âriskâ. Iâm not sure why you would want that, though).
-
For NON-STIX/TAXII consumers (example - REST clients which are STIX unaware), Option 2 seems far superior. The REST client could treat all top-level properties the same, rather than having to understand that some top-level properties are native STIX properties,
while others are custom STIX properties that are embedded under the âextensionsâ property.
Also attached is a chicken-scratch python code (process_vulns-json.txt) to illustrate what Iâm talking about. The original JSON and the Option 2 JSON can be processed without any knowledge of STIX or understanding of STIX extensions.
However, Option 1 requires an understanding of STIX extensions, and some hand-waving to unpack the custom properties.
Hope this makes sense. Please let me know if I misunderstood anything.
Sr. Tech Engineer, FS-ISAC
|