OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.


Help: OASIS Mailing Lists Help | MarkMail Help

cti message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]

Subject: Re: [cti] [EXT] [cti] STIX2.1 Extension Example - custom properties

I believe duplicate keys are explicitly illegal in STIX 2.1. It adopts RFC7493 as normative. That RFC disallows duplicate keys. 

On Oct 20, 2020, at 12:50 PM, Rich Piazza <rpiazza@mitre.org> wrote:


Duplicate keys (at the same level) arenât explicitly illegal in JSON, but I would hope they would be in STIX â although I donât know if there is a normative statement that says that.


I guess I see, maybe in the future, a common repository for STIX extension definitions.  Why reinvent an extension definition, if there is one defined to serve your needs that others are already using?  If you choose two from the repository, they may have property names in common.  Or you want to use one from the repository, but you already have your own extension definition that contains one of the property names.


Maybe it is an edge case â which the spec already says you should just avoid.


From: Chris Ricard <cricard@fsisac.com>
Date: Tuesday, October 20, 2020 at 12:33 PM
To: Rich Piazza <rpiazza@mitre.org>, "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: RE: [cti] [EXT] [cti] STIX2.1 Extension Example - custom properties


I would contend that having 2 properties with the same name in a single STIX object is a really bad idea, even if they are subcomponents of different extensions.


Unless you know that all of your subscribers properly implement STIX extensions, youâre asking for trouble.  And I think we all know thatâs a pipe dream.


For example, my âchicken scratchâ python for parsing custom subcomponent properties that I shared the other day wouldnât work.  You would need something much more elaborate.


Iâm struggling to see a use case where you would want to publish content containing a custom property developed by someone else, then turn around and also include your own version of that same custom property with the same property name in the same object?


The use cases I see (using my custom scoring methodology example) are:

  1. As Chris-ISAC, I want to use the Chris-ISAC scoring methodology.
  2. As Chris-ISAC, I want to use the FS-ISAC scoring methodology.
  3. As Chris-ISAC, I want to have a hybrid scoring methodology, that starts with the FS-ISAC scoring methodology, but extends it with additional scoring properties (example â maybe add a âcriticalityâ score).




From: Rich Piazza <rpiazza@mitre.org>
Sent: Tuesday, October 20, 2020 12:10 PM
To: Chris Ricard <cricard@fsisac.com>; cti@lists.oasis-open.org
Subject: Re: [cti] [EXT] [cti] STIX2.1 Extension Example - custom properties


NOTICE: External Email - Sender is rpiazza@mitre.org

I was more concerned if you wanted to use two top-level extensions in the same object, and they have a common name.

Maybe we should just not support that use case?




From: <cti@lists.oasis-open.org> on behalf of Chris Ricard <cricard@fsisac.com>
Date: Tuesday, October 20, 2020 at 11:42 AM
To: "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: RE: [cti] [EXT] [cti] STIX2.1 Extension Example - custom properties




Attached is an example of custom property reuse using Option 2 (top-level extensions).


In this example, two organizations (FS-ISAC and Chris-ISAC) both use custom Severity, Urgency, Credibility, and Risk scores.  Their scoring methodologies are different.


The attached example demonstrates:

  1. Custom properties being re-used by an organization in multiple STIX SDO types (vulnerability and incident).
  2. The same custom property names being reused by multiple organizations.

Iâm not seeing why this doesnât work, or what limitations this incurs.




From: Chris Ricard
Sent: Sunday, October 18, 2020 4:35 PM
To: aa tt <atcyber1000@gmail.com>
Cc: cti@lists.oasis-open.org
Subject: RE: [cti] [EXT] [cti] STIX2.1 Extension Example - custom properties




From a business perspective, I need to add custom properties to an existing STIX object.


The spec, as I read it, provides two mechanisms for accomplishing this:

  • Add them as sub-component properties under the âextensionsâ property (shown in Example - Adding properties to an existing STIX object instance, and what I represented as Option 1)
  • Add them as top-level properties (shown in Example 7 - Adding properties at the top-level to an existing STIX object instance,  and what I represented as Option 2 )

As I see it, the âstix-extensionâ object is what makes the extensions reusable, not the âextensionsâ property.  I donât see anything in Option 2 that would prevent or hamper reuse. The âextensionsâ property (in Option 1) DOES theoretically give you the ability to use the same property name  multiple times within a single STIX object instance, but I maintain that this is a really bad practice that shouldnât be encouraged.


Let me work on an example for reusing a custom property across STIX object instances, and across STIX object types.  Hopefully Iâll have something to share tonite.



From: aa tt <atcyber1000@gmail.com>
Sent: Sunday, October 18, 2020 2:26 PM
To: Chris Ricard <cricard@fsisac.com>
Cc: cti@lists.oasis-open.org
Subject: Re: [cti] [EXT] [cti] STIX2.1 Extension Example - custom properties


NOTICE: External Email - Sender is atcyber1000@gmail.com

Chris - I disagree that we are providing 2 ways to do the same thing. The examples being shown might imply this but there is an important distinction that has been defined in the spec between what a) a top level extension is vs b) a sub-component extension.


With many object modeling activities there is never a wrong or right but just best practices for certain uses.


A helpful reference to consider:


Object modeling often requires programmers to decide whether a construct represents an object or a sub component (another smaller object) of an object that is shared across objects.


A car can be an object where a wheel is an object that is shared across multiple cars. No one would probably model wheels as just a property of a top level car object. But they could if they wanted to. Itâs not a best practice in my mind though. 


The same choices apply to modeling cyber security data models.


The sub-component extension (option 1 in your email) is composition and importantly has many of the advantages of such in terms of reuse, modularity ...etc that are defined in that article above.


Whereas top-level extensions should be reserved for a) cases where backward compatibility is required on previously used custom properties or b) where the properties are adding vital properties that are considered fundamentally vital to the object theyâre defined in. By definition neither case suggests such an extension should be used with other objects.


Finally the potential naming conflicts introduced by top level extensions are very real. This industry history already proves that multiple vendors have their own scoring systems but similar naming of their properties. If all those vendors chose to use top level extensions it would be absolute chaos and the organizations worst impacted are the ones that aggregate data from multiple vendors and have the arduous task of normalizing it all. Sub component extension provides a very natural way to both model this and support multiple vendors at the same time.


Final statement. Weâre very unlikely to get rid of both options defined in the spec for top level and sub component extension. 


We will never get consensus on removing one of them so letâs not even try. 


We need to move forward and adopt the changes so we can let the industry use the extension methods weâve defined.




On Oct 18, 2020, at 10:25 AM, Chris Ricard <cricard@fsisac.com> wrote:



I we offer multiple options, then weâre back to there being multiple ways to do things.  And vendors may or may not support them all.


If we have an option between Option 1 and Option 2, we would pick Option 2, since it much better meets our needs.


However, some vendors may only implement Option 1, since the spec says (paraphrasing) option 1 is preferred.  And weâre right back where we were with STIX1.x, where 2 products can be following the standards, yet canât talk.


Specifically on Option 1 being easier for a STIX client to consume .  Iâm just not seeing it.  Especially if the âextension_propertiesâ was required for Option 2, then the STIX client would instantly know what properties are custom, and what namespace they belong to.  Then if âextension_propertiesâ isnât defined, or any custom property that isnât defined in âextension_propertiesâ, the STIX client could be free to discard, if it so chose. 


On name-collisions:  If you mean using the same property name multiple times (within separate extension namespaces), I agree that Option 1 would technically support this, while Option 2 would not, but I would maintain that someone doing this is a really bad idea, and not something the spec should encourage.


Maybe we could get the Mitre folks who developed the python stix2 library to weigh in on implementation challenges for Option 1 vs Option 2, since a significant percent of STIX/TAXII-aware applications use the python libraries they developed?


I do think itâs important to separate out the use case of (1) TAXII servers that send/receive STIX content without actually caring what the STIX content is, from (2) STIX clients that act on the STIX content, and thus have to be able to understand the STIX content.





From: aa tt <atcyber1000@gmail.com>
Sent: Friday, October 16, 2020 2:55 PM
To: Chris Ricard <cricard@fsisac.com>
Cc: cti@lists.oasis-open.org
Subject: Re: [cti] [EXT] [cti] STIX2.1 Extension Example - custom properties


NOTICE: External Email - Sender is atcyber1000@gmail.com

One other thing.


The reason why we have both Option 1 and Option 2 in the standard is *because* we feel that its a choice for orgs to decide which method they prefer. Thereâs also reasons why both should exist. 


So the debate over which one is better or not, is academic. The standard will support and we should just provide clear documentation on both and why one over the other should be used. 


We (the TC) will never agree that only 1 of them should be done. I also think we *should* have both.



On Oct 16, 2020, at 11:51 AM, aa tt <atcyber1000@gmail.com> wrote:


Thanks to Paul Patrick for sending me the attachments directly.


Chris - I reviewed your work and hereâs my feedback.


Option 1 - is sub-component extension and is the suggested best practice. There is no need to list the extension properties in this case as the expectation is that a JSON schema would define the extension including all properties.


To answer Richâs question why extension_properties is only for top-level extension. Its primarily to allow system that already have the schema for the STIX2.1 SO standard built-in to know what properties are added to the schema without having to parse a 2nd schema. Whereas with a sub-component extension the expectation is that the org defining the extension would publish the complete schema for the extension.


Whether the TC thinks this is a worthwhile reason needs to be agreed. I prefer not duplicating JSON schema if we can. Hence why its currently defined the way it is.


Option 2 - is top-level extension.


I didnât bother reviewing Option 3.


As you assessed both Option 1 and Option 2 could work but Option 1 has multiple benefits over Option 2 particularly when it comes to name-collisions, modularityâetc.


As you state consuming Option 1 or Option 2 are equivalent. I disagree with this assessment. The reason is that in object modeling terms if you are a vendor that defines a table for standard STIX objects then the columns in that table (if you assume a relational model) will be defined for the standard properties only. Any additional properties need to be modeled as either a) a separate table of objects that are then joined to the primary table b) a separate set of columns that are added for all extensions that you support on those standard objects.


In my opinion the better approach to modeling this would be an object-model where you have the primary objects being the STIX standard object definitions and any related sub-components be modeled as objects themselves and connected in a join (if relational db) or via a relationship (in an object db).


Iâm sure there are other ways to solve this but we need to think beyond the simple act of parsing the JSON. Hence the sub-component extension mechanism was designed with ingest to a db/data-store more easily.



On Oct 16, 2020, at 10:24 AM, Chris Ricard <cricard@fsisac.com> wrote:


Maybe remove me from your ban list?  ð


Your email filters may not like one or more of the attachments.  You can access the original email (with attachments) from the OASIS site:  https://www.oasis-open.org/apps/org/workgroup/cti/email/archives/202010/msg00046.html





From: aa tt <atcyber1000@gmail.com> 
Sent: Friday, October 16, 2020 1:12 PM
To: Rich Piazza <rpiazza@mitre.org>
Cc: Chris Ricard <cricard@fsisac.com>; cti@lists.oasis-open.org
Subject: Re: [cti] [EXT] [cti] STIX2.1 Extension Example - custom properties


NOTICE: External Email - Sender is atcyber1000@gmail.com

I never received Chrisâs original email. Checking spam folder also didnât yield it.


Can someone resend or forward me the attachments?


Chris - thank you for sharing the examples. Once I get them I can provide feedback if you would like some.



On Oct 16, 2020, at 9:57 AM, Rich Piazza <rpiazza@mitre.org> wrote:


Hi Chris,


Thanks Chris.  This is great!!


I think that the ability for you to quickly put this together shows that going from custom properties representation to extension properties representation is relatively trivial.


Here is my comments on your experiment, based on my understanding of the Extension proposal.


  • The extension object you have for Option 1 contains the property "extension_properties".  The spec currently says that you only use that property for top level property extensions.  Iâm not sure why that is.  We should probably put in some text into the spec explaining âwhyâ.
  • Top-level property extensions are there for backward compatibility.  The spec indicates that the other type of property extensions is preferred. I know you mention that Option 2 makes your RESTful API more straightforward, but I think you could probably do the processing you did in the client code as a subroutine that the RESTful API calls â making the use of either extension property method invisible.  Of course, I donât know what your API looks like ð
  • I donât see any advantage to Option 3, but maybe others will


Rich P.



Rich Piazza

Lead Cyber Security Engineer

The MITRE Corporation




From: <cti@lists.oasis-open.org> on behalf of Chris Ricard <cricard@fsisac.com>
Date: Friday, October 16, 2020 at 12:52 AM
To: "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: [EXT] [cti] STIX2.1 Extension Example - custom properties




On todayâs TC call, Rich asked folks who are using custom STIX extensions to kick the tires on the new extension proposal.


We (FS-ISAC) use custom properties on the STIX2.1 Vulnerability SDO, in order to make some custom vulnerability reporting available via a TAXII2.1 feed.


The intent is for the content to be STIX/TAXII-compliant (since itâs being published to our TAXII server), yet still easy for non-STIX/TAXII applications (such as a vulnerability management system that has no idea what STIX and TAXII are) to be able to ingest it as a RESTful API.

Iâve attached 4 JSON files:

  1. stix21-orig.json:  This is an example of what we are currently publishing.  Note that all of the âx-ctix-*â properties are custom top-level properties.
  2. stix21-option1.json:  This is my attempt to convert stix21-orig.json to the âOption 1â proposal (Adding properties to an existing STIX object instance)
  3. stix21-option2.json: This is my attempt to convert stix21-orig.json to the âOption 2â proposal for adding custom properties to an existing STIX object (Adding properties at the top-level to an existing STIX object instance).
  4. stix21-option3.json:  This was just me taking a stab at what it would look like if there was an option to define the extensions in-line, rather than in a separate object.  Obviously it would create additional, duplicative data, but I thought I would toss it out there for consideration, since it would likely be easier for a STIX client to consume.



My take-aways:

  1. Publishing: It appears that we could publish our current custom vulnerability feed using either Option 1 and Option 2 (or Option 3).
  2. Consuming: 
    1. For STIX/TAXII consumers, Option 1 and Option 2 seem equivalent to me.  As long as the STIX client properly understands the spec, either should work.  The one advantage I see to option 1 is that it allows you to overload the same custom property name in the same SDO defined in different extensions (example, it seems that I could have an âFS-ISACâ risk property, and an âIT-ISACâ risk property in the same SDO, both named âriskâ.  Iâm not sure why you would want that, though).
    2. For NON-STIX/TAXII consumers (example - REST clients which are STIX unaware), Option 2 seems far superior.  The REST client could treat all top-level properties the same, rather than having to understand that some top-level properties are native STIX properties, while others are custom STIX properties that are embedded under the âextensionsâ property.


Also attached is a chicken-scratch python code (process_vulns-json.txt) to illustrate what Iâm talking about.   The original JSON and the Option 2 JSON can be processed without any knowledge of STIX or understanding of STIX extensions. However, Option 1 requires an understanding of STIX extensions, and some hand-waving to unpack the custom properties.


Hope this makes sense.  Please let me know if I misunderstood anything.


Chris Ricard

Sr. Tech Engineer, FS-ISAC

work: +1 571-446-3888

cell: +1 703-673-8621





[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]