OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

cti-stix message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]


Subject: Re: [cti-stix] Applying data markings


  1. Agreed, an example of marking scope would really help.
  2. Good point, we need to specify error scenarios like you get at in (4) in the conformance requirements.
  3. Yep.
  4. Yes, though I would say the actual error conditions are slightly different:
    1. The _expression_ is invalid (incorrect syntax)
    2. The _expression_ doesn’t select anything (your case is a subset of this)
  5. For this writeup I assumed that a markable object cannot be embedded in a markable object based on us moving to reference-only. If we end up with another decision there we need to come back to this and handle this scenario.

From: Marlon Taylor <Marlon.Taylor@hq.dhs.gov>
Date: Thursday, December 10, 2015 at 6:12 PM
To: "Wunder, John A." <jwunder@mitre.org>, Sean Barnum <sbarnum@mitre.org>, Jason Keirstead <Jason.Keirstead@ca.ibm.com>, Aharon Chernin <achernin@soltra.com>
Cc: "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org>
Subject: RE: [cti-stix] Applying data markings

1.       Rewording and examples (like your response for #3) will go a long way. See #5.

2.       No technical-enforcement on full path result, got it. I would like to dive deeper into the expected behavior for incorrect (malicious/invalid) path expressions with my reply in #4.

3.       Your snippet of the active/current document for the given path expressions shows everyone what should be evaluated for a given L2 scope.

4.       As for “throwing an error” as mentioned in #2, I have some additional questions around the expected behavior for the scenarios sublisted below. Should the consumer log and continue/skip or stop/break?

a.       If the given path _expression_ is invalid (incorrect syntax)

b.      If the given path goes outside the scope (returns 0 results for the given scope)

5.       Going deeper into scoping I have the following scenario. I expanded the previous indicator[0] with composite_indicator but this can be any referenced/embedded object:

{

      "stix_version": "2.0",

      "marking_definitions": [{

            "id": "us-cert:TLP-RED",

            "marking_type": "tlp",

            "tlp": "RED"

      }, {

            "id": "us-cert:TLP-GREEN",

            "marking_type": "tlp",

            "tlp": "GREEN"

      }],

      "marking_refs": ["us-cert:TLP-GREEN"],

      "indicators": [{

            "id": "indicator-1234",

            "structured_markings": [{

                  "controlled_structure": [

                        "@.title",

                        "@.composite_indicators.indicators[0].description"

                  ],

                  "marking_refs": ["us-cert:TLP-RED"]

            }],

            "title": "First indicator",

            "composite_indicators": {

                  "logical_expression": "OR",

                  "indicators": [{

                        "id": "indicator-5678",

                        "structured_markings": [{

                              "controlled_structure": [

                                    "@.title"

                              ],

                              "marking_refs": ["us-cert:TLP-RED"]

                        }],

                        "title": "First Composite indicator",

                        "description": "This is how it all began ..."

                  }]

            }

      }, {

            "id": "indicator-4321",

            "title": "Second indicator"

      }]

}

This would result in Indicator-5678 being all TLP-RED, correct?

 

6.       Would the answer/result for #5 change depending on whether the nested Indicator was referenced or embedded?

 

-Marlon

From: Wunder, John A. [mailto:jwunder@mitre.org]
Sent: Thursday, December 10, 2015 4:00 PM
To: Taylor, Marlon; Barnum, Sean D.; Jason Keirstead; Aharon Chernin
Cc: cti-stix@lists.oasis-open.org
Subject: RE: [cti-stix] Applying data markings

 

So here’s my perspective:

 

1.       In the specification, the line is “When used on a top-level object, the root of the controlled structure specification is the object” (maybe that should be clarified to, “the root of the controlled structure specification is the top-level object”). In other words, when used within an indicator, the controlled_structure should be evaluated as if the document is JUST that indicator. Same for a TTP, etc. When used in a package, obviously the package is the root.

2.       By making the root the same as the scope, it makes it very natural to do the right thing. In order to get the right answer you need to evaluate it as if the object is the root, and thus you should not evaluate it with the whole document (you should pull that object out of the document and evaluate it). Obviously, you can write invalid JSONPath expressions, either maliciously or accidentally. These just have to throw an error.

3.       For the purposes of that evaluation, the document will look like this:

 

{

"id": "indicator-1234",

      "structured_markings":

[

{

          "controlled_structure": [

            "$.indicators[1].title"

          ],

          "marking_refs": ["us-cert:TLP-RED"]

}

]

"title": "First indicator"

}

 

Thus, evaluating that JSONPath will lead to no results, because there’s no indicators key in the document. Yes, a consumer could potentially evaluate it in the context of the document, but even for well-formed queries that will almost always lead to the wrong result. So while we’re not validating it in schema (t’s impossible to do so, either in XML or JSON) we are making it very natural and easy to do the right thing (unlike the current approach, which is asking for this type of problem).

 

John

 

From: Taylor, Marlon [mailto:Marlon.Taylor@hq.dhs.gov]
Sent: Thursday, December 10, 2015 2:46 PM
To: Wunder, John A. <jwunder@mitre.org>; Barnum, Sean D. <sbarnum@mitre.org>; Jason Keirstead <Jason.Keirstead@ca.ibm.com>; Aharon Chernin <achernin@soltra.com>
Cc: cti-stix@lists.oasis-open.org
Subject: RE: [cti-stix] Applying data markings

 

Grabbed this example from the option 5 approach overriding-package-markings-1 example (https://github.com/johnwunder/data-markings#overriding-package-markings-1):

{

  "stix_version": "2.0",

  "marking_definitions": [

    {

      "id": "us-cert:TLP-RED",

      "marking_type": "tlp",

      "tlp": "RED"

    },

    {

      "id": "us-cert:TLP-GREEN",

      "marking_type": "tlp",

      "tlp": "GREEN"

    }

  ],

  "marking_refs": ["us-cert:TLP-GREEN"],

  "indicators": [

    {

      "id": "indicator-1234",

      "structured_markings": [

        {

          "controlled_structure": [

            "@.title"

          ],

          "marking_refs": ["us-cert:TLP-RED"]

        }

      ],

      "title": "First indicator"

    },

    {

      "id": "indicator-4321",

      "title": "Second indicator"

    }

  ]

}

 

The following questions are referencing the first indicator, indicators[0], which uses L2 markings.

 

1.       Scoping: What is the scope"controlled_structure": is in  "structured_markings": which is a valid JSON object. This is valid JSON but is this the intended scope for the L2 marking?

2.       Validation: How can we restrict/enforce the path is limited to the intended object?

3.       Exceptions: What is the expected behavior if we receive something referencing something outside of the current object?

    {

      "id": "indicator-1234",

      "structured_markings": [

        {

          "controlled_structure": [

            "$.indicators[1].title"

          ],

          "marking_refs": ["us-cert:TLP-RED"]

        }

      ],

      "title": "First indicator"

    },

    {

      "id": "indicator-4321",

      "title": "Second indicator"

   }

 

-Marlon

From: Wunder, John A. [mailto:jwunder@mitre.org]
Sent: Thursday, December 10, 2015 10:00 AM
To: Taylor, Marlon; Barnum, Sean D.; Jason Keirstead; Aharon Chernin
Cc: cti-stix@lists.oasis-open.org
Subject: Re: [cti-stix] Applying data markings

 

That’s a great point that totally slipped my mind.

 

In STIX 1.2, we said that the “marking scope” was limited to the construct itself. We didn’t say anything about the root of the document for purposes of evaluation (I just checked) so effectively it’s the root of the full package. Thus, as you say, it was possible to mark things outside the actual scope. I don’t know why we did this…whether we just didn’t think about it or whether it was because it’s (relatively) hard to pull a snippet of XML out of a document and have it live on its own (due to namespace mappings and things like that “higher” in the document that impact things at “lower” levels). So, in STIX 1.2, it’s up to the implementation to ensure that the things it marks are limited to the correct marking scope.

 

When defining this approach for STIX 1.2 I took a queue from JSON/JSONPath and considered just “re-rooting” the document. It’s very easy to do this in JSON because things higher in the document don’t impact things lower in the document, you can just pull that chunk out and it will always be valid JSON. Given that freedom in the (presumed) MTI, for this proposal I defined it differently: the actual evaluation root for the controlled structure is the object itself, and therefore when evaluating it you can/should just pull the chunk out completely and evaluate against that. That would prevent people from marking things outside the scope without requiring extra work on the implementation side.

 

If we develop an XML binding, we’ll have to figure out whether we need to override that and go back to the old approach. I would GUESS that this approach will still be better in XML even though you have to deal with namespace prefixes, but I’m not 100% sure. We could always use one for JSON and a different one for XML though.

 

That actually brings up another important point: if we assume JSON MTI is one format among several, tools will have to evaluate data markings and then track them in their repositories separate from the MTI approach, then be able to build and retransmit those markings in another format. In other words, if I take in JSON MTI I would need to evaluate that on the incoming JSON and store it in my database somehow. If I want to transmit into protobuf, I can’t just use those JSONPath markings because they’re no longer valid…they have to be redone in whatever we define for Protobuf. Just something to think about regarding the “storing STIX natively” topic.

 

John

 

From: Marlon Taylor <Marlon.Taylor@hq.dhs.gov>
Date: Thursday, December 10, 2015 at 9:33 AM
To: Sean Barnum <sbarnum@mitre.org>, "Wunder, John A." <jwunder@mitre.org>, Jason Keirstead <Jason.Keirstead@ca.ibm.com>, Aharon Chernin <achernin@soltra.com>
Cc: "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org>
Subject: RE: [cti-stix] Applying data markings

 


Moving to go deeper on L2 markings I would like more information on the restriction/enforcement of the scope of the specific field path.

Background and Questions: The L2 path is supposed to be limited to the "current" object however path syntax allow you reference other (eg parent, sibling, non-self) objects.  Even given a regex to require the first characters reference the "current" object

a) how do we ensure the full path doesn't go outside of the current object?
b) if the full path goes outside of the current object, and what is the expected behavior?

Let's continue to narrow this discussion to conclusion.

-Marlon

 


From:cti-stix@lists.oasis-open.org on behalf of Barnum, Sean D.
Sent: Tuesday, December 08, 2015 2:33:39 PM
To: Wunder, John A.; Jason Keirstead; Aharon Chernin
Cc: cti-stix@lists.oasis-open.org
Subject: Re: [cti-stix] Applying data markings

I agree with John’s characterization here.

 

sean

 

From: "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org> on behalf of John Wunder <jwunder@mitre.org>
Date: Tuesday, December 8, 2015 at 2:15 PM
To: Jason Keirstead <Jason.Keirstead@ca.ibm.com>, Aharon Chernin <achernin@soltra.com>
Cc: "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org>
Subject: Re: [cti-stix] Applying data markings

 

Nope, for any of these approaches we rely on the client software to do the right thing and preserve data markings (as well as honor them in the first place).

 

For this reason, it seems to me that as long as we define the semantics for how markings are applied at the package level we don’t need to have them exist directly on the object. We can just define that they’re semantically equivalent and allow people who are retransmitting objects to move them between the package and the object as they want (assuming of course that they preserve those semantics).

 

In other words, to me this would be a valid retransmission:

 

Org A => Org B

·         Package (TLP:RED)

o    Object 1

o    Object 2

Org B => Org C

·         Package

o    Object 1 (TLP:RED)

o    Object 2 (TLP:RED)

Yes, Org B is re-writing the object in the sense that they’re moving a marking there. But since semantically the marking was already there when it received it from Org A this seems fine to me. IMO resharers should be free to do things like this so long as they don’t change the actual content of what they’re retransmitting.

 

OTOH if this is not something the community thinks is valid then I would lean more towards what Aharon is suggesting and require that people mark the objects directly, otherwise he’s right that they’ll get “stuck” because you can’t retransmit them outside the context of the package that they came in.

 

John

 

From: Jason Keirstead <Jason.Keirstead@ca.ibm.com>
Date: Tuesday, December 8, 2015 at 1:44 PM
To: Aharon Chernin <achernin@soltra.com>
Cc: "Wunder, John A." <jwunder@mitre.org>, "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org>
Subject: Re: [cti-stix] Applying data markings

 

" If you mark at the package level there is no enforcement within the object that it’s a specific TLP. I could take that object and use it in another document and document mark that object as TLP White.

This would work if and only if IDs are mandated to always be derrived based on hashing all of it's content, including any and all makings.

Is that something that was decided?


-
Jason Keirstead
Product Architect, Security Intelligence, IBM Security Systems
www.ibm.com/security | www.securityintelligence.com

Without data, all you are is just another person with an opinion - Unknown


Inactive hide details for Aharon Chernin ---12/07/2015 05:26:45 PM---I ask the group this, if your markings are at the package
Aharon Chernin ---12/07/2015 05:26:45 PM---I ask the group this, if your markings are at the package level, how do you reuse the objects? If yo

From: Aharon Chernin <achernin@soltra.com>
To: "Wunder, John A." <jwunder@mitre.org>, "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org>
Date: 12/07/2015 05:26 PM
Subject: Re: [cti-stix] Applying data markings
Sent by: <cti-stix@lists.oasis-open.org>





I ask the group this, if your markings are at the package level, how do you reuse the objects? If you mark at the package level there is no enforcement within the object that it’s a specific TLP. I could take that object and use it in another document and document mark that object as TLP White.

If we require markings at the object level, then you are protected by object revisioning rules, and potentially ID hashing. You could never change the TLP of the object and reuse that object without breaking the revisioning rules or the ID hash.

I am not going to put up a huge fight here, but it just seems so obvious to me.

Aharon

From: <cti-stix@lists.oasis-open.org> on behalf of "Wunder, John A." <jwunder@mitre.org>
Date:
Monday, December 7, 2015 at 3:42 PM
To:
"
cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org>
Subject:
Re: [cti-stix] Applying data markings


All,

I updated the proposal based on the comments from last week: https://github.com/johnwunder/data-markings.

A few comments (on the comments…):

- Mark suggested renaming “Level 1” to “Data Markings” and “Level 2” to “Field Level Data Markings”. I initially made that change but found the language got very confusing so I switched it back.

- Per Aharon’s comments, this proposal does have markings at the package level for both L1 and L2. This allows you to mark everything in a package as TLP:RED in a single statement (not duplicated across each indicator). OTOH it also means that if you discard the package you would need to have some way of tracking the markings separately. IMO that’s totally fine…it’s metadata about an object so it’s not like you’re changing the object itself if you add them to the object, and in any case per Eric’s earlier e-mail I would imagine that after ingest your tool would probably be dealing with markings in something outside of STIX anyway (I certainly would, in particular for L2).

- Aharon commented that we were getting rid of XPath and that’s an advantage of this proposal. That’s true……...kind of. Replacing XML with JSON means we use JSONPath instead of XPath, but fundamentally it’s still a controlled structure based approach with all the implementation complexity that it brings (as Bryan Worrell can tell you).

- That said, JSON is more straightforward than XML and so I think using it will minimize the occurrence of bugs like the ones we’ve had in the past with misunderstanding XPath.

To summarize things, here’s the open questions that I can think of (beyond “is this proposal good”):

1. Should the ability to handle Level 1 markings be MTI (mandatory to implement)? (
2. Should we spend further time investigating better approaches for L2 markings (Option 3, 4, others) or just go with this (Option 5)?
3. Should we keep markings at the package level or move all markings to the individual top-level objects?

My answers:

1. Not sure to be honest. Leaning towards not MTI.
2. We should use L2 as-is…it’s hard, but marking fields is hard and so IMO that’s fine. Most people will use L1 anyway.
3. Keep markings at the package level. This allows you to represent a “default” marking and override it. I also *think* (maybe I’m wrong) that certain entities in USG require package-level markings to represent “highest-classification”, which means other gov’t may as well.

John

On Dec 7, 2015, at 10:43 AM, Cory Casanave <cory-c@modeldriven.com> wrote:

This does look like a good approach for message level markings.
Any information exchange, except for open data, is done under some explicit or implicit agreement between the parties. Various forms of such “exchange agreements” have been around for years. For example, the “Collaborative Partner Profile Agreement” in EbXML, which came out of IBM. Having a reference to an exchange agreement provides the basis for trust between specific parties as well as an explicit representation of any categorization and/or restrictions on any exchange under that agreement. For example, asserting that telephone numbers are personally identifiable information and that personally identifiable information shall not be stored. Markings in the “instance” document augment the exchange agreement. It would simply not be practical or trustworthy to expect all such “markings” to be in every instance, so reference to an exchange agreement provides that level of trust and a place to put “generic” restrictions and categorizations. I suggest exchange agreements be considered for CTI. The exchange agreement reference can be a kind of marking.
-Cory

From:cti-stix@lists.oasis-open.org [mailto:cti-stix@lists.oasis-open.org] On Behalf Of Barnum, Sean D.
Sent:
Monday, December 07, 2015 9:49 AM
To:
Wunder, John A.;
cti-stix@lists.oasis-open.org
Subject:
Re: [cti-stix] Applying data markings

I fully support this option as would be expected.
Kudos to John for such a great writeup explaining the idea.

sean

From: "cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org> on behalf of John Wunder <jwunder@mitre.org>
Date:
Thursday, December 3, 2015 at 3:11 PM
To:
"
cti-stix@lists.oasis-open.org" <cti-stix@lists.oasis-open.org>
Subject:
[cti-stix] Applying data markings


All,

I developed this proposal to handle the application of data markings in STIX 2.0: https://github.com/johnwunder/data-markings. Note: it doesn't address the format of the markings themselves (improvements to TLP, the work in FIRST, etc), just how those markings get applied to content.

I this this meets the need for simplicity for object-level markings as we’ve talked about many times while still allowing for more complicated field-level markings for those that need them. Please review the proposal and let’s talk about feedback. If this looks good to everyone we could use it as the solution for issue #231 (currently #2 on our roadmap).

John

 



[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]