OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

cti message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]


Subject: Re: [cti] Re: Observable Patterning


Hi all,

I am trying to catch up on this thread and offer some thoughts.

I will start here but will likely add some thoughts farther along the thread as well.

Let me start by saying that I think we are talking about several different topics here and kind of munging them all together. I think we are going to need to separate them to deal with them appropriately.
So far I see the following topics:
  1. Should observable patterns be expressed using a dedicated patterning syntax separate from the structures used to express observable instances (observations)?
  2. If expressed separately, where should such an observable patterning language be defined (in CybOX, in STIX, in STIX/Indicator, etc.)?
  3. Are observable patterns the same thing as indicator patterns?
  4. Should there be one patterning language for both observable patterns and indicator patterns?
  5. Are STIX Indicators the only place observable patterns are used?
  6. Is there a semantic difference between STIX/Indicator/Observable (where a CybOX observable pattern is expressed) and STIX/Indicator/Test_Mechanism (where an observable pattern is expressed using some external syntax)?

My thoughts on these topics:
  1. I think this may be possible but will need to ensure that the appropriate expressivity is maintained. This is not as simple as it may seem on the surface (e.g. simply flat key-value pairs are definitely insufficient) and will take some work to get right.
  2. I would strongly assert that an observable patterning language (mechanism for expressing observable patterns) should remain as part of CybOX. CybOX is a language for characterizing the “facts” about cyber observables separate from the context of their meaning. This is due to the fact that many different domains have a need to express context about cyber observables but the contexts are typically different in nature. In this way, CybOX can act as a common language for cyber observables among a diverse set of domain-specific languages. The relevant “facts” can either be very specific and detailed values of events, objects and properties or can be a bit more fuzzy patterns on events, objects and properties. These more fuzzy patterns can be used to express a targeted pattern to look for among a set of detailed observable instances but they can also be simply less precise characterizations of cyber observations where  more detail is not possible  due to the precision of the sensors  making the observations, of the tools used to capture, analyze or communicate the observation, or the nature of the sharing source for the observation. No matter which way you cut it, I would assert that observable patterning should stay within the realm of CybOX and not be artificially lifted to any particular domain-specfic use of it which would hinder its ability to be used by other domain-specific uses that had a need. You may want to take a look at my thoughts on #3, #4 and #5 for more context.
  3. Observable patterns and indicator patterns are NOT the same thing. Observable patterns are a (arguably “the”) key part of any indicator as they express what patterns should be looked for in operational data to identify a sighting of that indicator. However, indicators can also have logical compositional patterning of their own that is separate from the underlying observable patterning in any particular indicator. There are specific reasons for this that are described here. It is important not to confuse these two forms of patterning. Indicator logical composition patterning (AND, OR, NOT) is done through the same approach as observable logical composition patterning but they are composing very different things and in addition, observable patterning also includes relational patterning which is not relevant for indicators.
  4. It may be possible to create one language for observable and indicator patterning but it may be difficult or impractical. I am not asserting that it will be but only that it could be. It would need to be structured in an abstract way that allowed assertions of logical composition on different types of things (e.g. Observables or indicators) and could not be tied directly to one or the other. On the other hand the requisite capabilities that would need to be included for defining relational compositions or even just property-based patterns for observables would need to be tied pretty closely with the appropriate event, object and property structures of CybOX which would be completely out of scope for STIX. In other words using a single patterning language would require STIX Indicators (and other areas discussed in #5 below) to leverage a language of which the large majority of its capability is tied to CybOX. This does not seem like good separation of concerns to me. Maybe it would make sense to structure a single patterning language in three parts: logical composition, relational composition and direct assertions (on events, objects or properties). The logical composition part could be designed abstractly to be leverageable by either STIX Indicators or by CybOX and the other two parts could be CybOX specific. That way CybOX could leverage all three parts together but STIX could just use the logical composition part. Just spitballing here.
  5. I think it is important that people remember that Indicators are not the only parts of STIX to leverage observable patterns. There are actually 5 different places within STIX that leverage observable patterns to express what needs to be expressed. Indicators are only 1 of these 5. The others are below with more detail and examples available here:
    1. Specify particular structured technical details for explicit characterization of a Course of Action's Parameter Observables.
    2. Specify what software is known to be affected by a given vulnerability via the Exploit Target AffectedSoftware's Observable characterization.
    3. Characterize specific technical infrastructure utilized for cyber attack via a TTP Infrastructure's Observable Characterization
    4. Characterize specific victim technical context details being targeted by an attacker in instances of TTP VictimTargeting's Technical Targeting Details.
  6. It looks like the difference between Observable and Test_Mechanism within STIX Indicators may be confusing for some. There really is not a strong semantic difference between these sub-structures of Indicator. They both are intended to express the observable pattern relevant to that Indicator. The only difference is that Observable is built on the presumption that the pattern is expressed using CybOX while Test_Mechanism is an extension point for expressing the pattern is any other format desired (e.g. YARA, OVAL, OpenIOC). They are currently separate structures because at the time they were designed there was a clear desire to demonstrate a preference for CybOX  over other approaches for expressing observable patterns due to the value of consistency as CybOX is used in many other areas of STIX (malware characterization, incident investigation, digital forensics, etc.) where these observable patterns would likely come from anyway. It may be possible (potentially desirable) in STIX 2.0 to remove the separation between STIX/Indicator/Observable and STIX/Indicator/Test_Mechanism with a single Observable Pattern structure that is basically what Test_Mechanism is today but with a default implementation of a CybOX based observable pattern. So basically the current Observable field would be renamed Observable Pattern and changed to TestMechanismType, and a new CybOXTestMechanismType would be created and asserted as the suggested default extension. It would also probably make sense to change the wording of Test Mechanism to Observable Pattern.

Sorry for the long email but wanted to comment at least briefly on all the various topics being combined in this thread.
Again, I would suggest we separate them out.

sean


From: <cti@lists.oasis-open.org> on behalf of Steve Cell <ikirillov@mitre.org>
Date: Thursday, September 24, 2015 at 9:30 AM
To: Mark Davidson <mdavidson@mitre.org>, Trey Darley <trey@soltra.com>, "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: [cti] Re: Observable Patterning

In order for a pattern to be specified in a STIX Indicator (e.g., ‘IP Address in 1.2.3.4/24’), would the pattern would need to use fields from a CybOX object? If the answer is yes, does that create a hard dependency between STIX Indicator and a particular CybOX object and version?

Having a separate patterning structure would mean that we would somehow need to reference CybOX Object fields, and unless we completely decouple CybOX from STIX Indicators (this seems unlikely, though I believe there have been a few suggestions to this effect), there would indeed be a hard dependency between a STIX Indicator and a particular CybOX Object version. This is necessary to explicitly specify the semantics of “what” the Indicator is looking for. 

What about consistency with other uses of STIX Indicator? Consider a SNORT signature or YARA rule – patterning is contained in those signatures and wouldn’t be at the STIX Indicator level, which seems inconsistent to me.

This seems to be more of an issue with how different types of patterns are captured in STIX Indicators. CybOX-based patterns are currently represented as a separate entity, whereas other types of patterns (SNORT, YARA, etc.) are captured as Test_Mechanisms. Given their functional overlap, it may make more sense to have a single “Pattern” field that can be extended to capture CybOX-based patterns, SNORT rules, YARA sigs, etc.

Regards,
Ivan

From: Mark Davidson
Date: Thursday, September 24, 2015 at 9:06 AM
To: Trey Darley, Ivan Kirillov, "cti@lists.oasis-open.org"
Subject: RE: Observable Patterning

What implication would patterning in STIX Indicators have for the relationship between STIX Indicators and CybOX objects?

 

In order for a pattern to be specified in a STIX Indicator (e.g., ‘IP Address in 1.2.3.4/24’), would the pattern would need to use fields from a CybOX object? If the answer is yes, does that create a hard dependency between STIX Indicator and a particular CybOX object and version?

 

What about consistency with other uses of STIX Indicator? Consider a SNORT signature or YARA rule – patterning is contained in those signatures and wouldn’t be at the STIX Indicator level, which seems inconsistent to me.

 

> Patterning should live in _one_ place and have _one_ syntax.

I really like this idea, and I’m asking questions that I hope will lead us in this direction.

 

Thank you.

-Mark

 

From:cti@lists.oasis-open.org [mailto:cti@lists.oasis-open.org] On Behalf Of Trey Darley
Sent: Thursday, September 24, 2015 4:30 AM
To: Kirillov, Ivan A. <ikirillov@mitre.org>; cti@lists.oasis-open.org
Subject: [cti] Re: Observable Patterning

 

+100, Ivan.

 

Patterning should live in _one_ place and have _one_ syntax.

 

I tend to think the STIX Indicator level is the right place for this to live.

 

Cheers,

Trey

--

Trey Darley

Senior Security Engineer

Soltra | An FS-ISAC & DTCC Company

 


From:cti@lists.oasis-open.org <cti@lists.oasis-open.org> on behalf of Kirillov, Ivan A. <ikirillov@mitre.org>
Sent: Monday, September 21, 2015 19:47
To:
cti@lists.oasis-open.org
Subject: [cti] Observable Patterning

 

Relaunching this as a separate thread based on Jerome’s suggestion, so as to not further hijack the discussion around Bernd’s suggested refactoring.

 

I think there are a few broad questions around this topic that we can touch upon here: 

 

1.

What capabilities should STIX/CybOX patterning have?

2.

Where should STIX/CybOX patterning live?

3.     Should patterns have their own implementation/structure, or continue to be embedded in Observables/Objects?

 

As far as 1) I would say the following, as a straw man:

 

* Conditional operators

    * AND

    * OR

    * NOT

* Basic temporal operators

    * FOLLOWED_BY

    * WITHIN

* String matching

    * EQUALS

    * DOES_NOT_EQUAL

    * CONTAINS

    * DOES_NOT_CONTAIN

* Basic arithmetic operators

    * >

    * <

    * >=

    * <=

* Regular Expressions

    * PCRE compatible

* Variable substitution

 

Regarding 2), it seems to me that patterning is inherent to Indicators, and thus should be part of STIX. I just can’t see the use case for a CybOX Observable with a pattern that stands on its own, or is used in a non-Indicator context. This would also serve to greatly simplify CybOX, as it would mean that CybOX can ONLY capture instances of cyber data. 

With regards to 3), I see lots of advantages to having a separate structure for patterns, which I’ve documented in the corresponding CybOX issue [1]. The short of it is that it will get rid of the instance/pattern duality, allow for the creation of a domain-specific patterning language (necessary for more complex pattern expressions), and also give us the flexibility to move patterning out to wherever makes the most sense (whether it’s STIX or somewhere else).

 

 

Regards,

Ivan



[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]