OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

cti message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]


Subject: Re: [cti] Re: Observable Patterning


Joep,

I don’t think there is a clear dichotomy as you suggest between the Observable (observable pattern) of an Indicator and any test mechanisms defined for it.
They are really semantically the same thing.
They are both simply observable patterns for identifying whether some particular factual observable conditions exist. These patterns are devoid of any context as to “malicious activity” or other wise. It is the enclosing Indicator that assigns such context to the pattern.
You are correct in that test mechanisms would most likely be used (rather than Observable) based on some certain use case where the other format provides expressivity that CybOX does not or the fact that the targeted recipient of the Indicator has particular tooling capabilities that you are looking to serve.

sean

From: <cti@lists.oasis-open.org> on behalf of Joep Gommers <joep@intelworks.com>
Date: Monday, September 28, 2015 at 11:57 AM
To: Steve Cell <ikirillov@mitre.org>, Jerome Athias <athiasjerome@gmail.com>, Patrick Maroney <Pmaroney@Specere.org>
Cc: Terry MacDonald <terry.macdonald@threatloop.com>, "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: Re: [cti] Re: Observable Patterning

Hi Ivan,

I’d be interested what the team feels about the relationship between these patterns and test mechanism concept. Please correct me if wrong;
- indicator patterns described a pattern of potentially malicious activity that is intended to convey the pattern in a way that a human can understand and that machines can translate into action-based use-cases (like detection)
- test mechanisms are a pre-computed or created pattern specific to a certain use-case (like snort detection, yara, etc.)

Since in both cases that might be conditional relationships and nested conditional calculations (e.g. If you don’t see X, then the previous variable in de flow is +1, if you do seeX, then the previous variable in the flow is –1) - it makes it ill-suited for graph representation in the context of graph querying.

J-

From: <Kirillov>, "Ivan A." <ikirillov@mitre.org>
Date: Monday, September 28, 2015 at 3:36 PM
To: Jerome Athias <athiasjerome@gmail.com>, Patrick Maroney <Pmaroney@specere.org>
Cc: Terry MacDonald <terry.macdonald@threatloop.com>, "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: Re: [cti] Re: Observable Patterning

>- Human readable
     Agreed.
>- Machine friendly (parsing/computation, algorithmically friendly)
     Agreed.
>- Data graph friendly
     Not so sure on this one. 

While I agree that STIX/TAXII querying (wherever it ends up) is inherently graph-based, I don’t believe the same is true for Indicator patterning. The main difference is that, in my view, the majority of Indicator patterns are meant to be parsed and executed against data that is inherently flat – lists of files, lists of processes, lists of IP addresses, etc. Thus while the two may overlap in certain places, I’d be very hesitant about overloading an Indicator patterning structure to support graph based querying, though perhaps it could be extended to do so. Again, I think a primary focus here should be to keep Indicator patterns SIMPLE, so that they can easily be written and consumed by analysts.

Regards,
Ivan

From: Jerome Athias
Date: Saturday, September 26, 2015 at 12:38 PM
To: Patrick Maroney
Cc: Terry MacDonald, "cti@lists.oasis-open.org", Ivan Kirillov
Subject: Re: [cti] Re: Observable Patterning

so should we try to capture the requirements in one place for this change request?
Quickly:
- Human readable
- Machine friendly (parsing/computation, algorithmically friendly)
- Data graph friendly

2015-09-26 17:57 GMT+03:00 Patrick Maroney <Pmaroney@specere.org>:
Probably not clear that I've similarly argued to keep TAXII focused on the transport of packages.  In the query method suggested for consideration, the Pattern would be passed as a STIX (Query/RFI) package and results similarly returned (again as a STIX package).

It's important to visualize this a Data Graph that maps to the Conceptual CTI model (which is the basis for suggesting Cypher as an example representation).  One can describe/query precise graphs of Objects/Relationships, any Nodes/Edges off of a Root Node, etc.

Hope this makes sense (at least in terms of framing the proposal).

Sent from Outlook




On Fri, Sep 25, 2015 at 6:01 PM -0700, "Terry MacDonald" <terry.macdonald@threatloop.com> wrote:

I agree with Patrick's comments about ensuring pattern support for querying, but not the idea querying should be done in TAXII.

Querying and answering should be done within STIX, so that the querying and answering done by the language contained within TAXII, and so that TAXII is free to concentrate on only delivering content. We should instead develop a STIX query object, and a STIX reply object in order to support question and answer directly within STIX. I won't go into the specific argument why here and clog this thread, but it has been discussed on the TAXII list recently.  

Cheers


Terry MacDonald | STIX, TAXII, CybOX Consultant




Disclaimer: The opinions expressed within this email do not represent the sentiment of any other party except my own. My views do not necessarily reflect those of my employers.

On 26 September 2015 at 01:33, Patrick Maroney <Pmaroney@specere.org> wrote:
I would like to add a couple of conceptual ideas to this discussion (along with a specific illustrative reference):

(1)  If we architect our flexible "patterning" model correctly it 'should' be directly applicable to our TAXII Query Language.

(2) Graph data models provide some interesting correlations/visualizations.  As previously suggested neo4j's Cypher Query language might serve as a good conceptual model for how to express "patterns"

http://assets.neo4j.org/download/Neo4j_CheatSheet_v3.pdf

Patrick Maroney
President
Integrated Networking Technologies, Inc.
Desk: (856)983-0001
Cell: (609)841-5104
Email: pmaroney@specere.org




On Fri, Sep 25, 2015 at 8:12 AM -0700, "Patrick Maroney" <Pmaroney@Specere.org> wrote:

A general [+1] for everything @Ivan said below (although agnostic on some of the specifics)

To this "flexible" patterning construct, advocate inclusion of fixed/relative temporal relationship _expression_ (highly simplified examples:  "a" followed by "b" , "a" during "b" ) along with compound boolean and set theory _expression_.

Patrick Maroney
President
Integrated Networking Technologies, Inc.
Desk: (856)983-0001
Cell: (609)841-5104
Email: pmaroney@specere.org




On Fri, Sep 25, 2015 at 6:13 AM -0700, "Kirillov, Ivan A." <ikirillov@mitre.org> wrote:

To Terry’s point about tool-specific versus generic patterns  - to me at least, this is rather relative. While it’s true that there is often a hard dependency between a particular pattern language and engine/tool that consumes it, this is not always the case; for example, YARA rules can be consumed by several different tools, including Cuckoo Sandbox. Thus, going back to Mark’s point below, I don’t see a true semantic distinction between STIX/CybOX patterns and those expressed in other forms (test mechanisms) – ultimately they are meant for detecting badness, and either your tool(s) can parse and make use of them, or they cannot. 

I’m of the opinion that the value that CybOX offers in terms of patterning is with regards to expressing patterns against its Objects and their corresponding data models (very similar to OVAL in that respect, as Jerome alluded to :)), and enhancing this capability should be our primary focus. Trying to create a monolithic pattern language that subsumes all others is untenable (not that I think anyone is really suggesting this), and I think having Snort, YARA, and the like out there is a great thing; they do what they do very well, and thus we should leave their capabilities to them. I don’t view it as a negative if an Indicator pattern can be better expressed, say, through a YARA rule than through a CybOX pattern.

Also, I’m supportive of Bernd’s idea of a top-level Pattern structure, and also one that is relatively abstract and amenable to being used in different domains (e.g., fraud). Ultimately, I think this structure needs to be able to express some set of constraints against a data model and its fields, whether it’s a CybOX Object or something else, like a structured representation of a bank account (as an example). Therefore, perhaps it could even live as a separate effort/work product – Generic LangUage for Patterning (GLUP), anyone? :-) 

So, to offer up another straw man (and I think Jason had something very similar), it would be interesting to see if we can converge on a YARA-like patterning structure that is easy for humans to read and write, and also machine-parseable:

pattern example_1 

{

  objects:

$OBJ1 = {type = AddressObject,

         fields = [{category:ipv4-addr”},

                   {address_value:192.168.2.3}]} 

  

$OBJ2 = {type = AddressObject,

         fields = [{category:ipv4-addr”},

                   {address_value:192.168.4.7}]} 

condition:

    OBJ1 and OBJ2

}

 

Regards,
Ivan

On 9/25/15, 8:02 AM, "Davidson II, Mark S" <mdavidson@mitre.org> wrote:

Adding my thoughts to the Test_Mechanism idea.

In line with 'one way of doing things', my preference would be for all signature-ish things to be thought of in the same way. In my mind, "signature-ish things" includes SNORT, YARA, and STIX/CybOX patterns. This is a little different than what we have today, where CybOX patterns are separate from Test Mechanisms [1], and Test Mechanisms are intended to identify the CybOX pattern(s). The way I'm thinking would have all signature-ish things become peers.

If I'm whiteboarding a use-case for signature/Indicator exchange, it would be something along the lines of:

Title: Send a signature

Description: An analyst creates a signature/indicator and chooses to share it with partner organizations.

Main Success Scenario:

1. Org 1's analyst develops a signature (e.g., SNORT, CybOX pattern)
2. Org 1's analyst decides to share the signature; posts the signature to a TAXII Channel (perhaps as a STIX Indicator)
3. Org 2, which is subscribed to the TAXII Channel, receives the signature (or STIX Indicator) and
3a. The signature (or STIX Indicator) is entered into Org 2's internal review queue to decide whether to deploy the signature (or STIX Indicator)
4. The signature (or STIX Indicator) is acted on - either manually or automatedly - and is discarded or deployed (or some other action)

Each one of these steps probably has a one-level-deeper set of steps associated with them (and a bunch of dependiencies/requirement), but at least for me, this is how I understand the workflow from Indicator creation to information sharing.

Thank you.
-Mark


-----Original Message-----
From: Grobauer, Bernd [mailto:Bernd.Grobauer@siemens.com]
Sent: Friday, September 25, 2015 7:02 AM
Subject: RE: [cti] Re: Observable Patterning

Hi,

I don't really see Test_Mechanisms as a candidate for reusing the
patterning that we've been discussing. Each Test_Mechanism rule is a
rule derived from the Indicators, and written in the tools native
language.

The principle of having a test-mechanism derive from what is described
within an indicator using observable patterns is the intended meaning,
but this principle is not always followed: it may be quite complicated
to describe in STIX/CybOX, what can be succinctly described in a
certain pattern language or the authoring party simply does not want to
bother with this -- so the STIX indicator is also used as a vehicle to
exchange such a test mechanism, enriched with contextual information
contained in the indicator and referenced/referencing STIX objects,
but not containing corresponding observable patterns. I would guess
that we will continue to see such use of a TestMechanism within an
otherwise rather empty indicator (empty with respect to Observables),
even if the pattern language is evolved such that in theory it could
describe everything that is expressible in any thinkable test
mechanism.

It doesn't make sense to me to have patterning in the
Test_Mechanism unless it is natively in the tool that uses the rules.

I had made that proposal in one of my emails with the intent of encapsulating
complexity. I can see why one would not want to use a test mechanism
as container for STIX/CybOX containers -- after all, no other test mechanism
would have references out of the test-mechanism blob, whereas a
STIX pattern could have many references to objects/object properties
defined elsewhere.

But I really would like to get rid of AND/OR/...-composition on
the level of indicators and push the whole pattern language
into (oh no, here it comes) yet another STIX top level object
STIX_Pattern or whatever. Then allow STIX_Patterns to reference
other patterns for composition, but not Indicators.

The reason is that the whole logical and temporal composition
is really complicated. The Indicator is absolutely central
to STIX, but like I said above, there may be use-cases to
communicate things to look out for *without* the use of
CybOX and CybOX patterning but via a test mechanism (or,
I haven't given up on that idea, a STIX-standardized key/value-pair
mechanism for communicating patterns).

Now, if we encapsulate the complications of the pattern language
in something that is removed from the Indicator level, then
one can implement the non-patterning-part of STIX much easier
in order to use it for use cases as described above. Also,
using STIX for other domains (fraud and what not) as has
been mentioned on the list yesterday, could become easier, since
for these other domains, the full might of STIX patterns
may not be needed ... or such domains may even need something
else entirely. (Hm, I just realize that this reasoning
would actually support making patterning part of CybOX
as a CybOX-pattern-top-level object.)

In closing, because I have proposed yet another top-level STIX entity:
in one of his mails about a possible top-level targeting object,
Aharon was concerned about making things too complicated by adding
more and more top-level objects. That is a valid concern, but on the
other hand, having huge schema definitions for objects that encompass
all kinds of substructures, is also quite complicated.  So, while we
certainly should have good reasons for introducing new top-level
objects, I would say that given the right circumstances, new top-level
objects will actually decrease complexity.

So, in short:

- I fully agree that decoupling patterning and instances
  is the right way to go.

- I would like to encapsulate patterns in a top-level construct
  that may be referenced by STIX indicators while removing
  not/and/or composition for indicators.

- I am not quite sure yet whether the 'Pattern' top-level object
  would better be part of STIX or part of CybOX.

  If we think the patterning is reusable also for extended domains such
  as a possible SocEngOX (Social Engineering Observable EXchange)
  language and we think we will want to write things like

  (SocEng-Observation "Phone call from Mr. Evil") THEN
  (CybOX Email Object with sender "lucifer.evil@hell.com)

  then patterning is probably part of STIX rather than CybOX; otherwise,
  as a cyber-focused language, it might well be part of CybOX.


Kind regards,

Bernd

------

Bernd Grobauer, Siemens CERT







[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]