Proposal: Re-occuring values within patterns

[Christian Doll <doll@cs.uni-bonn.de>]

Hi everyone,

While working with STIX patterns, we thought about the possibility to
model randomized values which re-appear across multiple Observable
objects. We would like to present our approach here and propose a small
change for future specifications of STIX.

Our approach is also currently open for discussion at the
cti-pattern-matcher repository [1], where we also prepared the necessary
code changes for the approach [2].


OVERVIEW
========

We want to model a pattern that describes the behaviour of a malware
sample. The sample is executed multiple times and we get the following
Observables from each execution (for simplicity we only consider the
'name'-property in the example for each Observable):

+-----------------+ +-----------------+ +-----------------+
|   RUNTIME 1     | |   RUNTIME 2     | |   RUNTIME 3     |
|File:  foo       | |File:  bar       | |File:  baz       |
|Mutex: Global\foo| |Mutex: Global\bar| |Mutex: Global\baz|
+-----------------+ +-----------------+ +-----------------+

Simplified STIX observed-data notation:
Execution 1: file:name = 'foo', mutex:name = 'Global\foo'
Execution 2: file:name = 'bar', mutex:name = 'Global\bar
Execution 3: file:name = 'baz', mutex:name = 'Global\baz'

The first step in creating a good pattern is to generelize properties
which change between executions. We chose regular expressions, so the
pattern for our example lools like this:

                 +-----------------------+
                 |         Pattern       |
                 |File: [a-z]{3}         |
                 |Mutex: Global\\[a-z]{3}|
                 +-----------------------+

STIX Pattern notation:
[file:name MATCHES '[a-z]{3}'] AND [mutex:name MATCHES 'Global\\[a-z]{3}']

In the next step, we want to express that the value within the file-name
property re-appears within the mutex-name property. The obvious solution
to our problem is the use of variables, which have to be usable at the
property-describtion level of a pattern. Luckily we can do this with
named capture groups from within regular expressions (which are within
the capabilities of PCRE, anyway). For our example, we would wrap each
character class inside a named capture group, acting as a variable:
[file:name MATCHES '(?P<var>[a-z]{3})'] AND [mutex:name MATCHES
'Global\\(?P<var>[a-z]{3})']

During evaluation of the pattern, we test whether each named capture
group with the same name within one pattern (i.e. each variable) matched
on the same strings. We call a pattern with this additional constraint
Inter-Observable Pattern, since we express an additional property which
spans across multiple Observables:

             +--------------------------------+
             |    Inter-Observable Pattern    |
             |File:  (?P<var>[a-z]{3})        |
             |Mutex: Global\\(?P<var>[a-z]{3})|
             +--------------------------------+


Our approach should be fully backwards compatible, as it does not
violate STIX 2.0 specifications, but extends it. Since named matching
groups are a PCRE required capability, any parser should be able to
consume a pattern like that.
However, a matcher capable to evaluate the matching group's names is
able to reduce false positives. Therefore, we propose to include our
approach for the handling of named capture groups within the
MATCHES-operator of patterns.
We are aware of the working concepts and plans for variables and
backreferencing for STIX [3], but believe our approach to be more
elegant for the use case detailed above and use cases that rely on
equality of substrings in general (e.g. multiple login attempts with the
same user name).

We appreciate your consideration of our proposal.
Please do not hesitate to contact us if any questions come up.

Best regards,
Christian Doll & Arnold Sykosch


[1]: https://github.com/oasis-open/cti-pattern-matcher/issues/57
[2]: https://github.com/oasis-open/cti-pattern-matcher/pull/58
[3]:
https://docs.google.com/document/d/15qD9KBQcVcY4FlG9n_VGhqacaeiLlNcQ7zVEjc8I3b4/edit#heading=h.vw8nwfvm4pvm