OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

cti-users message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]


Subject: Re: [cti-users] How does CybOX data is generated?


Great answers from Terry and Shawn so far.
Let me take a stab at answering your original question. Some of this will repeat what has been said already but hopefully add a little bit as well.

As Terry pointed out there are two sorts of observables in CybOX: observations (observable instances of things that were observed) and observable patterns (patterns of things that could be seen).

As a general rule, observable patterns are identified and abstracted from bodies of observations and can then be used for specifying Indicators to look for or as abstract characterization of things like attacker infrastructure, technologies targeted by attackers, etc. As Terry and Shawn point out this abstraction is often performed by humans but can be augmented by various technologies like anomaly detection, machine learning, etc.

I think the root of your question was more aligned to where do the actual observations come from. As the guys have said there are different sorts of observations that come from different places and are leveraged by different functions and people.
The first main division would be observations that come from incident response triggered by some event and observations that come from proactive “hunting” without any specific trigger required.

Within incident response most observations are going to come from a combination of historical logs, security tool alerts, digital forensic analysis (host, network, memory, etc.), malware analysis, and proactive querying and searching in response to other observations and interpretations discovered during the incident investigation. As you can imagine much of this could be machine generated with some human augmentation. The reality so far is that there are still few tools that output logs or alerts natively in CybOX. The digital forensic analysis space is being targeted by a related effort called the Digital Forensics Analysis _expression_ (DFAX) which leverages CybOX for observations. The malware analysis space is being targeted by a related effort called the Malware Atribute Enumeration and Characterization (MAEC) which also leverages CybOX for observations. CybOX is still a young representation but is making progress across this space. Much of incident response is about collecting and looking at relevant evidence (observations) and interpreting what they mean in order to know where to look further and in order to determine what actually occurred. This interpretation and assertion of context for the observations is applied through various parts of STIX including the incident itself, TTPs, threat actors, campaigns, indicators, etc. The really relevant bits and pieces out of all these observations can then be abstracted and captured as patterns to be applied within STIX to capture various forms of context.

Within hunting most observations are going to come from proactive querying and searching of various observation sources in response to higher-level threat intelligence context (the sort of thing Shawn describes). Observations from hunting may come from different initiating events but are really mostly from the same sources and of the same forms as those collected as part of incident response.

I will try to avoid going too far down the rabbit hole and stop there.

Is that useful in helping to answer your question?


sean



From: <cti-users@lists.oasis-open.org> on behalf of Shawn Riley <shawn.p.riley@gmail.com>
Date: Sunday, March 6, 2016 at 6:48 PM
To: alexander kipnis <alexander.kipnis85@gmail.com>
Cc: "cti-users@lists.oasis-open.org" <cti-users@lists.oasis-open.org>
Subject: Re: [cti-users] How does CybOX data is generated?

Alexander, 

Prior to the late 1990s the analysis of cyber threats was mainly performed by analyzing internal security logs and sensor output. As Terry notes, this is more commonly associated with incident response today. After the discovery of advanced threat campaigns such as TITAN RAIN in the late 1990s things changed and you had teams that started focusing more on malware analysis and analysis the infrastructure that supported the malware. Many of these teams didn't even look at internal logs or sensors but focused exclusively on the data sets attributed to the threat actors to produce analytic insights and intelligence. It really depends on the industry you are in as to what level of maturity the organization is in understanding the need to look at different types of data for full spectrum threat analysis. CYBOX and STIX was designed to support organizations at all levels of maturity from those still doing things like it was the early 1990s as well as those who more using more advanced tradecraft and science. 

Best, 
Shawn

On Sun, Mar 6, 2016 at 4:43 PM, Terry MacDonald <terry@soltra.com> wrote:

Hi Alexander,

 

There are many different types of analysts J. What you are describing I would classify as more of an incident response role – specifically performing ‘hunting’ where they inspect logs looking for anomalies, and monitoring tools that are performing a ‘detection’ role (with the rules that you have been describing).

 

There are also nowadays threat analyst roles, which are designed to understand the threats that an organisation faces, and try to understand and research which threat actors are most likely trying to target the organisation, and to track them. Their purpose is to eliminate most of the unnecessary threat intelligence and distil it down to the small amount of threat intelligence that is most likely to affect the organisation. The incident response role then takes that reduced set of data and tries to find it within the org.

 

It’s a combination of the two functions that makes the overall process stronger. The Threat Analyst works out who we should be looking for, and the Incident Responder tries to find them.

 

Cheers

 

Terry MacDonald

Senior STIX Subject Matter Expert

SOLTRA | An FS-ISAC and DTCC Company

+61 (407) 203 206 | terry@soltra.com

 

 

From: alexander kipnis [mailto:alexander.kipnis85@gmail.com]
Sent: Monday, 7 March 2016 8:20 AM
To: Terry MacDonald <terry@soltra.com>
Cc: cti-users@lists.oasis-open.org
Subject: Re: [cti-users] How does CybOX data is generated?

 

Hi,

 

Thanks for the detailed answer.

 

Correct if I am wrong, but as far as I understand the cyber analyst work is going through logs of some detection systems and identifying abnormal behavior, then creating CybOX observables and generalizing them to STIX Indicator and so on?

 

Further more, it means that cyber analyst crafts these observables by hand (or with some algorithms)?

 

I thought there are some rules like YARA that generate the observables and the analysts try to identify patterns of generalize it to bigger threats.

 

Cheers, 

Alexander

 

 

 

On Mon, Feb 29, 2016 at 12:14 AM, Terry MacDonald <terry@soltra.com> wrote:

Hi Alexander,

 

CybOX Objects are used within STIX and within another protocol called MAEC. STIX is all about sharing threat intel, so it uses CybOX objects to both describe what has happened as well as what you should be looking for. MAEC is more about describing what malware is doing, and uses CybOX objects to describe that.

 

CybOX objects are used in two different ways within STIX. As Observable Instances they record what happened in the past. As Observable Patterns they inform the recipient on what to look for in the future.

 

STIX Indicators use CybOX Observable Instance objects (and Observable Compositions) to describe the logic for things you should be looking for. At present a lot of this pattern development is done manually, and I expect that to happen for some time to come, at least until analysis systems become so smart that they can automatically cluster malicious traffic and software definitively into accurate groupings. Currently you really need an analyst in the loop to understand the malicious behaviour, then craft a pattern that describes it accurately.

 

Detection systems such as IDS will then use those patterns developed to detect and identify situations that require further investigation by incident response staff.

 

Development of new patterns requires that threat intelligence analysts in my opinion requires a hunting capability within your organization: http://detect-respond.blogspot.com.au/2015/10/a-simple-hunting-maturity-model.html. By looking at your logs and looking for anomalies one is able to identify badness, and after establishing the uniqueness that would allow detection of it, one can create a pattern to do exactly that.

 

Cheers

 

Terry MacDonald

Senior STIX Subject Matter Expert

SOLTRA | An FS-ISAC and DTCC Company

+61 (407) 203 206 | terry@soltra.com

 

 

From:cti-users@lists.oasis-open.org [mailto:cti-users@lists.oasis-open.org] On Behalf Of alexander kipnis
Sent: Monday, 29 February 2016 8:55 AM
To: cti-users@lists.oasis-open.org
Subject: [cti-users] How does CybOX data is generated?

 

Hi,

 

I am new to the field.

 

I have read the whitepapers of STIX and CybOX, and currently trying to understand from what data cyber analysts craft the CybOX observables?

 

For example, do they have an IDS system or a phishing detection system from which they extract the observables and then create patterns?

 

It seems like a chicken and egg problem, when one of the main use cases of STIX and CybOX is to create patterns for observables.

 

Thanks,

Alexander Kipnis

 




[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]