OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.


Help: OASIS Mailing Lists Help | MarkMail Help

cti message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]

Subject: RE: [cti] CTI TC Adoption and Interoperability SCs

I think there’s a middle ground here.


I agree completely with what Bret is saying: I think vendors will want a single dimension (or small set of dimensions) to compete on, because this is all most markets can handle (yes, I expect this statement to be contentious). If, in order to compete on STIX/TAXII functionality, the market needs to understand a complex matrix (nominally, product category x use case x capability level), general customers will not be able to evaluate their options (largely due to time constraints) and the market will instead compete on some other easier to understand dimension (ease of use) or something along the lines of brand recognition and golf outings (ugh).


I also think Ivan and Sean are completely right: The way STIX/CybOX are today, there is not only a single competitive dimension that makes sense – there are many dimensions. You have to break it up to make sense of it, and once you’ve broken it out you can begin to think about maturity levels. I will note that I see this as a natural outcome of the cyber security tool space: There are precious few product categories that vendor products neatly fall into, and many of these products do not compete on the same dimensions.


What I think we need to do, and what I think is the middle ground, is that as we move forward and mature we need to work toward clear, linear dimensions for vendors to compete on and for customers to evaluate. (This is far easier said than done.) I think Ivan’s listing could be a starting point for identifying what those dimensions are. This will be a huge challenge, as it will require pushing the needle in terms of better defining product categories. For instance, if we invented a “host based indicator sharing” dimension to compete on, that’s only valuable if the market knows what product category(ies) should be competing on that dimension.


Thank you.




From: cti@lists.oasis-open.org [mailto:cti@lists.oasis-open.org] On Behalf Of Barnum, Sean D.
Sent: Wednesday, July 08, 2015 10:16 AM
To: Kirillov, Ivan A.; Terry MacDonald; Jordan, Bret
Cc: David Eilken; Richard Struse; Eric Burger; cti@lists.oasis-open.org
Subject: Re: [cti] CTI TC Adoption and Interoperability SCs



I think Ivan hit the nail squarely on the head here. I think this is the only way that a maturity model can be effective. Blurring the lines together will generally mean the maturity levels are misleading.




From: <Kirillov>, Steve Cell <ikirillov@mitre.org>
Date: Wednesday, July 8, 2015 at 8:35 AM
To: Terry MacDonald <terry.macdonald@threatloop.com>, "Jordan, Bret" <bret.jordan@bluecoat.com>
Cc: David Eilken <deilken@fsisac.us>, "Struse, Richard" <Richard.Struse@HQ.DHS.GOV>, Eric Burger <Eric.Burger@georgetown.edu>, "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: Re: [cti] CTI TC Adoption and Interoperability SCs


Interoperability is important, but I’m a little wary of having any sort of abstract system of maturity levels without additional context. The thing is, STIX and CybOX support so many different use cases that just saying that one has to meet a certain threshold with regards to a particular maturity level isn’t terribly useful and in my view could be harmful. For example, if my product is only focused on the generation of simple network indicators (IPs, URLs, etc.), why should I have to support every CybOX Object? Also, just saying that one supports a CybOX Object doesn’t necessarily mean that one supports every field in the Object, which is quite relevant for some of the larger objects out there (e.g., the Windows Executable File). 


Instead, I would suggest that we begin by documenting, even at a high level, the particular use cases that we wish to support with STIX and CybOX. These could then, in turn, have their own native set of maturity levels (defined independently for STIX and/or CybOX as appropriate ). I could envision this as a taxonomy, e.g.,


·         Indicator Characterization/Sharing

o    Host-based Indicator Sharing


§  Level 1: Basic Context

§  Level 2: Level 1 + Supports Sightings

§  CybOX

§  Level 1: Supports File Object

§  File_Path field

§  Hashes field

§  Level 2: Level 1 + Supports Windows Registry Key Object

o    Network Indicator Sharing

·         TTP/Malware Characterization Sharing

o    Basic TTP/Malware Characterization

Still arbitrary, but at least we could then tell vendors that if they want to minimally/somewhat/fully support a particular use case in TAXII they need to do XYZ. Also, if we do go down this road of per-use case based maturity levels, I’d recommend limiting the number of levels to 3 to keep things sane:

·         Level 1: minimal support – very basic, incomplete support

·         Level 2: partial support – typical/average support (with what is commonly expected with regards to the use case)

·         Level 3: full support – fully supports the use case, in all facets





From: Terry MacDonald <terry.macdonald@threatloop.com>
Date: Tuesday, July 7, 2015 at 7:26 PM
To: Bret Jordan <bret.jordan@bluecoat.com>
Cc: David Eilken <deilken@fsisac.us>, Richard Struse <Richard.Struse@HQ.DHS.GOV>, Eric Burger <Eric.Burger@georgetown.edu>, "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
Subject: Re: [cti] CTI TC Adoption and Interoperability SCs


I agree. I like the way this is headed. 


I would like to see each SC produce a list of levels that help guide vendors towards a development roadmap for their products. The number of levels could be different for each SC if that makes sense to the SC, but would also align as closely as possible with the equivalent level of the other SCs e.g. TAXII Level 2 would work with STIX Level 2 and CybOX Level 2. I prefer separate maturity levels per SC as it frees products to support STIX and CybOX, but not TAXII, or to be fully STIX compliant but not support all CybOX objects. It will give vendors a better chance of documenting their maturity more accurately, and as mentioned gives each SC the ability to identify what the requirements of each level are independently.



Terry MacDonald | STIX, TAXII, CybOX Consultant


M: +61-407-203-026



Disclaimer: The opinions expressed within this email do not represent the sentiment of any other party except my own. My views do not necessarily reflect those of my employers.


On 7 July 2015 at 06:55, Jordan, Bret <bret.jordan@bluecoat.com> wrote:

The thing I like about this, if it is simple enough to understand and figure out/test for, is it gives developers/product managers a line in the sand to shoot for.  


I often get asked by many a vendor, "how much of STIX / TAXII do I need to implement to do XYZ?"








Bret Jordan CISSP

Director of Security Architecture and Standards | Office of the CTO

Blue Coat Systems

PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050

"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg." 


On Jul 6, 2015, at 14:50, David Eilken <deilken@fsisac.us> wrote:


Eric, I was just about to chime in that we do need some maturity measurements (vs testing/ validation), but I it didn't sound like you are necessarily in contention with that, and I wouldn't advocate for layers and layers of testing.

More on Maturity Measurements, since we recently took a stab at it (below). To Mark's previous points, there is value in essentially chunking the STIX profile into different levels of maturity and also identifying certain features (STIX Editing for example) as measurements of more mature implementations. Same could be done for TAXII (ingesting STIX from a Feed is simple compared to pushing data back to an Inbox). This approach starts to overlap with the more "qualitative" aspects that Jane referred to.

Level 1: Supports a subset of STIX object types (Indicators). Supports a subset of CybOx object types (observables)
Level 2: In addition to level 1, adds basic context like: title, description, types, date/times, producer, etc.
Level 3: In addition to level 2, adds support for Sightings
Level 4: In addition to level 3, adds all STIX and CybOx object types.
Level 5: In addition to level 4, adds support for STIX object updates and Revocation

Testing and measuring maturity would essentially be separate items. One would test sample data against a submitted profile for validation/ interoperability, and then measure it's maturity.


From: cti@lists.oasis-open.org <cti@lists.oasis-open.org> on behalf of Struse, Richard <Richard.Struse@HQ.DHS.GOV>
Sent: Monday, July 6, 2015 1:09 PM
To: Eric Burger; cti@lists.oasis-open.org
Subject: RE: [cti] CTI TC Adoption and Interoperability SCs

I think we are actually in agreement.  I'm not advocating heavyweight "suites" of exhaustive tests but I do think we will benefit from a shared understanding of the basic outlines of what it means to be interoperable.  Definitely a question that the SC would need to tackle.

-----Original Message-----
From: cti@lists.oasis-open.org [mailto:cti@lists.oasis-open.org] On Behalf Of Eric Burger
Sent: Monday, July 06, 2015 3:47 PM
To: cti@lists.oasis-open.org
Subject: Re: [cti] CTI TC Adoption and Interoperability SCs

Believe it or not, I would shy away from any measurements short of what we say someone needs to do to meet a profile, which is the profile definition itself.

Likewise, I would shy away from test or verification suites. The industry’s experience with standards with elaborate verification suites has been the protocols fail because it takes ages to build the verification suites, they are obsolete when published, which ossifies the underlying specification.

In a decade after everyone is using STIX and TAXII, we can think about formal acceptance suites.

If the specifications are so obfuscated we need specifications to specify what the specifications specify, we have failed.

On Jul 6, 2015, at 3:39 PM, Struse, Richard <Richard.Struse@hq.dhs.gov> wrote:

Makes sense.

If we (the CTI TC) focus on defining what interoperability means for
STIX/TAXII/CybOX and how to measure/verify it, that leaves the door open for
third-party organizations such as testing labs to conduct interoperability
tests using the CTI TC-defined benchmarks.

-----Original Message-----
From: cti@lists.oasis-open.org [mailto:cti@lists.oasis-open.org] On Behalf
Of Eric Burger
Sent: Monday, July 06, 2015 3:18 PM
To: cti@lists.oasis-open.org
Subject: Re: [cti] CTI TC Adoption and Interoperability SCs

When I was Chair of the SPEECHSC IETF Work Group (speech server control), we
setup a self-reported interoperability matrix:

Same for LEMONADE (mobile messaging):

The idea is OASIS will *not* become the protocol police. That is too
expensive and carries unlimited liability. What a self-reporting resource
does do is (1) advertise who is running with the standards and (2) who has
tested against whom. That is not a guarantee of interoperability, but it is
lightyears ahead of publishing and praying.

On Jul 6, 2015, at 2:31 PM, Tony Rutkowski <tony@yaanatech.com> wrote:

It would be useful if CTI maintained the equivalent of
the MILE Implementation Report ID just announced below.

Additionally, in a world of dramatic scaling of cloud data
center virtualization, especially NFV, instantiating a CTI
dedicated focus on the subject is important.




MILE C. Inacio
Internet-Draft CMU
Intended status: Informational                               D. Miyamoto
Expires: January 6, 2016                                          UTokyo
July 5, 2015

                    MILE Implementation Report


This document is a collection of implementation reports from vendors,
consortiums, and researchers who have implemented one or more of the
standards published from the IETF INCident Handling (INCH) and
Management Incident Lightweight Exchange (MILE) working groups.

To unsubscribe from this mail list, you must leave the OASIS TC that
generates this mail.  Follow this link to all your TCs in OASIS at:


To unsubscribe from this mail list, you must leave the OASIS TC that
generates this mail.  Follow this link to all your TCs in OASIS at:



[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]