OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.


Help: OASIS Mailing Lists Help | MarkMail Help

cti message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]

Subject: Re: [cti] CTI TC Adoption and Interoperability SCs

Eric, I was just about to chime in that we do need some maturity measurements (vs testing/ validation), but I it didn't sound like you are necessarily in contention with that, and I wouldn't advocate for layers and layers of testing. 

More on Maturity Measurements, since we recently took a stab at it (below). To Mark's previous points, there is value in essentially chunking the STIX profile into different levels of maturity and also identifying certain features (STIX Editing for example) as measurements of more mature implementations. Same could be done for TAXII (ingesting STIX from a Feed is simple compared to pushing data back to an Inbox). This approach starts to overlap with the more "qualitative" aspects that Jane referred to.

Level 1: Supports a subset of STIX object types (Indicators). Supports a subset of CybOx object types (observables)
Level 2: In addition to level 1, adds basic context like: title, description, types, date/times, producer, etc.
Level 3: In addition to level 2, adds support for Sightings 
Level 4: In addition to level 3, adds all STIX and CybOx object types.
Level 5: In addition to level 4, adds support for STIX object updates and Revocation

Testing and measuring maturity would essentially be separate items. One would test sample data against a submitted profile for validation/ interoperability, and then measure it's maturity. 


From: cti@lists.oasis-open.org <cti@lists.oasis-open.org> on behalf of Struse, Richard <Richard.Struse@HQ.DHS.GOV>
Sent: Monday, July 6, 2015 1:09 PM
To: Eric Burger; cti@lists.oasis-open.org
Subject: RE: [cti] CTI TC Adoption and Interoperability SCs

I think we are actually in agreement.  I'm not advocating heavyweight "suites" of exhaustive tests but I do think we will benefit from a shared understanding of the basic outlines of what it means to be interoperable.  Definitely a question that the SC would need to tackle.

-----Original Message-----
From: cti@lists.oasis-open.org [mailto:cti@lists.oasis-open.org] On Behalf Of Eric Burger
Sent: Monday, July 06, 2015 3:47 PM
To: cti@lists.oasis-open.org
Subject: Re: [cti] CTI TC Adoption and Interoperability SCs

Believe it or not, I would shy away from any measurements short of what we say someone needs to do to meet a profile, which is the profile definition itself.

Likewise, I would shy away from test or verification suites. The industry’s experience with standards with elaborate verification suites has been the protocols fail because it takes ages to build the verification suites, they are obsolete when published, which ossifies the underlying specification.

In a decade after everyone is using STIX and TAXII, we can think about formal acceptance suites.

If the specifications are so obfuscated we need specifications to specify what the specifications specify, we have failed.

> On Jul 6, 2015, at 3:39 PM, Struse, Richard <Richard.Struse@hq.dhs.gov> wrote:
> Makes sense.
> If we (the CTI TC) focus on defining what interoperability means for
> STIX/TAXII/CybOX and how to measure/verify it, that leaves the door open for
> third-party organizations such as testing labs to conduct interoperability
> tests using the CTI TC-defined benchmarks.
> -----Original Message-----
> From: cti@lists.oasis-open.org [mailto:cti@lists.oasis-open.org] On Behalf
> Of Eric Burger
> Sent: Monday, July 06, 2015 3:18 PM
> To: cti@lists.oasis-open.org
> Subject: Re: [cti] CTI TC Adoption and Interoperability SCs
> When I was Chair of the SPEECHSC IETF Work Group (speech server control), we
> setup a self-reported interoperability matrix:
> http://standardstrack.com/ietf/speechsc/MRCPv2-Plans.html
> Same for LEMONADE (mobile messaging):
> http://standardstrack.com/ietf/lemonade/Lemonade-Plans.html
> The idea is OASIS will *not* become the protocol police. That is too
> expensive and carries unlimited liability. What a self-reporting resource
> does do is (1) advertise who is running with the standards and (2) who has
> tested against whom. That is not a guarantee of interoperability, but it is
> lightyears ahead of publishing and praying.
>> On Jul 6, 2015, at 2:31 PM, Tony Rutkowski <tony@yaanatech.com> wrote:
>> It would be useful if CTI maintained the equivalent of
>> the MILE Implementation Report ID just announced below.
>> Additionally, in a world of dramatic scaling of cloud data
>> center virtualization, especially NFV, instantiating a CTI
>> dedicated focus on the subject is important.
>> --tony
>> ---------------
>> https://tools.ietf.org/html/draft-ietf-mile-implementreport-05
>> MILE C. Inacio
>> Internet-Draft CMU
>> Intended status: Informational                               D. Miyamoto
>> Expires: January 6, 2016                                          UTokyo
>> July 5, 2015
>>                      MILE Implementation Report
>>                  draft-ietf-mile-implementreport-05
>> Abstract
>>  This document is a collection of implementation reports from vendors,
>>  consortiums, and researchers who have implemented one or more of the
>>  standards published from the IETF INCident Handling (INCH) and
>>  Management Incident Lightweight Exchange (MILE) working groups.
>> ---------------------------------------------------------------------
>> To unsubscribe from this mail list, you must leave the OASIS TC that
>> generates this mail.  Follow this link to all your TCs in OASIS at:
>> https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]