[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [cti] RE: [EXT] Re: [cti] TAXII definition of "Done"
Drew - we have generally defined 'interop tests" as "tests added to a STIXPreferred Interoperability Test Document used for certification".
Allan Thomson
CTO (+1-408-331-6646)
LookingGlass Cyber Solutions <http://www.lookingglasscyber.com/>
ïOn 12/10/18, 10:04 AM, "Drew Varner" <drew.varner@ninefx.com> wrote:
Allan,
When you say âsome interoperability testsâ, do you mean in the STIX/TAXII Preferred Interoperability Test suite or something else? I feel like the Preferred Interop Tests are a balance between interoperability and encouraging vendor adoption. Many features are not in the current test suites.
Thanks,
Drew
> On Dec 10, 2018, at 10:16 AM, Allan Thomson <athomson@lookingglasscyber.com> wrote:
>
> Bret â hereâs some input on the TAXII criteria
>
> â Two sponsors
> â AT: At least 1 TXS and 1 TAXII Client (different orgs). If we wanted to be consistent with STIX then it should really be 2 TXS and at least 1 TAXII Client.
> â Needs to be implemented in two different code bases
> â Agreed with at least 1 on server-side and 1 on client-side from different orgs
> â Needs to have proof-of-concept code
> â Agreed on both server and client sides.
> â Needs all specification text and some interoperability tests
> â Agreed (on both server and client sides).
>
> Q: What constitutes a new feature?
>
> AT: This is somewhat subjective based on the change but here some suggestions:
> â One or more new additional endpoints/methods to support change
> â Non-compatible (forward or backward) change to previous version of TAXII
> â Additional semantic behavior on existing method/endpoints (with possible additional arguments to support) that is more complex than a few lines of code to support
>
> Allan Thomson
> CTO (+1-408-331-6646)
> LookingGlass Cyber Solutions
>
> From: "cti@lists.oasis-open.org" <cti@lists.oasis-open.org> on behalf of Bret Jordan <Bret_Jordan@symantec.com>
> Date: Saturday, December 8, 2018 at 1:51 PM
> To: John-Mark Gurney <jmg@newcontext.com>, Matt Pladna <mpladna@lookingglasscyber.com>
> Cc: "Kelley, Sarah E." <skelley@mitre.org>, "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
> Subject: Re: [cti] RE: [EXT] Re: [cti] TAXII definition of "Done"
>
> So for STIX we have a rule system that new features need:
>
> 1) Two sponsors
> 2) Needs to be implemented in two different code bases
> 3) Needs to have proof-of-concept code
> 4) Needs all specification text and some interoperability tests
>
> A question for TAXII is, is this rule set still applicable? Or does it need some changes? If so, what constitutes a new feature? In STIX this is generally understood to mean a new object. but what is the equivalent in TAXII. Also additional properties, changes to properties, new relationship types, new vocab entries do not have a formal process for STIX. So where is that line for TAXII ?
>
> Clearly I think there are some things that we could easily identify as candidates for a more formal vetting process, those would be things like a solution for Query and TAXII Channels. But what else? Does a new endpoint need this? If so, are there times when it would not?
>
> Just some questions for the TC to think about. Also, as TAXII 2.1 does not currently have anything "new", just a bunch of changes / fixes from 2.0, I am not sure if this process is needed for 2.1, but I would be happy to hear if people think differently.
>
> One of the things that is not yet addressed is how does the TC decide to work on something new, either for STIX or TAXII, also, how does the TC decide if something should be included in a release of STIX or TAXII? I wrote up a draft proposal for this, that we may also need to address at the same time as any process changes for TAXII.
>
> Bret
> From: John-Mark Gurney <jmg@newcontext.com>
> Sent: Friday, December 7, 2018 5:38:37 PM
> To: Matt Pladna
> Cc: Bret Jordan; Kelley, Sarah E.; cti@lists.oasis-open.org
> Subject: Re: [cti] RE: [EXT] Re: [cti] TAXII definition of "Done"
>
> Matt Pladna wrote this message on Fri, Dec 07, 2018 at 19:08 +0000:
> > I support having TAXII meet some definition of Done Kelley.
>
> I agree that TAXII needs to have the same definition of done as the rest
> of the TC...
>
> > Also, isnât this a process change vs a feature change to the spec? Or is the point that there should be a vote for this before itâs decided?
> >
> > What would the # of people out of 200 be required to make this happen?
> >
> > From: <cti@lists.oasis-open.org> on behalf of Bret Jordan <Bret_Jordan@symantec.com>
> > Date: Friday, December 7, 2018 at 14:04
> > To: "Kelley, Sarah E." <skelley@mitre.org>
> > Cc: "cti@lists.oasis-open.org" <cti@lists.oasis-open.org>
> > Subject: Re: [cti] RE: [EXT] Re: [cti] TAXII definition of "Done"
> >
> > And as I said in my last email...
> >
> > âWhile I fundamentally do not disagree, I have yet to see the TC push for this. By this philosophy we should have adopted the whole content process that I sent to the TC as a fully developed working draft several months ago.
> >
> >
> > So unless there is a ground swell of the TC that is pushing for this, I would object simply out of principle. This is the same reason why tons of new things are not just added to TAXII, there is no demand for them and it is not up to me to just add them.â
> >
> >
> > So I object on principle. You have 4 people out of over 200 which is less than 2%. The TC has made it really clear as of late that talking about process is not something people want to do. And if we are going to try and talk about process then once again I want to bring up the fully fleshed out Draft I submitted to the TC, which basically codifies what we have been doing behind the scenes.
> >
> > Bret
> >
> > Sent from my Commodore 128D
> >
> >
> > PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050
> >
> > On Dec 7, 2018, at 10:08 AM, Kelley, Sarah E. <skelley@mitre.org<mailto:skelley@mitre.org>> wrote:
> > Bret,
> >
> > As of last week, the people in support of having TAXII meet some definition of âdoneâ was:
> >
> > Sarah Kelley
> > Jason Keirstead
> > John Wunder
> > Allan Thompson
> > and you:
> >
> > âI am fine with that (since this is what I am doing behind the scenes anyways), but this would need to be taken to a ballot just like we did for STIX. It would need to be binding, not just a casual agreement. â
> > Since no one objected to the idea, and at least five people supported it, the goal was to move the ball further down the court and see if there was an appetite for taking this to a ballot and making it more official, and if so, to figure out when we might want to do that.
> >
> > Thanks,
> >
> > Sarah Kelley
> > Lead Cybersecurity Engineer, T8B2
> > Defensive Operations
> > The MITRE Corporation
> > 703-983-6242
> > skelley@mitre.org<mailto:skelley@mitre.org>
> > <image003.jpg><https://clicktime.symantec.com/a/1/vncIdxSA3A4xal5shRajaDYU9DQpDQJrpTsBsIQYQT0=?d=ItDYMdKTUJlWiicY2nDNmyqFNII0tqf5Pi46Zp5KtCLRpKQg8SNHxmyrlL4EOK4HmyAfOTgA4Q9Q7Sk0NDox8-zxxIz01Og0Pzj2DbzvbUAHkcOYLaJmH8mwz26zgSn0vjNSh4H_Dyfm4QC2Mbdq9QoqElEWvgQbWogiLFm5Ib_KzA71P_uzGGFrFwNoWvmw9hzRDf4YkoRQIs6_i-PY_efoVGFnvmd0zR-hTYe9BuXxA2HnUrceX2a1Qo29Zz2a62Mt-1izHM1GYUOPNc8WYE7fX_U6X2mM8X1dxd9clDG5VBh16ZCTJrdh9EpEaLncI2gFzkxbpXKvjUblkAzmk8P26RPSjXyxAbiPSwJ1mpIQlKHc_Pw8b-fI5UD9qPMvQ23J_9yPEPVovKQkVokgNTYbqgxSrKx34F6wJ40Mf6NxiBTNWwnTkWdlNupAMVJ-GL5oEvDFzN_J7Fp8jML54Z3FnrTkgAK0csiCzz6KjH4ECxMIgpUVKsO_&u=http%3A%2F%2Fwww.mitre.org%2F>
> >
> > From: Bret Jordan <Bret_Jordan@symantec.com<mailto:Bret_Jordan@symantec.com>>
> > Sent: Thursday, December 6, 2018 4:58 PM
> > To: Kelley, Sarah E. <skelley@mitre.org<mailto:skelley@mitre.org>>
> > Cc: cti@lists.oasis-open.org<mailto:cti@lists.oasis-open.org>
> > Subject: Re: [cti] RE: [EXT] Re: [cti] TAXII definition of "Done"
> >
> > In all things consensus based, there is the âdoes anyone objectâ and two, âwho supports and is driving thisâ.
> >
> > It is generally not good form to do things by âobjectionâ but rather first by demand and then by objection.
> >
> > While I fundamentally do not disagree, I have yet to see the TC push for this. By this philosophy we should have adopted the whole content process that I sent to the TC as a fully developed working draft several months ago.
> >
> > So unless there is a ground swell of the TC that is pushing for this, I would object simply out of principle. This is the same reason why tons of new things are not just added to TAXII, there is no demand for them and it is not up to me to just add them.
> >
> > Bret
> > Sent from my Commodore 128D
> >
> >
> >
> > PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050
> >
> > On Dec 7, 2018, at 2:23 AM, Kelley, Sarah E. <skelley@mitre.org<mailto:skelley@mitre.org>> wrote:
> > All,
> >
> > Having seen no objections to the idea of instituting a mandate of âdoneâ for TAXII (in addition to STIX), I believe the next step would be to decide when we want to institute that policy. As with STIX, the best way to institute that new policy would be to have a ballot on it, so we would need to decide when to open that ballot.
> >
> > In my understanding of the changes in the current WD that is open for ballot, the only new âthingâ is the client user-agent. From my perspective, this seems like a relatively small change to hold up with the addition of this new process, however something like TAXII query would make sense to have proven out in code and to pre-build interop tests for.
> >
> > What do the TC members think about when we should start the ball rolling on implementing this policy?
> >
> > Thanks,
> >
> > Sarah Kelley
> > Lead Cybersecurity Engineer, T8B2
> > Defensive Operations
> > The MITRE Corporation
> > 703-983-6242
> > skelley@mitre.org<mailto:skelley@mitre.org>
> > <image003.jpg><https://clicktime.symantec.com/a/1/lwDjxdQuulFcm7NdywUBnSZUR_C1zd9konyJfqfsEsY=?d=_VqGAtgxpuV0IihxvFhFPcPCaw5WlSzgXBXhLEDnfl0u1wBn3yk1h15q_FmLrtE2IKq8fovIPVep9N5rUqa9MYOg6MPIL_72-oESgkCFJAhRBrhiOGER2BCH092BrZADxR7BfgcE8XUGtnZ_aGIn9XdnAr7sMvxWBPPiefl46oMUf6n1dPEMqpwCA2PYJd7gdLuVwamjpR9GjZs3HxVD4GShJkU6ZoLo0NCwqlnqLUVGVVN_GhRrf4XYLSz3nXfwy6LjN2ttIw1NooiV5UeVKbFSD5u7d5UQaZYc7zxAGWLmx6cXKeK985MAXXVtMhXpAjGKUSYS2NlqSIjJ02egbSUDJOP8l84V9I0hjzF_YPTjiADe1PqA9B-eXthYSC_fqFGJvjziJHjrU_xFIBCfq2kEKtnWnh6lxlqxP7ScyML01NwaNvpF3pdn8JurxILn8fh1lwlFlvY5lLPfjUUE79nsZuDvqyFcVvCKBqW8KzL8nR1IN9E%3D&u=http%3A%2F%2Fwww.mitre.org%2F>
> >
> > From: Bret Jordan <Bret_Jordan@symantec.com<mailto:Bret_Jordan@symantec.com>>
> > Sent: Tuesday, November 27, 2018 5:29 PM
> > To: Allan Thomson <athomson@lookingglasscyber.com<mailto:athomson@lookingglasscyber.com>>; Wunder, John A. <jwunder@mitre.org<mailto:jwunder@mitre.org>>; Jason Keirstead <Jason.Keirstead@ca.ibm.com<mailto:Jason.Keirstead@ca.ibm.com>>; Kelley, Sarah E. <skelley@mitre.org<mailto:skelley@mitre.org>>
> > Cc: cti@lists.oasis-open.org<mailto:cti@lists.oasis-open.org>
> > Subject: Re: [EXT] Re: [cti] TAXII definition of "Done"
> >
> >
> > I am fine with that (since this is what I am doing behind the scenes anyways), but this would need to be taken to a ballot just like we did for STIX. It would need to be binding, not just a casual agreement.
> >
> >
> >
> > What I am doing right now is making sure every feature that gets added to TAXII is actually implemented in my libraries and test server (at some level). I am doing this to help prevent the problems we had with TAXII 2.0, where 20 minutes in to coding we realized that, that design does not work in code. Some of the issues we have resolved in TAXII 2.1 have come about because of this code work that I and others have done and the plugfests we have held. I am a firm believer in "working code" and easy to implement in code. I think those are two of the pillars to adoption.
> >
> >
> >
> > One of the differences we have in TAXII versus STIX though is, TAXII does not have features that are just conceptual models. STIX on the other hand can just be "modeled" and not implemented. This is why it was so important to have the "written in code" clause for STIX.
> >
> >
> >
> > Bret
> >
> > ________________________________
> > From: cti@lists.oasis-open.org<mailto:cti@lists.oasis-open.org> <cti@lists.oasis-open.org<mailto:cti@lists.oasis-open.org>> on behalf of Allan Thomson <athomson@lookingglasscyber.com<mailto:athomson@lookingglasscyber.com>>
> > Sent: Tuesday, November 27, 2018 2:26:44 PM
> > To: Wunder, John A.; Jason Keirstead; Kelley, Sarah E.
> > Cc: cti@lists.oasis-open.org<mailto:cti@lists.oasis-open.org>
> > Subject: [EXT] Re: [cti] TAXII definition of "Done"
> >
> >
> > +1 to TAXII features starting to require the same level of doneness as STIX changes.
> >
> >
> >
> > Allan
> >
> >
> >
> > From: "cti@lists.oasis-open.org<mailto:cti@lists.oasis-open.org>" <cti@lists.oasis-open.org<mailto:cti@lists.oasis-open.org>> on behalf of "Wunder, John" <jwunder@mitre.org<mailto:jwunder@mitre.org>>
> > Date: Tuesday, November 27, 2018 at 1:21 PM
> > To: Jason Keirstead <Jason.Keirstead@ca.ibm.com<mailto:Jason.Keirstead@ca.ibm.com>>, "Kelley, Sarah E." <skelley@mitre.org<mailto:skelley@mitre.org>>
> > Cc: "cti@lists.oasis-open.org<mailto:cti@lists.oasis-open.org>" <cti@lists.oasis-open.org<mailto:cti@lists.oasis-open.org>>
> > Subject: Re: [cti] TAXII definition of "Done"
> >
> >
> >
> > Agreed, the same motivation for wanting to do this for STIX applies to TAXII. Iâd also keep in mind that requiring sponsors and interop text makes it so that youâre not just evaluating technical feasibility (the implementation piece), youâre also ensuring that thereâs defined use cases and a real scenario where it can be used (a concern discussed on the call). Itâs way easier to say yes to something new than to say no, so itâs important to have these checks in place to make sure we donât end up with something overly broad again.
> >
> >
> >
> > John
> >
> >
> >
> > From: <cti@lists.oasis-open.org<mailto:cti@lists.oasis-open.org>> on behalf of Jason Keirstead <Jason.Keirstead@ca.ibm.com<mailto:Jason.Keirstead@ca.ibm.com>>
> > Date: Tuesday, November 27, 2018 at 4:15 PM
> > To: "Kelley, Sarah E." <skelley@mitre.org<mailto:skelley@mitre.org>>
> > Cc: "cti@lists.oasis-open.org<mailto:cti@lists.oasis-open.org>" <cti@lists.oasis-open.org<mailto:cti@lists.oasis-open.org>>
> > Subject: Re: [cti] TAXII definition of "Done"
> >
> >
> >
> > I would also agree that TAXII features should also meet the STIX definition of "done" in order to be included in the spec.
> >
> > -
> > Jason Keirstead
> > Lead Architect - IBM Security Connect
> > www.ibm.com/security
> >
> > "Things may come to those who wait, but only the things left by those who hustle." - Unknown
> >
> >
> >
> >
> > From: "Kelley, Sarah E." <skelley@mitre.org<mailto:skelley@mitre.org>>
> > To: "cti@lists.oasis-open.org<mailto:cti@lists.oasis-open.org>" <cti@lists.oasis-open.org<mailto:cti@lists.oasis-open.org>>
> > Date: 11/27/2018 04:56 PM
> > Subject: [cti] TAXII definition of "Done"
> > Sent by: <cti@lists.oasis-open.org<mailto:cti@lists.oasis-open.org>>
> >
> > ________________________________
> >
> >
> >
> > All,
> >
> > As I mentioned on the working call today, we have imposed a very strict definition of âDoneâ for new features/objects in STIX, however, we have never agreed as a TC to impose the same rigorous standards to TAXII. Given the fact that some of the issues that prompted us to implement this definition came about when people attempted to implement TAXII, it seems only logical to me that we would impose the same standards to both specifications.
> >
> > As a reminder, the definition of âDoneâ for STIX includes:
> >
> > 1. Written specification text
> >
> > 1. Proof of concept code from at least two different developers/companies
> >
> > 1. Corresponding Interop tests
> >
> >
> > For some of the newer features in TAXII, namely TAXII query, it seems to make sense to me that it should be proved in code before we finalize it in the specification.
> >
> > I wanted to bring this topic to the list and see what other people thought about this.
> >
> > Thanks,
> >
> > Sarah Kelley
> > Lead Cybersecurity Engineer, T8B2
> > Defensive Operations
> > The MITRE Corporation
> > 703-983-6242
> > skelley@mitre.org<mailto:skelley@mitre.org>
> >
> > [attachment "image003.jpg" deleted by Jason Keirstead/CanEast/IBM]
> >
> >
> >
> >
> >
>
> --
> John-Mark
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]