OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

kmip message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]


Subject: Re: [kmip] Groups - kmip-cs-profile-v1.0-wd01-review.doc uploaded


On 8/07/2013 1:03 PM, John Leiseboer wrote:
I recommend that the test cases in this profile document be non-normative. My reason for this is that the test cases do not exercise the conformance requirements sufficiently to verify that an implementation is conformant.

None of the profiles with the test cases listed are meant to be an exhaustive regression test for a KMIP server (or client) as if that were the objective the profile documents would most likely all be thousands of pages to cover off on each element referenced by the conformance clauses. They will remain normative as they are necessary but not sufficient as previously discussed.

The conformance clauses are written to cover off on the requirements and the test cases themselves are in addition to those base conformance statements and there are no statements within the documents which state that if you pass the test cases you are conformant - in fact the opposite statement is made - if you don't pass them you are non-conformant (that being one of the conformance statements).

I think you have a fundamentally different view as to the purpose of the profiles and the test cases within them - a view which you haven't expressed until reviewing this particular profile despite the approach being discussed at the face to face and on numerous calls and used in the last round of interop which you participated in. It would be beneficial if you could separate out your meta discussion from the specific review feedback so it is clearer to others what the nature of the feedback is - which items are generic and which items are actually specific to the given profile.

I assume you'll be reviewing all the other profile documents at the same level of detail and providing comments on those too?

I do expect the test cases against profiles to expand over time based on experience with conformance testing - however we do have to have a starting point.
I'll continue to expand on the test cases in the document as time permits but remember the object was to have the test cases documents out to support interop testing - so any additions may not be part of this interop round depending on the consensus view of the participants involved.

I'll attempt to address more of the non-generic items you have raised separately.

Thanks,
Tim.




[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]