RE: [kmip-comment] Possible solution to referencing issues

["Mark Joseph" <mark@p6r.com>]

You need some reference implementation but who is going to build that it is expensive.   So the next best thing is what other vendors have implemented.   In the IETF world that is acceptable so we are not the first or only group to do so.   


Mark Joseph, PhD
President
P6R, Inc
408-205-0361
www.p6r.com



-------- Original message --------
From: Patrick Durusau <patrick@durusau.net> 
Date:04/01/2014  5:46 PM  (GMT-08:00) 
To: Mark Joseph <mark@p6r.com>,Eric Hibbard <Eric.Hibbard@hds.com>,Tim Hudson <tjh@cryptsoft.com> 
Cc: kmip <kmip-comment@lists.oasis-open.org>,tab@lists.oasis-open.org 
Subject: Re: [kmip-comment] Possible solution to referencing issues 

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Mark,

I agree that prose can lead to different interpretations but most of
the traps spring from poor writing practices. Such as assuming rather
than stating the obvious.

Such as saying "string" when actually you mean Unicode characters in a
particular encoding and no longer than a particular length. String
describes both but the more explicit definition avoids one sort of
ambiguity.

Hope you are having a great day!

Patrick

PS: I thought an interop test shows that different implementations can
process the test cases successfully. Not so much conformance being
tested but that different implementations have the same reading of the
standard. Not exactly the same as testing conformance, which is
against the standard and not another implementation.

On 04/01/2014 12:51 PM, Mark Joseph wrote:
> 
> Actually my use of the phrase /"ultimate form of reference"/ is 
> incorrect.   The test cases are not used quite in that way.   From
> what I have seen during the interop a combination of documents are
> used to determine whether something was conforming or not.
> 
> But what I was trying to communicate was that prose alone is 
> insufficient to define conformance because prose can typically lead
> to multiple interpretations.
> 
> Here is a recent example of a new ECMA document :: ECMA-404 
> (http://www.ecma-international.org/publications/standards/Ecma-404.htm)
>
> 
which defines JSON.
> This is basically the same thing that can be found at www.json.org 
> It is a simple enough specification and uses a degree of formal
> definition.
> 
> However, this simple definition has several interpretations. For 
> example, can JSON start with a top level array element []?   The
> JSON inventor and others created JsonML which leads one to believe
> that the answer is yes, yet others say no.   What the document
> needs are some examples of what is allowed.  (There is also some
> confusion about escaping characters a simple query on google will
> show you that).   The result are several commercially available
> non-conforming JSON parsers depending on who you talk to.
> 
> 
> Best Regards,
> 
> Mark Joseph, Ph.D. President P6R, Inc 408-205-0361 mark@p6r.com 
> Skype: markjoseph_sc 
> http://www.linkedin.com/pub/mark-joseph/0/752/4b4
> 
> 
> Patrick Durusau<patrick@durusau.net> , 4/1/2014 7:07 AM:
> 
> Mark,
> 
> I think you "hit the nail on the head" when you say:
> 
> ***** What I really liked about KMIP was the conformance test
> details.   My company P6R ships a KMIP client.   And those
> conformance test details where extremely helpful in both building
> our client and testing with KMIP servers.   When there is a
> disagreement about what should go over the wire, as there always
> is, those detailed test details are the ultimate form of reference
> for both server and client. *****
> 
> The "ultimate form of reference" caught my eye.
> 
> Yes, test cases are very important but test cases only test the set
> of values specified. That is if a range of values is allowed, a
> test case only tests one or more of those values, not all of them.
> 
> There is no question that test cases are great for "testing" but 
> "testing" isn't conformance, which is an entirely different
> matter.
> 
> Many applications may conform but only some of their capabilities
> are tested by any set of test cases. Things of test cases as
> subsets of the permissible values.
> 
> Is supporting a subset of permissible values conformance? It's 
> certainly a sign of possible conformance but what of the values
> not tested?
> 
> That to me seems to be the nub of the test cases issue, well,
> along with the notion that what isn't specified in the prose is
> covered by the test cases.
> 
> The prose really should cover all that needs to be said and then
> use test cases to their best advantage as you say, for testing one
> aspect of conformance.
> 
> Hope you are having a great day!
> 
> Patrick
> 
> 
> On 03/31/2014 05:55 PM, Mark Joseph wrote:
> 
> 
> 
>> My second related beef, is that the text of the profiles should
>> be the authoritative source for what compliance actually means
>> (i.e., the normative text). If you have to look at the
>> conformance test details, which in ISO would simply be
>> "informative" text, you've potentially lost the battle to ensure
>> interoperability  with KMIP.
> 
> 
>> Hi Eric, I want to make sure I understand what you are saying
>> here. The "conformance test details" you are referring to are
>> the detailed test case documents?
> 
>> I have done a lot of work with IETF standards, for example the
>> SSH RFCs.   I got to tell you they are terrible.   There are so
>> many different interpretations that when I was building my
>> company's version of an SSH server (from scratch) the only way I
>> got it to work was to test it against the SSH server on linux.
>> There is no other way that would work.    I have also worked on
>> several IETF email protocols and even though those specs are
>> better the only way to get an implementation to work is again by
>> testing it against many other implementations.   I have been
>> doing this protocol development for too long.
> 
> 
>> What I really liked about KMIP was the conformance test details. 
>> My company P6R ships a KMIP client.   And those conformance test 
>> details where extremely helpful in both building our client and 
>> testing with KMIP servers.   When there is a disagreement about 
>> what should go over the wire, as there always is, those detailed 
>> test details are the ultimate form of reference for both server
>> and client. I only wish that the SSH IETF documents had
>> something similar.
> 
> 
>> Best Regards, Mark Joseph, PhD President, P6R Inc. mark@p6r.com
> <mailto:mark@p6r.com>
> 
> 
> 
> 
> 

- -- 
Patrick Durusau
patrick@durusau.net
Technical Advisory Board, OASIS (TAB)
Co-Chair, OpenDocument Format TC (OASIS)
Editor, OpenDocument Format TC, Project Editor ISO/IEC 26300
Former Chair, V1 - US TAG to JTC 1/SC 34
Convener, JTC 1/SC 34/WG 3 (Topic Maps)
Co-Editor, ISO 13250-5 (Topic Maps)

Another Word For It (blog): http://tm.durusau.net
Homepage: http://www.durusau.net
Twitter: patrickDurusau
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJTO12LAAoJEAudyeI2QFGouaAP/iVX4u5Gjv+N4KobrrQtza3D
Ipv6Mmpn2qHYBBPJJvTGcZpWW36NJ8uo9uLO8fxfoAM5Tr3ALOgWpiX05bxvd8v6
+e+N5CjnJfhXxI1as2PIjS5BE8Gn/l6p+nQ6ZEOeHVexL+oDe1jsqMIdlngDraZW
Wwi6vINodowWZqn35/x3ViTFFPCbE3jPelfp6jQP8eoFKpNHOsK2aSFQ25j6Qz9L
LUDmtUSaSz+5sjNfP1jkY43zOhzRga2ZwrlOiuPiS5fYZlVFuk5RcKG/YB4krLl6
58++MXjY6CNhw6Xg8bwlTFgOEj9DvsdTWiZEM7VTiQBqXV+vpFXrqlbC3+rpWfOg
of1HuN/D/fMg0N4LHmOQGOhj7oyLby0draMJdfje5UKDsq8X2kf6l3swSuYS7J0T
hg/vMlbi6EAeE2CM9Rcq4dT50cQwODbCrqGpSWK4suBvpeQazp4y5qdZncuKdStd
Mqyl/rr32RpKke2etRkARxKZ0MGMVUvTVgplgMHpL6cwkGha09sNn7C592cJ3g1t
4pmtYvJHT9XzXsveadpdC2EkpxKa2UqsyxKH/rnglAqiroMAPiAsReCPya20+aUF
p6lfSZIuHdSPv6bknralLyH9JHs6GMkp874XkgOdDqaEiZJRyBIJAB5qYo33WTDw
oxUePE6YhQYyGUsfMoYu
=Wv2g
-----END PGP SIGNATURE-----