[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [kmip-comment] Possible solution to referencing issues
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Mark, I agree that prose can lead to different interpretations but most of the traps spring from poor writing practices. Such as assuming rather than stating the obvious. Such as saying "string" when actually you mean Unicode characters in a particular encoding and no longer than a particular length. String describes both but the more explicit definition avoids one sort of ambiguity. Hope you are having a great day! Patrick PS: I thought an interop test shows that different implementations can process the test cases successfully. Not so much conformance being tested but that different implementations have the same reading of the standard. Not exactly the same as testing conformance, which is against the standard and not another implementation. On 04/01/2014 12:51 PM, Mark Joseph wrote: > > Actually my use of the phrase /"ultimate form of reference"/ is > incorrect. The test cases are not used quite in that way. From > what I have seen during the interop a combination of documents are > used to determine whether something was conforming or not. > > But what I was trying to communicate was that prose alone is > insufficient to define conformance because prose can typically lead > to multiple interpretations. > > Here is a recent example of a new ECMA document :: ECMA-404 > (http://www.ecma-international.org/publications/standards/Ecma-404.htm) > > which defines JSON. > This is basically the same thing that can be found at www.json.org > It is a simple enough specification and uses a degree of formal > definition. > > However, this simple definition has several interpretations. For > example, can JSON start with a top level array element []? The > JSON inventor and others created JsonML which leads one to believe > that the answer is yes, yet others say no. What the document > needs are some examples of what is allowed. (There is also some > confusion about escaping characters a simple query on google will > show you that). The result are several commercially available > non-conforming JSON parsers depending on who you talk to. > > > Best Regards, > > Mark Joseph, Ph.D. President P6R, Inc 408-205-0361 mark@p6r.com > Skype: markjoseph_sc > http://www.linkedin.com/pub/mark-joseph/0/752/4b4 > > > Patrick Durusau<patrick@durusau.net> , 4/1/2014 7:07 AM: > > Mark, > > I think you "hit the nail on the head" when you say: > > ***** What I really liked about KMIP was the conformance test > details. My company P6R ships a KMIP client. And those > conformance test details where extremely helpful in both building > our client and testing with KMIP servers. When there is a > disagreement about what should go over the wire, as there always > is, those detailed test details are the ultimate form of reference > for both server and client. ***** > > The "ultimate form of reference" caught my eye. > > Yes, test cases are very important but test cases only test the set > of values specified. That is if a range of values is allowed, a > test case only tests one or more of those values, not all of them. > > There is no question that test cases are great for "testing" but > "testing" isn't conformance, which is an entirely different > matter. > > Many applications may conform but only some of their capabilities > are tested by any set of test cases. Things of test cases as > subsets of the permissible values. > > Is supporting a subset of permissible values conformance? It's > certainly a sign of possible conformance but what of the values > not tested? > > That to me seems to be the nub of the test cases issue, well, > along with the notion that what isn't specified in the prose is > covered by the test cases. > > The prose really should cover all that needs to be said and then > use test cases to their best advantage as you say, for testing one > aspect of conformance. > > Hope you are having a great day! > > Patrick > > > On 03/31/2014 05:55 PM, Mark Joseph wrote: > > > >> My second related beef, is that the text of the profiles should >> be the authoritative source for what compliance actually means >> (i.e., the normative text). If you have to look at the >> conformance test details, which in ISO would simply be >> "informative" text, you've potentially lost the battle to ensure >> interoperability with KMIP. > > >> Hi Eric, I want to make sure I understand what you are saying >> here. The "conformance test details" you are referring to are >> the detailed test case documents? > >> I have done a lot of work with IETF standards, for example the >> SSH RFCs. I got to tell you they are terrible. There are so >> many different interpretations that when I was building my >> company's version of an SSH server (from scratch) the only way I >> got it to work was to test it against the SSH server on linux. >> There is no other way that would work. I have also worked on >> several IETF email protocols and even though those specs are >> better the only way to get an implementation to work is again by >> testing it against many other implementations. I have been >> doing this protocol development for too long. > > >> What I really liked about KMIP was the conformance test details. >> My company P6R ships a KMIP client. And those conformance test >> details where extremely helpful in both building our client and >> testing with KMIP servers. When there is a disagreement about >> what should go over the wire, as there always is, those detailed >> test details are the ultimate form of reference for both server >> and client. I only wish that the SSH IETF documents had >> something similar. > > >> Best Regards, Mark Joseph, PhD President, P6R Inc. mark@p6r.com > <mailto:mark@p6r.com> > > > > > - -- Patrick Durusau patrick@durusau.net Technical Advisory Board, OASIS (TAB) Co-Chair, OpenDocument Format TC (OASIS) Editor, OpenDocument Format TC, Project Editor ISO/IEC 26300 Former Chair, V1 - US TAG to JTC 1/SC 34 Convener, JTC 1/SC 34/WG 3 (Topic Maps) Co-Editor, ISO 13250-5 (Topic Maps) Another Word For It (blog): http://tm.durusau.net Homepage: http://www.durusau.net Twitter: patrickDurusau -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJTO12LAAoJEAudyeI2QFGouaAP/iVX4u5Gjv+N4KobrrQtza3D Ipv6Mmpn2qHYBBPJJvTGcZpWW36NJ8uo9uLO8fxfoAM5Tr3ALOgWpiX05bxvd8v6 +e+N5CjnJfhXxI1as2PIjS5BE8Gn/l6p+nQ6ZEOeHVexL+oDe1jsqMIdlngDraZW Wwi6vINodowWZqn35/x3ViTFFPCbE3jPelfp6jQP8eoFKpNHOsK2aSFQ25j6Qz9L LUDmtUSaSz+5sjNfP1jkY43zOhzRga2ZwrlOiuPiS5fYZlVFuk5RcKG/YB4krLl6 58++MXjY6CNhw6Xg8bwlTFgOEj9DvsdTWiZEM7VTiQBqXV+vpFXrqlbC3+rpWfOg of1HuN/D/fMg0N4LHmOQGOhj7oyLby0draMJdfje5UKDsq8X2kf6l3swSuYS7J0T hg/vMlbi6EAeE2CM9Rcq4dT50cQwODbCrqGpSWK4suBvpeQazp4y5qdZncuKdStd Mqyl/rr32RpKke2etRkARxKZ0MGMVUvTVgplgMHpL6cwkGha09sNn7C592cJ3g1t 4pmtYvJHT9XzXsveadpdC2EkpxKa2UqsyxKH/rnglAqiroMAPiAsReCPya20+aUF p6lfSZIuHdSPv6bknralLyH9JHs6GMkp874XkgOdDqaEiZJRyBIJAB5qYo33WTDw oxUePE6YhQYyGUsfMoYu =Wv2g -----END PGP SIGNATURE-----
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]