[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [kmip-comment] Possible solution to referencing issues
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Mark, I think you "hit the nail on the head" when you say: ***** What I really liked about KMIP was the conformance test details. My company P6R ships a KMIP client. And those conformance test details where extremely helpful in both building our client and testing with KMIP servers. When there is a disagreement about what should go over the wire, as there always is, those detailed test details are the ultimate form of reference for both server and client. ***** The "ultimate form of reference" caught my eye. Yes, test cases are very important but test cases only test the set of values specified. That is if a range of values is allowed, a test case only tests one or more of those values, not all of them. There is no question that test cases are great for "testing" but "testing" isn't conformance, which is an entirely different matter. Many applications may conform but only some of their capabilities are tested by any set of test cases. Things of test cases as subsets of the permissible values. Is supporting a subset of permissible values conformance? It's certainly a sign of possible conformance but what of the values not tested? That to me seems to be the nub of the test cases issue, well, along with the notion that what isn't specified in the prose is covered by the test cases. The prose really should cover all that needs to be said and then use test cases to their best advantage as you say, for testing one aspect of conformance. Hope you are having a great day! Patrick On 03/31/2014 05:55 PM, Mark Joseph wrote: > > > > My second related beef, is that the text of the profiles should be > the authoritative source for what compliance actually means (i.e., > the normative text). If you have to look at the conformance test > details, which in ISO would simply be "informative" text, you've > potentially lost the battle to ensure interoperability with KMIP. > > > Hi Eric, I want to make sure I understand what you are saying here. > The "conformance test details" you are referring to are the > detailed test case documents? > > I have done a lot of work with IETF standards, for example the SSH > RFCs. I got to tell you they are terrible. There are so many > different interpretations that when I was building my company's > version of an SSH server (from scratch) the only way I got it to > work was to test it against the SSH server on linux. There is no > other way that would work. I have also worked on several IETF > email protocols and even though those specs are better the only way > to get an implementation to work is again by testing it against > many other implementations. I have been doing this protocol > development for too long. > > > What I really liked about KMIP was the conformance test details. > My company P6R ships a KMIP client. And those conformance test > details where extremely helpful in both building our client and > testing with KMIP servers. When there is a disagreement about > what should go over the wire, as there always is, those detailed > test details are the ultimate form of reference for both server and > client. I only wish that the SSH IETF documents had something > similar. > > > Best Regards, Mark Joseph, PhD President, P6R Inc. mark@p6r.com > > > - -- Patrick Durusau patrick@durusau.net Technical Advisory Board, OASIS (TAB) Co-Chair, OpenDocument Format TC (OASIS) Editor, OpenDocument Format TC, Project Editor ISO/IEC 26300 Former Chair, V1 - US TAG to JTC 1/SC 34 Convener, JTC 1/SC 34/WG 3 (Topic Maps) Co-Editor, ISO 13250-5 (Topic Maps) Another Word For It (blog): http://tm.durusau.net Homepage: http://www.durusau.net Twitter: patrickDurusau -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJTOsgCAAoJEAudyeI2QFGolHwQAIsDOZEcrqott9415a/u/SjY v4RsWaoN8L8g+8zMEMFGfDzF7WsYWqQrq8QY9h+zMf/RSn/B0K+M0VP0NwbKhM7X a36YF0yXiaewssk3XkeCldhIobxV7gqH7hDxKHGXHMLcfXfP2fWgYaMRiZaZtQ1P uvXUGJpg6szdKI6/5wX6r6QXVPAjgCqli+nRyaOhTVRYIQC7HKuho0sgzpAco0Vw io4UHyLUyeuYHKMoEKeOU3HwubaGWXpT69qsgVQfdfs5fKZoEqCik7pKAmK5ZqJw Rd4INROdoz/ngDZ6dJn7xuFoSAhc+1WQAGwS0mspH8GBI18ttPgYBgf35/kapctN 6tKWjov7TjKeg7lim0uMyNNgv4xcBQLbWqfRG3EAWu+7+BiS9eHtYHch9LoOJk+x 8/RUJZ1aH2wugu9SsvO4wXjYUmlpoDqFoCK/SmH79NN6bNtvbGLyYlfqHIylizBy lIUf1UK8r096WVNFgVl90oL8X354FRvMM4SvAySb5XwsEJ5rwIr9PzXsY+/foQlt 0Jo+C4qJCTxMP6B6mG81f0puLEgjCjwl18gf4Anb/9PV0ANtHxpvaOQH1V3EWnPi O1DN5GYF/UGVl4xinZnnXeHxmGewRMkdOQ4rRnsjzudkoHQ2tbQv90WZCCGfNU+0 O6XE2tfvB2L07FnQqqDv =t5nz -----END PGP SIGNATURE-----
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]