Formula: test cases

[Patrick Durusau <patrick@durusau.net>]

Greetings!

I am deeply uncertain about the test cases for reasons similar to my 
concerns about the "hidden" text. If the required behavior is 
sufficiently specified, then the test cases should not be adding 
anything to the standard. That is to say if I implement the normative 
text, it should not be possible to obtain a result that is inconsistent 
with the test cases.

No doubt the test cases are very valuable as in drafting or 
post-adoption, implementors can test their implementations to insure 
that implementing the normative text does result in the correct 
outcome(s), but that really isn't part of the normative text. That is to 
say that I should not simply implement the test cases and think that I 
have implemented the normative text. If someone "cheats" in that 
fashion, they may well encounter an edge case that is not covered by the 
test cases but is covered by the normative language.

So, test cases are invaluable, but I am leaning towards suggesting that 
they should not appear in the normative text of the standard. Actually I 
am not entirely sure they should even appear in a non-normative annex. 
In part because if there is any conflict between the normative text and 
the test cases, which controls? The normative language or the test cases?

That is one reason why standards strive to only say any rule once and 
only once. Not entirely possible if you want a readable result but it is 
something that is a good rule to follow in general. That reduces the 
grounds for reaching different interpretations.

Actually I would argue that if I need the test cases to understand the 
normative text, that is a good sign there is a problem with the 
normative text.

I am starting a slow read of the formula proposal this weekend and will 
post comments on specific sections as I reach them. This and the prior 
post on notes, rationales, etc., are general comments that I won't 
repeat as I encounter those aspects of the various sections.

Hope everyone is having a great day!

Patrick

PS: Actually the test cases offer an interesting way to proof the 
normative text. Remember the "check yourself exercises" in textbooks? 
Simply cover up the results of the test cases and after reading the 
normative text, see if you get the same results as are set forth in the 
test cases. If you don't, it might indicate a problem with the normative 
text. Will take longer but should result in a very clean normative text.

-- 
Patrick Durusau
Patrick@Durusau.net
Chair, V1 - Text Processing: Office and Publishing Systems Interface
Co-Editor, ISO 13250, Topic Maps -- Reference Model
Member, Text Encoding Initiative Board of Directors, 2003-2005

Topic Maps: Human, not artificial, intelligence at work!