OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

office message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]


Subject: RE: [office] Formula: test cases


Comments from two perspectives:
 
1. As a linguist (Ph.D., Germanic historical linguistics and comparative literature)
 
In a comment earlier today, someone remarked that English is ambiguous. All human language is intrinsically ambiguous, not just English. (Take a look at a variety of sources from the New Critics to Wittgenstein, or look at the archives of the IEEE Standard Upper Ontology standards committee!)
 
The point of formal mathematical notations is to eliminate the ambiguity of natural language. James Clark has the traditional British university background, complete with the training in philosophy and logic, so he can make himself write things like the definition from XPath that Rob cited. But that definition isn't normal discursive English; it's a translation into words of a formula, the sort of thing that appeared in mathematical texts down through the Renaissance, before modern notations were invented. We have notations now, and it shouldn't be necessary to do everything just in words.
 
I prefer to see standardization done in as formal as possible language, but as I know from developing SGML, and Charles Goldfarb's subsequent efforts to disambiguate himself in "The SGML Handbook", even the best intentioned efforts don't always work. The notational parts of ISO 8879 are much better than the text.
 
2. As a standards developer (26 years in ANSI and ISO)
 
I've never seen "test cases" in a published standard. As I commented in a reply to Patrick yesterday, for SGML we standardized a methodology for reporting test results but stayed entirely away from creating a suite of test cases.
 
For me, the problem with test cases is that they need to provide both necessary and sufficient tests for compliance. Otherwise, they're just examples, not complete tests.
 
See, for example, 6.14.29, "FACT". There's obviously no way to provide an exhaustive set of test cases, since the potential range of values for the argument to the function is infinite. But why this set of test cases? Isn't the meaning of factorial known from any statistics text? What do the cases add that isn't known from standard definitions?
 
Or take some of the more advanced statistical functions, such as 6.16.28, "GAMMADIST". What is not said by the formulae included in the text? Again, why these particular "test cases": are they special cases that must be defined (as is the case of 0! = 1), are they interesting for some particular reasons, or are they sufficient to test all the options for input to the function?
 
To me, the "test cases" appear to be examples, not unlike the examples in the ECMA OOXML. (see, for example, their 15.5.6.115. "FACT", where there are similar things listed as "examples". They appear to assume traditional textbook definitions of functions and so don't give the formulae for things like GAMMADIST in 15.5.6.131.)
 
I don't necessarily object to examples, though very early in the SC34 contacts with ECMA, I complained to a Microsoft representative and later to their editor that their document would have been much easier to handle (and far shorter than its 4000-plus pages) if the examples had been put off into an annex.
 
But I do have a problem with examples, even interesting/representative ones being called "tests" and being part of normative text. If the examples are normative, it means that someone can write code that implements those examples, and just those examples, and claim compliance with the standard, even if the code gets everything else wrong!
 
Given the choice of treating the formula in the present 6.14.31, "GAMMA" as normative and the "text cases" as normative, I'd go with the formula every time. The "test cases" simply aren't sufficient to define the function or even adequately exercise its implementation.
 
I grant that there is a limit to what can be done in notation in the scope of a standard. I watched with considerable amusement the old JTC1/SC18/WG3 in their efforts to do a "Formal Specification of ODA". I goaded them into that because ISO 8613 was originally defined only in discursive prose, and I held up the example of SGML, defined as a production grammar. So they fell into logic notation and tried to write an existence proof for ODA. It was all wheel-spinning, and while they wasted some man-years on the effort they missed the fact that their standard had totally lost out in the marketplace, never even having been implemented.
 
Nonetheless, it's important that what is put in for normative contents in a standard be what you want to be standardized, no more, no less. In some cases, you can provide a piece of sample data that can exactly exercise a piece of code (e.g., the 1024-character limit on cell contents in Excel), but in other cases (such as statistical functions) you simply can't provide sufficient tests. (BTW, hitting the 1024-character limit doesn't cause Excel to crash, just to do odd things: I've had too much experience with engineers who write memos in Excel, rather than Word.)
 
If this document comes forward to ISO with the "test cases" normative, I think you can expect lots of comments on them, including requests simply to remove them.
 
 
 
In commenting on the use of "test cases", I do not intend to criticize the overall quality of the work here. It's much better than its ECMA counterpart. For one thing, it's based on more comprehensive research, looking at a lot more products than MS did for their specification. And it's absolutely necessary to have this work in ODF: When MS first started to stir in ECMA, a certain MS XML specialist (probably known to a number of you) called me up to find out how SC34 would react to their submission. Along the way, he chose to criticize ODF, and one of his main points was that ODF didn't have a formula language. So this specification really needs to get adopted and sent on to ISO to take away his argument.
 
Jim Mason
SC 34 Liaison to OASIS


[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]