office message
[Date Prev]
| [Thread Prev]
| [Thread Next]
| [Date Next]
--
[Date Index]
| [Thread Index]
| [List Home]
Subject: RE: [office] Formula: test cases
- From: "Mason, James David \(MXM\)" <masonjd@y12.doe.gov>
- To: "odf list" <office@lists.oasis-open.org>
- Date: Fri, 30 Mar 2007 16:34:48 -0400
Comments from two perspectives:
1. As
a linguist (Ph.D., Germanic historical linguistics and comparative
literature)
In a
comment earlier today, someone remarked that English is ambiguous. All human
language is intrinsically ambiguous, not just English. (Take a look at a variety
of sources from the New Critics to Wittgenstein, or look at the archives of the
IEEE Standard Upper Ontology standards committee!)
The
point of formal mathematical notations is to eliminate the ambiguity of natural
language. James Clark has the traditional British university background,
complete with the training in philosophy and logic, so he can make himself write
things like the definition from XPath that Rob cited. But that definition isn't
normal discursive English; it's a translation into words of a formula, the sort
of thing that appeared in mathematical texts down through the Renaissance,
before modern notations were invented. We have notations now, and it shouldn't
be necessary to do everything just in words.
I
prefer to see standardization done in as formal as possible language, but as I
know from developing SGML, and Charles Goldfarb's subsequent efforts to
disambiguate himself in "The SGML Handbook", even the best intentioned efforts
don't always work. The notational parts of ISO 8879 are much better than the
text.
2. As
a standards developer (26 years in ANSI and ISO)
I've
never seen "test cases" in a published standard. As I commented in a reply
to Patrick yesterday, for SGML we standardized a methodology for reporting test
results but stayed entirely away from creating a suite of test
cases.
For
me, the problem with test cases is that they need to provide both necessary and
sufficient tests for compliance. Otherwise, they're just examples, not complete
tests.
See,
for example, 6.14.29, "FACT". There's obviously no way to provide an exhaustive
set of test cases, since the potential range of values for the argument to the
function is infinite. But why this set of test cases? Isn't the meaning of
factorial known from any statistics text? What do the cases add that isn't known
from standard definitions?
Or
take some of the more advanced statistical functions, such as 6.16.28,
"GAMMADIST". What is not said by the formulae included in the text? Again, why
these particular "test cases": are they special cases that must be defined (as
is the case of 0! = 1), are they interesting for some particular reasons, or are
they sufficient to test all the options for input to the
function?
To me,
the "test cases" appear to be examples, not unlike the examples in the ECMA
OOXML. (see, for example, their 15.5.6.115. "FACT",
where there are similar things listed as "examples". They appear to assume
traditional textbook definitions of functions and so don't give the formulae for
things like GAMMADIST in 15.5.6.131.)
I
don't necessarily object to examples, though very early in the
SC34 contacts with ECMA, I complained to a Microsoft representative and
later to their editor that their document would have been much easier to handle
(and far shorter than its 4000-plus pages) if the examples had been put off into
an annex.
But I
do have a problem with examples, even interesting/representative ones being
called "tests" and being part of normative text. If the examples are normative,
it means that someone can write code that implements those examples, and just
those examples, and claim compliance with the standard, even if the code gets
everything else wrong!
Given
the choice of treating the formula in the present 6.14.31, "GAMMA" as normative
and the "text cases" as normative, I'd go with the formula every time. The "test
cases" simply aren't sufficient to define the function or even adequately
exercise its implementation.
I
grant that there is a limit to what can be done in notation in the scope of a
standard. I watched with considerable amusement the old JTC1/SC18/WG3 in their
efforts to do a "Formal Specification of ODA". I goaded them into that because
ISO 8613 was originally defined only in discursive prose, and I held up the
example of SGML, defined as a production grammar. So they fell into logic
notation and tried to write an existence proof for ODA. It was all
wheel-spinning, and while they wasted some man-years on the effort they missed
the fact that their standard had totally lost out in the marketplace, never even
having been implemented.
Nonetheless, it's important that what is put in for normative contents in
a standard be what you want to be standardized, no more, no less. In some cases,
you can provide a piece of sample data that can exactly exercise a piece of code
(e.g., the 1024-character limit on cell contents in Excel), but in other cases
(such as statistical functions) you simply can't provide sufficient tests. (BTW,
hitting the 1024-character limit doesn't cause Excel to crash, just to do odd
things: I've had too much experience with engineers who write memos in Excel,
rather than Word.)
If
this document comes forward to ISO with the "test cases" normative, I think you
can expect lots of comments on them, including requests simply to remove
them.
In
commenting on the use of "test cases", I do not intend to criticize the overall
quality of the work here. It's much better than its ECMA counterpart. For one
thing, it's based on more comprehensive research, looking at a lot more products
than MS did for their specification. And it's absolutely necessary to have this
work in ODF: When MS first started to stir in ECMA, a certain MS XML specialist
(probably known to a number of you) called me up to find out how SC34 would
react to their submission. Along the way, he chose to criticize ODF, and one of
his main points was that ODF didn't have a formula language. So this
specification really needs to get adopted and sent on to ISO to take away his
argument.
Jim
Mason
SC 34
Liaison to OASIS
[Date Prev]
| [Thread Prev]
| [Thread Next]
| [Date Next]
--
[Date Index]
| [Thread Index]
| [List Home]