OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

office message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]


Subject: Re: [office] Formula: test cases


Mason, James David:
> Test cases can be quite useful, but it is difficult to demonstrate that
> a test suite exhaustively exercises all the conforming variations
> supported by a standard or even exercises all the interesting/useful
> options.

The test cases ensure that the specification is interpreted unambiguously, specify special cases, and in a few cases ensure that common errors (of implementation or interpretation) can be avoided.  Think back to any book you've read describing something technical - didn't they include examples, to ensure that you got the correct understanding?

The test cases are NOT a full conformance test suite, my apologies if my statements were interpreted otherwise. The purpose of these test cases is NOT to exhaustively exercise all conforming variations, or even all the interesting/useful options.  I agree, that IS a difficult thing to do.  For example, we do NOT define conformance as simply "conforming to the embedded test suite". Passing these test cases isn't enough for that.  But they're enough to ensure that many latent, unidentified ambiguities are suddenly NOT ambiguous.

Stupid example: Let's say that we defined SIN, etc., and nowhere noted if it took degrees or radians. (We DO say it's in radians, by the way.).  It's often hard to spot errors like that.  Add a few test cases, and suddenly, it's impossible to make a non-interoperable solution... because you cannot meet the text and test cases without making the correct determination (radians).


> The validtation of a test suite opens up all sorts of new
> issues that may not even be relevant to the original standard.

In the general case, true.  The restricted domain of formulas eliminates most of those concerns, though.  This is a very narrow domain, and what wouldn't work elsewhere actually makes sense in this domain.

My experience is that many validation test suite efforts take a lot of time trying to figure what the spec actually MEANT.  In the ideal case, that doesn't happen, but in the real world it does.  Including a few test cases in the spec itself can eliminate much of that ambiguity.

Anyway, I'm fully aware that specifications don't "normally" include test cases.  But this is merely a statement of current practice, and says nothing about what SHOULD be there. I'm arguing that in this narrow domain there is a better way. My hope is that once you examine it in practice, you'll agree.

--- David A. Wheeler


[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]