Re: ODF 1.2 Formulas Test Suite Generator

[Andreas J Guelzow <andreas.guelzow@concordia.ab.ca>]

Hi,

On 2020-08-10 1:10 p.m., Svante Schubert wrote:

Hello Andreas,

Please correct me if I misunderstood you, you mentioned in our today's call that having (formula) tests at the specification would not be of a benefit to the ODF user, as a test case usually is just testing a specific dataset (a subset of all possible values), while the specification defines a formula with far more coverage than ever being able to test in full.

    What I meant is that for formula test cases to be useful for
    implementation purposes it has to include a wide range of cases
    including many edge cases. Moreover, they would have to be very
    carefully crafted to address all edge cases to ensure that an
    implementation that passes these test cases is in fact likely to be
    correct in general.

But the same argument would apply to examples, which are so helpful giving readers the quick glimpse of the idea by providing a certain data constellation.

    I think that is s completely different issue. These examples are
    never intended to provide a full range of possibilities.

ForÂexample, the visualizations of charts were very helpful to me in ODF 1.0.

And I don't deny that passing a test will not guarantee that an ODF application is implementing the tested formula correctly, but failing should indicate that there is a problem - this often helps already a lot. From my experience, these smoke tests would be precious for developers.

With our latest established GitHub repository, it was never as easy to add automated functionality like generating an ODS test document including some formula tests.

    Adding formula tests would not be easy: interesting tests would
    primarily be edge cases which would require extensive review.

Shouldn't we take a look at this old test suite generator and see what we can reuse or if tests are too complex to touch at all?

No need to provide full formula coverage by the TC, but we might start to reuse, refactor and provide a test framework to allow others - even external - to contribute.

Any test cases would still need to be vetted by the TC to be sure they are in fact correct. A collection of tests, some correct and some incorrect under the banner of the TC would be much worse than having none at all.

Andreas

No time pressure on my side, it was just because of today's issue:Âhttps://issues.oasis-open.org/browse/OFFICE-3689

I would like to state in this issue - before we closing it - that we are interested in tests and evaluating the former formula tests for ODF 1.4 in the context of our new GitHub repository.Â

Likely even creating and referencing a new enhancement ODF 1.4 task of evaluating the former "pre-ODF 1.2 formula test suite generator".Â

Best regards,

Svante

PS: Sure we might want to move the tests - containing a lot of test data - outside the specification in additional files, but these are implementation details for later.

Am Do., 16. Juli 2020 um 20:14ÂUhr schrieb Svante Schubert <svante.schubert@gmail.com>:

Dear all,

Rob (Weir) (on CC) was so kind as to provide me today with Daniel Carrera's test suite generator script (Daniel on CC as well).Â

I have added the script to our repository:Âhttps://github.com/oasis-tcs/odf-tc/tree/master/src/test/resources/odf1.2/tools/formulas-testsuite-generatorÂÂ

Best regards,

Svante

PS: May I quote Rob: "The ODF TC is still meeting? That brings a smileÂto my face." ;-)

--

Andreas J. Guelzow, PhD FTICA

Registrar & Director of Enrolment Services

Professor, Mathematical & Computing Sciences

Direct: +1 780 479 9290
Toll-Free: +1 866 479 5200

concordia.ab.ca