Hello Andreas,
Please correct me if I misunderstood you, you mentioned in our today's call that having (formula) tests at the specification would not
be of a benefit to the ODF user, as a test case usually is just testing a specific dataset (a subset of all possible values), while the specification defines a formula with far more coverage than ever being able to test in full.
But the same argument would apply to examples, which are so helpful giving readers the quick glimpse of the idea by providing a certain data constellation.Â
ForÂexample, the visualizations of charts were very helpful to me in ODF 1.0.
And I don't deny that passing a test will not guarantee that an ODF application is implementing the tested formula correctly, but failing should indicate that there is a problem - this often helps already a lot. From my experience, these smoke tests would be precious for developers.
With our latest established GitHub repository, it was never as easy to add automated functionality like generating an ODS test document including some formula tests.
Shouldn't we take a look at this old test suite generator and see what we can reuse or if tests are too complex to touch at all?
No need to provide full formula coverage by the TC, but we might start to reuse, refactor and provide a test framework to allow others - even external - to contribute.
I would like to state in this issue - before we closing it - that we are interested in tests and evaluating the former formula tests for ODF 1.4 in the context of our new GitHub repository.Â
Best regards,
Svante
PS: Sure we might want to move the tests - containing a lot of test data - outside the specification in additional files, but these are implementation details for later.