Re: [oiic-formation-discuss] My perspective

[Thomas Zander <zander@kde.org>]

On Monday 30. June 2008 14:56:13 robert_weir@us.ibm.com wrote:
> I suggest
> these
>
> > points to be something along the lines of
> >   a) Loading the test data and displaying it on screen (correctly ;)
> >   b) Saving the loaded document out again and not loosing information.
> >   c) Having GUI to alter the ODF feature to all the supported values.
>
> So you are starting with a document that is already correct (well-formed,
> valid, etc.) and checking how an ODF application treats it.
>
> We've been describing such tests as falling into one or two buckets:
>
> 1) Conformance tests would be tests that are traceable to formal
> provisions of the ODF standard.[]

> 2)Interoperability tests would contain further tests which might not be
> formally traceable to a shall or a should, but are stated as definitions
> in the text, or are clearly implied.[]  

I think we are talking at slightly different levels of the testing-stack. We 
probably agree, however ;)
You seem to be talking about defining which odf-features from the spec to test 
which is something I didn't look at at all.
My point you quoted above is that *each* of those tests should be run 3 times 
and have the result of A, B and C be recorded separately.

> But on the other hand, in some cases you want data loss.  For example,
> there is a utility that scans ODF presentations and searches for embedded
> bitmap images, and if they are at a a very high resolution reduces down
> samples them to something more reasonable for screen presentations, this
> reducing the size of the ODP file.

Right, the point of my 3 different tests is to make it clear that this 
application does something specific because you will see that it scores very 
different on point A than on point B, for specific features like images.
Or, a more clear example, an odf-viewer example will have a zero percent score 
on point B.

> > Does
> > implementation X support lists or tables very well.  If not, I won't check
> > it 
> > out.  If I need to write my formula, I go for the app that supports 80%
> > of the spec instead of the one with 20% support. Etc.
>
> I think of a profile (or at least one kind of a profile) as a
> specification that records a set of "feature-groups" that together solve a
> recurring problem.

Any sort of grouping to make this more useful for the end user sounds sane, I 
was more assuming market values would sort this out for us.  A magazine for a 
specific set of users may use our testresults to recommend something to their 
readers.  I'm not sure that defining profiles is actually solving anything. I 
would not go further then to conclude it makes the problem more manageable 
for people. That naturally is a good thing, but I think we have to attract a 
different set of people to OASIS to actually come up with proper profiles.
You and me (as to backers of our implementation) are probably the worst 
possible people to come up with good profiles. Actual users in actual 
usecases probably are much better at that.

> Thanks for your thoughts.
Thanks for yours!
-- 
Thomas Zander

This is a digitally signed message part.