Re: [oiic-formation-discuss] Draft Interoperability and Conformance TC formation proposal (0.2)

["Peter Dolding" <oiaohm@gmail.com>]

On Sat, Jul 26, 2008 at 3:56 PM, Shawn <sgrover@open2space.com> wrote:
> Peter Dolding wrote:
>>
>> Production of software not needed for testing also not fine.
>
> Peter, this is the sticking point.  This TC will NOT be doing any tests.
> Only providing the DEFINITION of what tests can be done to check conformity
> and interoperability.  The intent is that these definitions would then be
> used by third party testing shops, or developers.
>
> I see the TC as defining the requirements (in terms of the traditional
> software life cycle), then handing the resulting documents to developers (by
> making them available) who will then write code to meet the requirements.
>
> Of course, this is only my take on the conversations that have taken place.
>  Perhaps I've missed something and am wrong.
>
You have missed something.  Lets say you are building a report on ODF
compadiblity.  You find a glitch in X Y and Z.

Now how are you going to confirm this.   Particularly if the
implementers say something must have been wrong with your testing
machines or testing software.  Being able to issue a test suite
backing up a report when disputed is critical.  Even running acid
testing to back report to confirm broad existence of fault.

Ok we hope implementers will not be pig headed and dispute correct
reports.    By being able to issue the test suite used to create the
report implementers also get to pull up defect and say hey that
section of report on us is wrong and we can prove it.

Its like a key bit of the compatibility report to get it past
question.   Without it report is only a bit of paper a person can say
is wrong that is not simply provable.   It also make the report a
black box unable to be correctly checked if the information collected
is right.

Its part of the process we cannot be without.  Idea of thrid party
testing shops only get you so far in a dispute.  You will be sure one
implementer will accuse some of the testing shops of rigging the
tests.

Clean cycle.   TC approves and stores the tests.   Approved tests get
used by report writers even sent to test shops for running tests.
Dispute handling stored tests get dug out sent to all parties to find
defect in the tests or confirm report is right or confirm error in
report this way report can be validated.  Also a option of these tests
to be send out for broader acid testing if reports of errors from
standard that don't match with testing shops data.

The Clean cycle does not give room for a dispute getting out of
control.   Now not having the clean cycle leaves a lot of options for
disputes over that the report writer was bias,  the tests had to be
wrong,  broad enough testing must not have been done and so on.  As
part of the clean cycle report writer may need to create tests in the
form of test kits to send out.

Other important thing not everyone is going to be in the TC.  So
allowing people to run the tests that created the data in the report
is a good thing.

Peter Dolding