RE: [oic] Groups - svn-metadata-barth.zip uploaded

["Hanssens Bart" <Bart.Hanssens@fedict.be>]

Thanks for the comments Rob.


> We encode the test metadata in XML and use that to generate detailed
> HTML instructions.

Well, they start out as plain text (POD / Wiki - style), then get
converted to XML for future processing.
Calls for some discipline but works rather well using nothing but a text
editor.


> Did you have a tool to create the ODF documents?  Or were they
> created manually?

Notepad++: I manually create the .xmls in a directory, and then run a
script
to zip it the correct way (mimetype first, stored instead of deflated).

Once in a while, I'll check the result with the online ODF validator
(yes,
this validating part could be automated, but most of the time I'm
punching
in XML or reading the spec anyway...)


> In particular, what is the prescription level?  Is something a 
> requirement ('shall'), a recommendation ('should') or something else?

Unfortunately, we'll end up with the 'shoulds' most of the time
(the whole metadata fits in that category, for instance)

Basically, ODF says: you don't have to do anything, but if you do foo or
bar, this is how we want you to do it.


> And what is the source or authority of the provision?

The TC's interpretation of the spec ?


> However, we need to be  very careful to track the authority for every
test
> case that we create.  Otherwise we will have difficulty noting which
> subset of our test cases  (ones where the authority is the ODF
standard)
> must pass in a conformance test versus the broader set of test cases
that
> may pertain to an interoperability test.

So what we can do is, by default, leave it as "a test", and mark the
conformance tests. Afterwards a tool can filter out the conformance
tests.


> The OASIS Test Assertion Guidelines TC has come out with a 
> new CD that is worth reading over (only 37 pages long)

Interesting


> It may be possible to add this additional information to the XML you
> already have.  I'm not sure.  I think we're mainly missing the
explicit
> statement of prescription level and pre-requisite.  

Two-stage approach ?

1) write test cases like the one I've done on metadata
2) discuss them and add "this is a conformance" test when appropriate
and check if prerequisites are clear 


> So I would not be opposed to tracking test assertions on their own,
either
> as text or in a simple XML encoding.  We could then discuss and
approve
> the test assertions as a TC.  
> The translation of each test assertion into a specific test case
should be 
> straight forward and mechanical and may not elicit as much debate.

Mm, we need to approve both the assertion and the test case...

Personally, I'd rather write the test case first, isolate the assertion
and discuss them at the same time (at the risk of building a test case
on the wrong assertion, but it helps me to get a better overview of the
end result and gives me something to throw at real implementations)

Others may prefer to get the assertion approved first, and then write up
the test case.


Best regards,

Bart