Re: submitting test documents, simplified

[Ming Fei Jia <jiamingf@cn.ibm.com>]

Hi Bart,

The ideal way of submitting test cases may be through filling a web form, but it seems the facility is unavailable as you mentioned before. The current proposed method also sounds a good alternative. Firstly, just want to confirm the presenting of test metadata will be a html document that is generated from the XML, which in turn is generated from the template in the contributor's mail, right? Second, a question: for a test case, we hope its test metadata(html doc) and test document are put together on the document repository.OASIS document repository seems not be able to do that. How about put them together on SVN? For example, the SVN structure is a tree structure which is mapping to ODF specification sections. The leaf node in SVN is a folder that is corresponding to the lowest level section in ODF specification. And under this folder, we put both the test metadata html file and the test document there. Also I saw Dennis is building a SpecAnalysis tree in SVN, maybe it is better if the test suite tree can have a direct relationship with the SpecAnalysis tree. But I do not know how to do that right now? Add some link somewhere? Or combine the 2 things together? etc...

"Hanssens Bart" <Bart.Hanssens@fedict.be> wrote on 03/11/2009 11:53:02 PM:

> From:
> 
> Subject:
> 
> submitting test documents, simplified
> 
> Rob / Mingfei,
> 
> 
> Thinking through the whole metadata proposal/testscenario once again, it
> seems that creating the "doctest" XML by hand isn't the most user
> friendly way.
> 
> So the suggestion would be to use a simple mail template (like the ODF
> TC uses for gathering the requirements for ODF Next), and create an XML
> out of it, like this:
> 
> 
> 
> * Step 1) Member can non-members can create small or large test
> documents (either using their favorite implementation or hand crafted).
> The most important thing is to make sure that it is clearly specified
> what exactly is tested, and how to do it with the submitted test
> document.
> 
> 
> * Step 2) In addition, they must fill out the following template (can be
> put on the Wiki and/or xml.org and/or OIC TC document repository)
> 
> 
> + CONTACT (mandatory)
> Your name (Your@email.com)
> 
> + APPLICATION (pick one or more, remove what doesn't apply)
> Text / spreadsheet / chart / presentation / drawing / form / template
> 
> + CATEGORY (pick one or more: remove what doesn't apply)
> Data / visualization / behavior
> 
> + DESCRIPTION (mandatory)
> A short, general description of the test scenario
> Multiple lines are allowed
> 
> + STEPS (mandatory)
> - This is the first step, like "open the test document"
> The step can span multiple lines, just start each new step with a "-"
> - The second step, like "go to page 3"
> You can add as many steps you want
> 
> + EXPECTATION (mandatory)
> - What is the expected result ? Like "page 3 must have a blue
> background"
> - Better even: results, there can be more than one 
> And they can span multiple lines of course
> 
> + SPECS (optional, for specialists)
> - ODF 1.1 / 1.1 / Section title goes here
> Optionally, quote the specific paragraph(s) that contains the aspect to
> be tested. Once again, this can span multiple lines  
> - ODF 1.1 / a.bc / Second section title
> You may test multiple parts of the specification
> - ODF 1.2 / x.yz /
> Probably the test can be used for different ODF versions
> 
> 
> 
> (Maybe EXPECTATION and STEPS can be mixed together, reducing the
> template even more)
> 
> 
> 
> * Step 3) Send the filled out template with the test document(s) in
> attachment to the OIC comment list. The subject must start with
> something like "[TestDocument]". Then the mail archive can be processed
> in a (semi-)automatic way.
> 
> 
> 
> I do have some thoughts on the next steps and how to process it as well,
> but perhaps it's easier to first discuss the first three steps listed
> above, and see if that's the way to go (or not)
> 
> 
> Best regards,
> 
> Bart