Re: [dss-x] Test case generation from the core spec

[Juan Carlos Cruellas <cruellas@ac.upc.edu>]

Dear Andreas,

Thank you very much for your message and effort....see below intermixed.

Regards

Juan Carlos.
kuehne@trustable.de escribió:
> Hi all,
> 
> I like to give you a short status about my effort to derive conformance test cases from the core spec. The 'Test assertion guidelines' spec provides a good tutorial how find the assertions. The first step is to find all 'MUST' and 'MUST NOT', define the product ( in our case 'client' and 'server' ) and mark the whole condition.
> 
> In case of the core spec I started enthusiastically and all the MUSTs are marked quikly. But getting into details the problems grow : 
> 
> Looking at section 3.1 SignRequest I stuck at the RequestID :
> Even though the section is about the the SigningRequest, what's build by the client, the condition says something about the server ! Moreover this condition is a copy of the corresponding paragraph in 3.2 SignResponse.
> 
I guess that this is one of the effects of re-reading standards thinking 
in specifying how they will be tested in terms of interoperability, 
which is a way of testing how they should be implemented... I agree with 
you that the element that is being specified is an element generated by 
the client and that the text incorporates a requirement for the 
server....very likely at the moment of creating the standard, we thought 
that it would be good to justify the presence of such an element by 
mentioning the correlation with the RequestID to be generated by the 
server, but also very likely, a simple note or some not mandatory text 
would have been enough here, leaving the requirement for the 
SignResponse part....

In the light of this, my feeling is that the task of identifying the 
MUST, MAY, etc will be the first one, which will have to be followed by 
a kind of filtering for identifying things like the ones that you have 
mentioned....a third task could be to take notice of all these sources 
of potential missunderstandings, ambiguities or bad placement of 
requirements texts and incorporate them in the proces of the core 
review...For instance, what about to include this comment by you in the 
process of core review? If so  we could even take notice of a proposal 
for resolution: add an explanatory note referencing the SignResponse 
element instead of the requirement in the SignRequest.

> Next problem in SignRequest is the section about InputDocuments. The heading denotes this element as 'Optional'  but the text says that it is required in a SignRequest.
> 
Actually the InputDocuments is optional in the RequestBaseType and 
SignRequest inherits it... I think that there was some rationale for 
making the InputDocument optional in general while mandatory in the 
precise SignRequest...
I do not see any problem in keeping the xml schema as it is...te
> Generally the term 'REQUIRED' is a bit vague what system is responsible for checking and what should happen in error cases.
RFC 2119 is clear on the semantics of REQUIRED. I quote it:
1. MUST   This word, or the terms "REQUIRED" or "SHALL", mean that the
    definition is an absolute requirement of the specification.

The text in the core reads:
<InputDocuments> [Optional]

[.....]This element, while optional in RequestBaseType, is REQUIRED for 
the <SignRequest> element.

The way I read is that <InputDocuments> is Mandatory for the SignRequest 
and in consequence is the obligation of the client to incorporate it to 
the SignRequest (as these requests are built by clients, not by 
servers). Finally, I agree with you that there is not here any 
indication of what should happen if this element is not included and it 
arrives without this element to the server....but in section 2.6, we 
read that one of the values for <ResultMajor> is:

urn:oasis:names:tc:dss:1.0:resultmajor:RequesterError
The request could not be satisfied due to an error on the part of the 
requester.

I would say that this is precisely what should happen in this case....
should we also add a note on the issue that you have mentioned?

> 
> Another general point is the definition of a condition for the client. But there is no definition what should happen when the client sends a non-conformant request. In my ideal version there should a set of conditions for a conformant client and a conformant server.
> 
Maybe I am wrong but I assumed that the server must reply with a 
<ResultMajor>
urn:oasis:names:tc:dss:1.0:resultmajor:RequesterError
The request could not be satisfied due to an error on the part of the 
requester.

> So generally there is much more work to be done than just extracting the conditions semi-automatically.  In some cases this may even affect the core. A workaround to preserve the core could am additional version of the where the original text remains unchanged and the conformance statements are mixed into it marked as conformance requirements.
I have some difficulties in understanding the last sentence...let us 
talk this afternooon

> 
> So many work to, opinions welcome.
> 
> Greetings
> 
> Andreas
> 
> ---------------------------------------------------------------------
> To unsubscribe from this mail list, you must leave the OASIS TC that
> generates this mail.  Follow this link to all your TCs in OASIS at:
> https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php 
>