[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: RE: [tag-comment] Test Assertions for Normative Statements in theebBP v2.0.4 spec
>… if we can base the subject and predicates on a meta-model of a standard.
Indeed – the difficulty is to get agreement on that meta-model though... But I agree that more accurate expressions (yet still English text) of a specification requirements based on recognized ontologies would help TAs writing. (but would you expect this ontology to be common/reused by all standards? Or specific to a standard? A category of standards?)
Where I see – from a testing viewpoint – how ontologies could help, is when it comes to “translate” spec normative statements into something testable / observable. Some ontology about test artifacts, test material (e.g. the concept of “log” or “trace”, “event”, “API”, “specification implementation”, SUT (system under test) with blackbox behaviour and particular states) would also help. For example, a test assertion about a “business transaction” could more precisely be expressed in terms of property of the “trace” left by this BT .
NOTE: The concept of Conformance Clause recently added to OASIS specs, will now describe what possible “implementations” are, for a spec.
Also I suspect your efforts might find more “Return on Investment” in standards / specifications related to specific Industry sectors (i.e. crafted by actual end-users for a specific business domain) because they already know the high cost of non-aligned concepts across business sectors or even within same sector... (while technologists doing infrastructure standards are mostly engineers and they think in terms of implementations, technologies they already know about and have a common understanding about – they may not be the right folks to be “early adopters” of ontologies to their specs, because engineers can read between the lines of an approximate wording... just my opinion. But what you do for ebXML/ebBP at least is a demonstration of feasibility.)
> David Snelling might have already shown it to you. Otherwise, please let me know and I will send it to you. I would be most interested to know your opinion on the methodology.
Because it is related somehow to testing methodologies, feel free to send it to this comment list – or if you prefer not to put it in the spotlights for to me. I’ll have a bit of time in April.
Thank you very much for your comments. It is always very good to have your opinion.
The TAB works seems very interesting to me and the categories seem quite similar to mine. My interest in the categories in particular is in how to treat them when defining TAs in this case and in my work when defining the normative statements in the ontological manner, which to me seems quite similar to defining TAs. Not all the categories would be possible to define in the ontological manner I believe.
Regarding the TAs, now that you defined them I have a better understanding of them. However, I think it would be a good step forward if we can base the subject and predicates on a meta-model of a standard. This, in my opinion, would be another step towards a test case.
As you probably know, I am working on a methodology for ontology-based standards development. David Snelling might have already shown it to you. Otherwise, please let me know and I will send it to you. I would be most interested to know your opinion on the methodology.
Thanks for your interest.
Dennis has addressed some of your points (thanks!) - let me take the time to add a few things: see inline <JD> of your *two* emails below:
Hope that helps…
Thank you very much for your response.
I believe there should possibly be a guideline for defining normative statements themselves, if already doesn’t exist (I am not aware of such thing).
<JD> Hard to get very detailed on this - what you are asking for is a guide to write specifications, no less (beyond our scope).... For now we only define: " [normative statement] A statement made in the body of a specification that defines prescriptive requirements on a conformance target."</JD>
If it does exist, I guess there are rooms for significant changes / improvements. Furthermore, I am thinking of categorising the normative statements, so that we may employ different scales and measurements for them. By category, I mean something different to their prescription level. Now I can think of some categories:
- Static: which are mainly about well-formedness rules and relationships between classes / entities / concepts.
- Dynamic: which are mainly concerned with state management and transitions. This is a bit tricky! Especially in ontology world that I am working in.
- Run-time: the ones which should be handled by the implementation, BSI in case of ebXML.
<JD>You may be close to a similar classification we drafted in OASIS TAB (of which I am a member): see Data artifacts/State machines/Processors under section 3 of this wiki: http://wiki.oasis-open.org/tab/TestingPolicy
Although our immediate concern is conformance assessment, these could extend to helping write normative statements. Feel free to reuse/refine. </JD>
If we can distinguish the types of the normative statements, it might be easier to think of ways of measuring the conformance of an implementation or an XML document to a specific spec.
What do you think?
<JD>Yes but that is part of a more ambitious guide I think, beyond testing concerns.</JD>
From: Bahareh Heravi [mailto:Bahareh.Heravi@brunel.ac.uk]
Sent: Tuesday, March 01, 2011 07:16
Cc: 'firstname.lastname@example.org'; 'JDurand@us.fujitsu.com'
Subject: [tag-comment] Test Assertions for Normative Statements in the ebBP v2.0.4 spec
I am trying to define some Test Assertions for the ebBP v2.0.4 spec. However, in some cases it doesn’t seem possible for me to do so. I have found quite a lot of Normative Statements, which I couldn’t define a TA for. A number of such normative statements are as follows:
<JD> as you mention at the end, to be useful the TA should bring us one step closer to the actual testing. In some cases unfortunately it is not obvious how to do that. Yet, just identifying clearly the Target, and stating clearly the abstract true/false property to verify in the predicate, will help the next step – whether automated or human-performed – more than you’d think. SO let me attempt to answer yours below (some TAs may appear to provide little benefit, but still are a step forward in helping whoever will verify later the normative statement).
A document referenced by an include element MUST be inserted before schema or DTD validation is attempted.
<JD>The condition “before schema or DTD validation is attempted “ may be handled either as a prerequisite, or as predicate:
- target=document, predicate=(document has all includes resolved/inserted), prerequisite=(document is candidate and ready for validation)
(option 2) (in case we don’t know how to verify “ready for validation”, we rely on predicate to express the whole scenario that MUST NOT happen for ANY document)
- target=document, predicate=(NOT(document has not includes resolved/inserted AND document validation has been attempted)) , which is same as:
- target=document, predicate=(document has includes resolved/inserted OR document validation has not been attempted)</JD>
If specified, the content of the receipt and the legibility of a business message (if required) MUST be reviewed prior to the processing of the Requesting or Responding Business Document or the evaluation of condition expressions in the message's Business Documents or Document Envelope.
<JD>Here only for “receipt”:
- target=(receipt message), predicate=(content of receipt is reviewed AND legibility of message is reviewed) preprequisite=(receipt is candidate/ready for processing)
- same comments as for p.29 (could have folded prereq in predicate)
However, to achieve this result (state alignment), the Business Transaction protocol MUST be implemented on top of a reliable messaging service that provides guaranteed message delivery at the transport level.
<JD> That is a strange normative statement rather high level (general “state alignment” property predicated on an implementation architecture statement). I think the spec meant simply “must”, not MUST.
When multiple activities are nested within ComplexBTA, these activities MUST be executed in series.
<JD> if we have an idea of how to observe activity execution e.g. via a trace:
Target= ComplexBTA element, Prerequisite=(target is executed), Predicate=(exec trace for target shows same order for activity execution as activity elements in target)
NOTE: I have made here an “interpretation” of the original normative statement. This interpretation (assuming execution tracing) should be mentioned/assumed in the “normativeSource” element of the TA, i.e. in addition to the pointer to original statement.
Whether BeginsWhen, EndsWhen, or Pre- or PostCondition, the information MUST be visible to the parties involved.
<JD>Also not sure what is meant by “visible” (i.e. how to interpret it). The target= these elements, Predicate=(a more concrete interpretation of “visible”)
The Business Transaction is an atomic unit of work. All of the interactions in a Business Transaction MUST succeed or each party MUST revert their state to the state prior to the start of the BTA.
<JD> In the absence of a more concrete “interpretation” available for such statement, the TA will remain abstract: target=BT, prerequisite=(BT is executed), predicate=(BT outcome = success OR state(party prior execution) = state(party after execution))
So here at least we identify the measurable entities: “party state”, BT outcome. Sounds modest, but is a step forward.
A ReceiptAcknowledgement (if required) MUST always occur before an AcceptanceAcknowledgement (if required), and an AcceptanceAcknowledgement MUST always occur before a Response (if required).
<JD>target=BTdefinition, predicate=(no(RA) or no(AA) or RA<AA) and (AA<R or no(R) ) )
A timeout parameter MUST be specified whenever a Requesting or Responding party expects Business Signals in return to the Business Document Request or Response. A Requesting party MUST NOT remain in an infinite wait state.
<JD> (only for the 1st statement here)
target=BTdefinition, predicate=(no(BusSignal) or timeout has value)
A Receipt Exception signals an error condition in the management of a Business Transaction. This Business Signal is returned to the initiating activity that originated the request. This exception MUST terminate the Business Transaction.
<JD> target=BTexecution, prepreq=( Receipt Exception occurred ), predicate=( Receipt Exception = last event in the BTexecution)
I can imagine of defining some TAs which are not that close to the test case and are not based on any meta-model, which I don’t think would be of much use. An example would be something like the following:
TA id: ebBP-P47
Normative Source: specification requirement zzz
Target: business message
Predicate: content of the receipt and the legibility (perhaps to be defined in another TA) [of a Business Message] to be reviewed prior to the processing of the Requesting or Responding Business Document or the evaluation of condition expressions in the message's Business Documents or Document Envelope.
Prescription Level: mandatory
This doesn’t seem much useful to me and is not that much more formal than the text in the spec. I understand that I have probably not defined it exactly the way I should have, but still I don’t think this normative statement could be formalised in an appropriate way and based on a defined meta-model.
I would appreciate if you could let me know the way you would define TAs for the above normative statements. Or perhaps TAs could be only used for certain types of normative statements?
Many thanks and Best regards
Bahareh R. Heravi
Department of Information Systems & Computing
Uxbridge, Middlesex, London, UB8 3PH, United Kingdom.
Web: http://people.brunel.ac.uk/~cspgbrh <http://people.brunel.ac.uk/~cspgbrh>
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]