[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [Elist Home]
Subject: RE: [ebxml-iic-conform] MS Level 2 & 3 test reqs
Jacques and all, Here are the latest versions of levels 1,2 and 3 ebXML MS Conformance Testing Requirements. Incorporated are changes ( and a few not ) based upon comments. Further modifications based upon discussion is possible ( but we seem to be iterating to a conclusion :) I am also attaching comments to Monica's and Jacques comments ( already sent out comments for Michael Wang's post ). All my comments begin with [MIKE3] Regards, MikeTitle: OASIS ebxml-iic TC -> Requirements
ID | Name | Test Coverage | Specification Ref | Precondition | Requirement Level | Assertion |
---|---|---|---|---|---|---|
r0.1 | Global Requirements for All Tests | ebMS-2#1.3 | ||||
r0.1.1 | SchemaValidation | full | ebMS-2#1.3 | ( For each generated message ) | REQUIRED | Supports all mandatory syntax defined in Core plus Additional Features |
r1.1 | PackagingSpecification | ebMS-2#2 | ||||
r1.1.2.1 | GenerateConformantSOAPWithAttachMIMEHeaders | partial | ebMS-2#2.1.2 | ( For each generated mesage, if it is multipart MIME or not text/xml ) | REQUIRED | The primary SOAP message is carried in the root body part of the message. |
r1.1.2.2 | GenerateConformantSOAPWithAttachMIMEHeaders | partial | ebMS-2#2.1.2 | ( For each generated mesage, if it is multipart MIME or not text/xml ) | REQUIRED | The type parameter of the Multipart/Related media header is "text/xml", the MIME parts must contain a CID MIME header or a Content-Location MIME header structured in accordanced with RFC 2557 |
r1.1.2.3 | GenerateConformantSOAPWithAttachMIMEHeaders | partial | ebMS-2#2.1.2 | ( For each generated mesage, if it is multipart MIME or not text/xml ) | STRONGLY RECOMMENDED | The "start" parameter is present. |
r1.1.3 | GenerateCorrectMessagePackageContent-Type | full | ebMS-2#2.1.2 | ( For each generated message ) | REQUIRED | The Content-Type MIME header in the Message Package contains a type attribute of "text/xml". |
r1.1.4 | GenerateContent-IDStartValues | full | ebMS-2#2.1.2 | ( For each generated message ) | RECOMMENDED | The Content-ID MIME header in any generated Message Package contains a start attribute identifying the first MIME part. |
r1.1.5.1 | ProcessNon-MultipartMessages | full | ebMS-2#2.1.2 | ( the message is not multipart MIME AND the message has no payload ) | REQUIRED | The MSH should accept the message. |
r1.1.5.2 | GenerateNon-MultipartMessages | full | ebMS-2#2.1.2 | ( For each generated message, if the message is not multipart MIME ) | REQUIRED | It must not have a payload. |
r1.1.6 | GenerateCorrectSOAPMessageContentType | full | ebMS-2#2.1.3.1 | ( For each generated message ) | REQUIRED | The MIME Content-Type header for each generated SOAP Message has the value "text/xml". |
r1.1.7 | GenerateSpecificSOAPMessageCharacterSet | partial | ebMS-2#2.1.3.1 | ( For each generated message ) | REQUIRED | The MIME Content-Type header of each generated SOAP Message specifies the character set used to generate the message. |
r1.1.8 | GenerateDefaultSOAPMessageCharacterSet | none | ebMS-2#2.1.3.2 | ( For each generated message ) | RECOMMENDED | The UTF-8 character set is used by default when encoding each SOAP Message. |
r1.1.9 | ProvideEmptyManifestAndPayloadIntegrity | full | ebMS-2#2.1.4 | ( For each generated message, if there are no application payloads identified in the message header manifest” ) | REQUIRED | There must not be any payload MIME parts |
r1.1.10 | ProvideManifestAndPayloadIntegrity | full | ebMS-2#2.1.4 | ( For each generated message ) | REQUIRED | The contents of each payload MIME part are identified in the Manifest element within any generated SOAP body |
r1.1.11 | ProcessUnrecognizedMIMEHeaders | partial | ebMS-2#2.1.5 | ( For each received message containing unrecognized MIME headers ) | REQUIRED | Unrecognized MIME headers in a MIME part are ignored and no Error message is returned. |
r1.1.12 | ProvidePayloadContainerIntegrity | full | ebMS-2#2.1.4 | ( For each generated message, if there is no application payload within the Message Package” ) | REQUIRED | There must not be any payload containers. |
r1.1.13 | GenerateXMLVersionInProlog | full | ebMS-2#2.2.1 | ( For each generated message, if the XML Prolog exists in the SOAP message ) | REQUIRED | XML version is declared |
r1.1.14 | ProvideXMLPrologEncodingAndMIMEContent-TypeIntegrity | full | ebMS-2#2.2.2 | ( For each generated message, if the XML Prolog exists in SOAP message AND Encoding is declared in the XML Prolog AND charset attribute is present in SOAP header container ) | REQUIRED | The encoding declaration matches the charset attribute of the Content-Type MIME header in the Header Container. |
r1.1.15 | GenerateCorrectExtensionElementNamespace | full | ebMS-2#2.3 | ( For each generated message ) | REQUIRED | All ebXML extension elements included within generated SOAP Envelope, Header and Body elements are namespace qualified to: "http://www.oasis-open.org/committees/ebxml-msg/schema/msg-header-2_0.xsd" |
r1.1.16 | GenerateCorrectEnvelopeSchemaLocation | full | ebMS-2#2.3.2 | ( For each generated message ) | RECOMMENDED | SOAP Envelope elements include the XMLSchema-instance namespace qualified schemaLocation attribute indicating the extended ebXML envelope schema: "http://www.oasis-open.org/committees/ebxml-msg/schema/envelope.xsd" |
r1.1.17 | GenerateCorrectSOAPHeaderAndBodySchemaLocation | full | ebMS-2#2.3.2 | ( For each generated message ) | RECOMMENDED | SOAP Header and Body attributes both include a XMLSchema-instance namespace qualified schemaLocation attribute indicating the extended ebXML Msg Header schema "http://www.oasis-open.org/committees/ebxml-msg/schema/msg-header-2_0.xsd" |
r1.1.19 | GenerateCorrectSOAPBodyNamespace | full | ebMS-2#2.3.4 | ( For each generated message ) | REQUIRED | A SOAP Body element is namespace qualified as per the SOAP namespace declaration in the SOAP Envelope element. |
r1.1.20 | GenerateOneMessageHeaderInSOAPHeader | full | ebMS-2#2.3.5.1 | ( For each generated message ) | REQUIRED | A SOAP Header element always contains one ebXML MessageHeader element. |
r1.1.21 | GenerateCorrectForeignElementNamespaces | full | ebMS-2#2.3.6 | ( For each generated message ) | REQUIRED | Any foreign namespace qualified elements present within generated ebXML extension elements are namespace qualified with a namespace that is not "http://www.oasis-open.org/committees/ebxml-msg/schema/msg-header-2_0.xsd". |
r1.1.22 | GenerateIdAttributeToExtensionElements | full | ebMS-2#2.3.7 | ( For each generated message ) | OPTIONAL | An XML ID attribute is supplied for each generated ebXML element (to assist with such tasks as specifying elements included in a digital signature). |
r1.1.23 | GenerateCorrectMessageHeaderVersion | full | ebXML-2#2.3.8 | ( For each generated message ) | RECOMMENDED | An ebXML MessageHeader element always contains a version attribute with a value of "2.0" |
r1.1.24 | GenerateCorrectSOAPMustUnderstandNamespace | full | ebMS-2#2.3.9 | ( For each generated message ) | REQUIRED | All ebXML extensions of the SOAP Header element (MessageHeader, SyncReply, MessageOrder, ...) contain the mustUnderstand attribute namespace qualified to the SOAP namespace (http://schemas.xmlsoap.org/soap/envelope). |
r1.1.25 | ProcessMustUnderstand | partial | ebMS-2#2.3.9 | ( For each received message containing a SOAP Header extension with a mustUnderstand attribute set to "1" and not understood by the MSH. ) | REQUIRED | The message is rejected in accordance with SOAP. |
r1.2 | CoreExtensionElements | ebMS-2#3.1.1 | ||||
r1.2.3 | GenerateUniquePartyId | full | ebMS-2#3.1.1 | ( For each generated message ) | REQUIRED | The value of any given type attribute must be unique within the list of PartyId elements contained within either the From or To element. |
r1.2.5 | ReportInconsistentPartyIdContent | full | ebMS-2#3.1.1.1 | ( For each received message, PartyId does not contain a type attribute AND PartyId text node is not a URI ) | STRONGLY RECOMMENDED | MSH responds with an error (Inconsistent/Error) |
r1.2.5.1 | GenerateValidPartyIdContent | full | ebMS-2#3.1.1.1 | ( For each generated message, if generated PartyId contains a type attribute ) | RECOMMENDED | Its value is a URI |
r1.2.5.2 | GenerateValidPartyIdContent | full | ebMS-2#3.1.1.1 | ( For each generated message, if generated PartyId does not contain a type attribute ) | REQUIRED | Text content of the PartyId element must be a URI |
r1.2.6 | ReportFailedCPAIDResolution | full | ebMS-2#3.1.2 | ( For each received message, if value of the CPAId element on an inbound message cannot be resolved ) | REQUIRED | The MSH responds with an error (ValueNotRecognized/Error). |
r1.2.9.1 | ProvideConversationIdIntegrity | partial | ebMS-2#3.1.3 | ( For each generated message within the context of the specified CPAId ) | REQUIRED | The generated ConversationId will be present in all messages pertaining to the given conversation. |
r1.2.9.2 | ReportConversationIdIntegrity | partial | ebMS-2#3.1.3 | ( For each received message, if a message is received that is outside of the context of the specified CPAId ) | RECOMMENDED | An Error message is generated. |
r1.2.10 | ReportInconsistentServiceElementContent | full | ebMS-2#3.1.4.1 | ( For each received message, if the received "type" attribute is not set. AND If the Service element content is not a URI. ) | REQUIRED | MSH Responds with an error (Inconsistent/Error) |
r1.2.10.1 | GenerateConsistentServiceElementContent | full | ebMS-2#3.1.4.1 | ( For each generated message, if the generated Service element "type" attribute is not set. ) | REQUIRED | Generated Service element content must be a URI |
r1.2.11 | ReportUnrecognizedServiceAndOrAction | partial | ebMS-2#3.1.5 | ( For each received message, if the receiving MSH does not recognize both the Service and Action values of an incoming message ) | REQUIRED | It responds with an error (ValueNotRecognised/Error). |
r1.2.14 | ProvideRefToMessageIdIntegrity | none | ebMS-2#3.1.6.3 | ( For each generated message, if the RefToMessageId element within the MessageData element is present ) | REQUIRED | It contains the MessageId value of an earlier ebXML Message to which this message relates. |
r1.2.15 | GenerateRefToMessageId | full | ebMS-2#3.1.6.3 | ( For each generated message, if there is no earlier related message ) | REQUIRED | The RefToMessageId element is never present. |
r1.2.16 | GenerateErrorRefToMessageId | full | ebMS-2#3.1.6.3 | ( For each generated message, if a previous message generated and error ) | REQUIRED | The RefToMessageId element is always present with a value indicating the message in error. |
r1.2.18 | ProcessTimeToLive | full | ebMS-2#3.1.6.4 | ( For each received message, if the MSH receives a message for which it is the To Party MSH and the time of the internal clock is greater than TimeToLive ( adjusted to UTC ) ) | REQUIRED | An error message is returned to the From Party MSH (TimeToLiveExpired/Error). |
r1.2.18.1 | GenerateValidUTCTime | full | ebMS-2#3.1.6.4 | ( For each generated message, if a TimeToLive element is present in a generated message. ) | REQUIRED | The TimeToLive element expresses time in UTC, and conform to the XML Schema dateTime. |
r1.2.21 | GenerateDistinctLangValuesForDescription | full | ebMS-2#3.1.8 | ( For each generated message ) | RECOMMENDED | No two Description elements must have the same xml:lang attribute value |
r1.2.22 | GenerateNoPayloadOrApplicationDataInBodyOrManifest | full | ebMS-2#3.2 | ( For each generated message ) | RECOMMENDED | No payload/application data is present in generated SOAP Body / ebXML Manifest elements. |
r1.2.23 | ReportNon-ExistentMIMEPartForManifestReference | full | ebMS-2#3.2.2 | ( For each received message, ff there is not a matching payload for the xlink:href element of a generated Manidfest/Reference element ) | REQUIRED | An error message is directed to the From Party MSH (MimeProblem/Error). |
r1.2.24 | ReportUnresolvableHREFInManifest | full | ebMS-2#3.2.2 | ( For each received message, if the xlink:href element of a Manifest/Reference element on an inbound message specifies a URI that is not a content id (not "cid:"), and that cannot be resolved ) | OPTIONAL | The MSH reports an error to the From Party MSH (MimeProblem/Error) |
r1.2.24.1 | GenerateResolvableHREFInManifest | full | ebMS-2#3.2.2 | ( For each generated message ) | OPTIONAL | The xlink:href element of a Manifest/Reference element on an inbound message specifies a URI that is a content id ("cid:"), and can be resolved |
r1.2.25 | ProcessUnreferencedPayloads | none | ebMS-2#3.2.2 | ( For each received message, if a MIME payload part exists on an incoming message that is not referenced by a Manifest/Reference element ) | STRONGLY RECOMMENDED | It is discarded. |
r1.4 | ErrorHandling | ebMS-2#4.2 | ||||
r1.4.1 | ProcessUpstreamSOAPFault | none | ebMS-2#4.2 | ( For each received message ) | REQUIRED | The MSH can accept and process SOAP Fault values from a downstream SOAP processor. |
r1.4.2 | GenerateCompliantSOAPFaults | full | ebMS-2#4.2 | ( For each generated message, if an MSH returns a SOAP Fault message to the sender of a SOAP message ) | REQUIRED | The returned message conforms to the SOAP specification guidelines for SOAP Fault values. |
r1.4.3 | GenerateWarnings | full | ebMS-2#4.2 | ( For each generated message, when an ebXML message is reporting an error with a highestSeverity value of 'Warning' ) | STRONGLY RECOMMENDED | It is not reported or returned as a SOAP Fault. |
r1.4.4 | ReportDataCommunicationErrors | none | ebMS-2#4.2.2 | ( For each received message ) | STRONGLY RECOMMENDED | Errors associated with data communications protocols are detected and reported using the standard mechanisms supported by that protocol and do not use ebXML reporting mechanisms. |
r1.4.5 | GenerateNoErrorList | full | ebMS-2#4.2.3 | ( For each generated message ) | REQUIRED | The ErrorList extension element of the SOAP Header element is never present if there are no errors to be reported. |
r1.4.6 | ProvideErrorAndHighestSeverityIntegrity | partial | ebMS-2#4.2.3.1 | ( For each generated message ) | REQUIRED | For each generated message, the highestSeverity attribute contains the highest severity of any Error elements generated in an outbound message. |
r1.4.7 | GenerateCorrectCodeContextValue | full | ebMS-2#4.2.3.2.2 | ( For each generated message ) | REQUIRED | The codeContext attribute of any generated Error element is always a URI. |
r1.4.8 | GenerateCorrectCodeContextNamespaceValue | full | ebMS-2#4.2.3.2.2 | ( For each generated message ) | RECOMMENDED | The namespace/scheme specified by codeContext for identifying errorCodes is the default value of urn:oasis:names:tc:ebxml-msg:service:errors. |
r1.4.10 | GenerateCorrectErrorSeverityValue | full | ebMS-2#4.2.3.2.4 | ( For each generated message ) | REQUIRED | Each Error element severity attribute has the value of Warning or Error indicating the severity of the error. |
r1.4.11 | ProvideXPointerAndErrorIntegrity | partial | ebMS-2#4.2.3.2.5 | ( For each generated message, if an error exists in an ebXML element ) | REQUIRED | The location attribute of the Error element is an XPointer to the erroneous element. |
r1.4.12 | GenerateReferencedMIMEPartErrorsWithCID | partial | ebMS-2#4.2.3.2.5 | ( For each generated message, if an error exists in a generated payload MIME part ) | REQUIRED | The location attribute of the generated Error element contains the content-id (via a well-formed "cid:") of the erroroneous MIME part. |
r1.4.13 | GenerateErrorCodesUsingLongDescription | full | ebMS-2#4.2.3.4 | ( For each generated message ) | REQUIRED | The "Long Description" text for each error code provided by the Message Service Specification appears in any relevant Error element. |
r1.4.14 | ReportErrorToMessageOrigin | full | ebMS-2#4.2.4.1 | ( For each received message, when an MSH detects an error in a message ) | RECOMMENDED | The error is reported to the MSH that sent the original message in error. |
r1.4.15 | ProcessWithNoErrorReportLocation | none | ebMS-2#4.2.4.1 | ( For each received message, if the error reporting location cannot be found ) | RECOMMENDED | The error is: Logged; Resolved by other means; and, No further action is taken. |
r1.4.16 | ProcessCPPAErrorURI | full | ebMS-2#4.2.4.2 | ( For each received message, if the ErrorURI is implied in the relevant CPPA ) | OPTIONAL | Then this is used as the Error Report Location. |
r1.4.17 | ProcessWithNoCPPAErrorURIPresent | full | ebMS-2#4.2.4.2 | ( For each received message, if the ErrorURI is unavailable in the relevant CPPA ) | OPTIONAL | A URI specified in the From Party of the message is used as the Error Report Location. |
r1.4.19 | GenerateServiceAndActionForErrorsFromIndependentMessage | full | ebMS-2#4.2.4.3 | ( For each generated message, if an ErrorList is included as part of an independent message ) | REQUIRED | The values of Service and Action are; Service: urn:oasis:names:tc:ebxml-msg:service Action: MessageError |
r1.4.20 | ProcessErrorListWhereHighestSeverityEqualsError | none | ebMS-2#4.2.4.1 | ( For each received message, if the message in error has an ErrorList element with highestSeverity set to Error ) | RECOMMENDED | The error is: Logged; Resolved by other means; and, No further action is taken. |
r1.5 | SynchReply | ebMS-24.3 | ||||
r1.5.1 | ProcessSyncReply | full | ebMS-2#4.3.1 | ( For each received message, if a SyncReply element is present in a message received over a synchronous communications protocol ) | OPTIONAL | That connection is kept open in expectation of the response message using the same connection. |
r1.5.3 | ReportMessageAndCPASyncReplyConflict | full | ebMS-2#4.3.1 | ( For each received message, if the CPPA syncReplyMode is set to none AND SyncReply element is present in an inbound message ) | OPTIONAL | The MSH issues an error (Inconsistent/Error). |
r1.5.4 | GenerateAgreeingMessageAndCPASyncReply | full | ebMS-2#4.3.1 | ( For each generated message, if the CPPA syncReplyMode is set to none ) | OPTIONAL | SyncReply must not be present in generated message. |
ID | Name | Test Coverage | Specification Ref | Precondition | Requirement Level | Assertion |
---|---|---|---|---|---|---|
r2.1 | ReliableMsg | ebMS-2#6 | ||||
r2.1.1.1 | ResendToAckReceivedOrExceeded | full | ebMS-#6 | ( For each reliably generated message, if the candidate MSH fails to receive an Acknowledgment message from a receiving MSH ) | RECOMMENDED | The candidate sends successive retries until an Acknowledgment is received |
r2.1.1.2 | ResendToAckReceivedOrExceeded | full | ebMS-#6 | ( For each reliably generated message, if the candidate MSH fails to receive an Acknowledgment mesage from a receiving MSH ) | RECOMMENDED | The candidate sends successive retries until a predetermined number of retries is exceeded. |
r2.1.1.3 | ResumeAfterAckReceived | full | ebMS-#6 | ( For each reliably generated message, if the candidate MSH receives an Acknowledgment from a receiving MSH ) | RECOMMENDED | The MSH stops resending the message and behaveas as if the message was successfully delivered. |
r2.1.2 | NotifyDeliveryFailureOnExceed | full | ebMS-#6 | ( For each reliably generated message, if the MSH is configured to resend AND the Sending MSH fails to receive any Acknowledgment message from the receiving MSH ) | REQUIRED | The Sending MSH sends successive retries at expected time intervals, then notifies the From party of delivery failure |
r2.1.3.1 | PersistReliableSentMsg | partial | ebMS-2#6.1 | ( For each reliably sent message, after a system interruption AND the system recovers within the TimeToLive window. ) | REQUIRED | The message is kept in persistent storage and processsed as if the interruption had not occured. |
r2.1.3.2 | PersistReliableSentMsgNoAck | partial | ebMS-2#6.1 | ( For each reliably sent message, after a system interruption AND no Ack was received prior to the interruption AND the system recovers within the TimeToLive window. ) | REQUIRED | The message is kept in persistent storage and processsed as if the interruption had not occured. |
r2.1.3.3 | PersistReliableSentMsg | partial | ebMS-2#6.1 | ( For each reliably sent message, after a system interruption ) | REQUIRED | The message is kept in persistent storage. |
r2.1.3.4 | PersistReliableReceivedMsg | partial | ebMS-2#6.1 | ( For each reliably received message, after a system interruption AND the system recovers within the TimeToLive window. ) | REQUIRED | The message is kept in persistent storage and processsed as if the interruption had not occured. |
r2.1.3.5 | PersistReliableReceivedMsgNoAck | partial | ebMS-2#6.1 | ( For each reliably received message, after a system interruption AND no Ack was sent prior to the interruption AND the system recovers within the TimeToLive window. ) | REQUIRED | The message is kept in persistent storage and processsed as if the interruption had not occured. |
r2.1.3.6 | PersistReliableReceivedMsg | partial | ebMS-2#6.1 | ( For each reliably received message, after a system interruption ) | REQUIRED | The message is kept in persistent storage. |
r2.1.4.1 | PersistReliableSentMsg | partial | ebMS-2#6.1 | ( For each reliably sent message, after a system failure AND the system recovers within the TimeToLive window. ) | REQUIRED | The message is kept in persistent storage and processsed as if the interruption had not occured. |
r2.1.4.2 | PersistReliableSentMsg | partial | ebMS-2#6.1 | ( For each reliably sent message, after a system failure AND no Ack was received prior to the interruption AND the system recovers within the TimeToLive window. ) | REQUIRED | The message is kept in persistent storage and processsed as if the interruption had not occured. |
r2.1.4.3 | PersistReliableReceivedMsg | none | ebMS-2#6.1 | ( For each reliably received message, after a system failure ) | REQUIRED | The message is kept in persistent storage. |
r2.1.5 | PersistReceivedMsgID | none | ebMS-2#6.1 | ( For each reliably received message ) | REQUIRED | The MessageId of the received messaged is recorded in persistent storage. |
r2.1.6 | PersistRecdMsg | none | ebMS-2#6.1 | ( For each reliably received message ) | RECOMMENDED | The received message is recorded in its entirety at least until the information in the message has been passed to the application needing to process it. |
r2.1.7 | PersistReceivedMsgTimestamp | none | ebMS-2#6.1 | ( For each reliably received message ) | RECOMMENDED | The time at which a message is received is recorded in persistent storage. |
r2.1.8 | PersistResponseMsg | none | ebMS-2#6.1 | ( For each reliably received message ) | RECOMMENDED | Each response message is stored in its entirety in persistent storage. |
r2.1.12 | TargetAckRequestedToOrNextMSH | full | ebMS-2#6.3.1.1 | ( For each generated non-multi-hop message ) | REQUIRED | The AckRequested element is targeted at the To Party |
r2.1.13.1 | SetAckRequestedSigned | full | ebMS-2#6.3.1.2 | ( For each Acknowledgment request message, If the signed attribute of the AckRequested element is set to "true'". ) | REQUIRED | The Acknowledgment message is signed. |
r2.1.13.2 | SetAckRequestedUnSigned | full | ebMS-2#6.3.1.2 | ( For each Acknowledgment request message, If the signed attribute of the AckRequested element is set to "false'". ) | REQUIRED | The Acknowledgment message is unsigned. |
r2.1.14.1 | SetSignedAttributeAfterVerifyReceivingMSHAckSupport | full | ebMS-2#6.3.1.2 | ( For each received message with an AckRequested with the Signed attribute set to True consistent with the CPPA, if the Receiving MSH supports signed acknowledgment messages of the type requested ) | REQUIRED | The Sending MSH sends back a signed Acknowledgment. |
r2.1.14.2 | SetSignedAttributeAfterVerifyReceivingMSHAckSupport | full | ebMS-2#6.3.1.2 | ( For each received message with an AckRequested with the Signed attribute set to True consistent with the CPPA, if the Receiving MSH does not support signed acknowledgment messages of the type requested ) | REQUIRED | The MSH generates an Error of type Inconsistent, and severity = Error . |
r2.1.14.3 | SetSignedAttributeAfterVerifyReceivingMSHAckSupport | full | ebMS-2#6.3.1.2 | ( For each received message with an AckRequested with the Signed attribute set to True and NOT consistent with the CPPA, if the Receiving MSH supports signed acknowledgment messages of the type requested ) | REQUIRED | The MSH generates an Error of type Inconsistent, and severity = Warning . |
r2.1.14.4 | SetSignedAttributeAfterVerifyReceivingMSHAckSupport | full | ebMS-2#6.3.1.2 | ( For each received message with an AckRequested with the Signed attribute set to False, if the Receiving MSH supports unsigned acknowledgment messages ) | REQUIRED | The MSH returns an unsigned Acknowledgment. |
r2.1.15 | ReturnAckMsg | full | ebMS-2#6.3.1.2 | ( For each Acknowledgement request message, if an Acknowledgment of the type requested on an inbound message can be produced ) | REQUIRED | A message containing an Acknowlegment element is returned to the Sending MSH. |
r2.1.16a | GenerateInconsistentErrorOnAck | full | ebMS-2#6.3.1.2 | ( For each Acknowledgement request message, if an Acknowledgment of the type requested cannot be produced ) | REQUIRED | An error is reported to the Sending MSH. The error is Inconsistent/Error if the request in inconsistent with the relevant CPA. |
r2.1.16b | GenerateInconsistentWarningOnAck | full | ebMS-2#6.3.1.2 | ( For each Acknowledgment request message, if an Acknowledgment Message of the type requested cannot be produced ) | REQUIRED | An error is reported to the Sending MSH. The error is Inconsistent/Warning if the mode is not supported. |
r2.1.17 | SendAckToFromParty | full | ebMS-2#6.3.1.3 | ( For each Acknowledgment request message, If an Acknowledgment is requested of the MSH node acting in the role of To Party ) | REQUIRED | The Acknowledgment element generated is targeted to the MSH node acting in the role of From Party. |
r2.1.18 | GenererateAckWithNoPayloadAndNoAckRequested | full | ebMS-2#6.3.1.4 | ( For each generated Acknowledgment message, if the message contains no payloads ) | REQUIRED | The message does not include an AckRequested element. |
r2.1.19 | ReportErrorWithoutAckRequeseted | full | ebMS-2#6.3.1.4 | ( For each Acknowledgment message, if the message contains an ErrorList element ) | REQUIRED | The message does not include an AckRequested element. |
r2.1.20.1 | SpecifyNoSOAPActorToPartyAck | full | ebMS-2#6.3.2.1 | ( For each generated Acknowledgment message, if there is no SOAP actor atrribute present on an Acknowledgement element ) | STRONGLY RECOMMENDED | The default target is the ToParty MSH. |
r2.1.20.2 | SpecifySOAPActorToPartyAck | full | ebMS-2#6.3.2.1 | ( For each Acknowledgment message ) | STRONGLY RECOMMENDED | The SOAP actor attribute in a generated Acknowledgment element has a value corresponding to the AckRequested element of the message being acknowledged. |
r2.1.21 | GenerateAckMsgTimestamp | full | ebMS-2#6.3.2.2 | ( For each generated Acknowledgment message, if the From element is present ) | REQUIRED | The Timestamp element is present within any generated Acknowledgment element. The value is in XML Schema dateTime format in the UTC timezone and represents the time at which the message being acknowledged was received by the MSH generating the Ackowledgement Message. |
r2.1.22 | GenerateAckUsingMsgIDInRefToMessageID | full | ebMS-2#6.3.2.3 | ( For each generated Acknowledgment message ) | REQUIRED | The RefToMessageId element contains the MessageId of the message whose delivery is being acknowledged. |
r2.1.23 | IdentifyPartyWithAckFromElement | full | ebMS-2#6.3.2.4 | ( For each generated Acknowledgment message, if the From element is present in an inbound message ) | REQUIRED | The From element in a generated Acknowledgment element contains an identifier of the party sending the Acknowledgment Message. |
r2.1.24 | IdentifyPartyWithoutAckFromElement | full | ebMS-2#6.3.2.4 | ( For each generated Acknowledgment message, if the From element is omitted in an inbound message ) | REQUIRED | The value of the From element in the MessageHeader is used to identify the party sending the acknowledgment. |
r2.1.25 | UseSignedAckMustContainRef | full | ebMS-#6.3.2.5 | ( For each generated Acknowledgment message, if the message being acknowledged contains an AckRequested element with the signed attribute set to "true" ) | REQUIRED | One or more Reference elements are included in the generated Acknowledgment element. |
r2.1.26 | QualifyRefElementByNamespace | full | ebMS-#6.3.2.5 | ( For each generated Acknowledgment message ) | REQUIRED | Any Reference elements included in a generated Acknowledgment element are namespace qualified to the XML Signature namespace and conform to the XML Signature specification. |
r2.1.27 | NotifyClientOfAckDelivery | none | ebMS-#6.3.2.5 | ( For each received Acknowledgment message ) | OPTIONAL | The From Party MSH notifies the client application of successful delivery of the referenced message. |
r2.1.28 | IgnoreDuplicateRefToMessageID | none | ebMS-#6.2.2.5 | ( For each received Acknowledgment message, if any subsequent Error or Acknowledgment messages with a RefToMessageId value equal to an already received Acknowledgment Message are received ) | OPTIONAL | The messages are ignored. |
r2.1.29 | SetAckServiceActionValues | full | ebMS-#6.3.2.7 | ( For each generated Acknowledgment message, If no errors were detected in the message received AND the Acknowledgment Message is being sent with no payload data ) | REQUIRED | The Service and Action values are: Service - urn:oasis:names:tc:ebxml-msg:service Action - Acknowledgment |
r2.1.30.1 | SetDuplicateEliminationAlways | full | ebMS-#6.4.1 | ( For each generated message, if the CPPA DuplicateElimination element = "always" ) | REQUIRED | The DuplicateElimination element is included to indicate to a Receiving MSH that it must eliminate duplicates. |
r2.1.30.2 | SetDuplicateEliminationtoNever | full | ebMS-#6.4.1 | ( For each generated message, if the CPPA DuplicateElimination element = "never" ) | REQUIRED | The DuplicateElimination element not present. |
r2.1.30.3 | SetDuplicateEliminationPerMessage | full | ebMS-#6.4.1 | ( For each generated message, if the CPPA DuplicateElimination element = "per message" AND the party requires duplicate elimination ) | REQUIRED | The DuplicateElimination element is present in the header of the message. |
r2.1.30.4 | ReceiveDuplicateEliminationAlways | full | ebMS-#6.4.1 | ( For each received message, if the CPPA DuplicateElimination element = "always" AND the received message does not contain a DuplicateElimination element ) | RECOMMENDED | The receiving MSH generates an Error message with an errorCode of Inconsistent and a Severity of Error. |
r2.1.30.5 | ReceiveDuplicateEliminationtoNever | full | ebMS-#6.4.1 | ( For each received message, if the CPPA DuplicateElimination element = "never" AND the received message contains a DuplicateElimination element ) | REQUIRED | The receiving MSH generates an Error message with an errorCode of Inconsistent and a Severity of Error. |
r2.1.32.1 | PersistMsgWithDuplicateElimination | full | ebMS-#6.4.1 | ( For each reliably sent message, if Duplication element is present on an inbound message ) | REQUIRED | The message is presented to the To Party Application at-most-once. |
r2.1.32.2 | PersistMsgWithDuplicateEliminationAndInterruption | full | ebMS-#6.4.1 | ( For each reliably sent message, if Duplication element is present on an inbound message AND the system recovers from an interruption within the TimeToLive window. ) | REQUIRED | The message is presented to the To Party Application at-most-once. |
r2.1.33.1 | ReportErrorIfDuplicateEliminationUnsupported | full | ebMS-#6.4.1 | ( For each received message containing a DuplicationElimination element, if duplicate elimination is not supported ) | RECOMMENDED | An Inconsistent/Error is reported to the From Party. |
r2.1.33.2 | ReportErrorDuplicateEliminationMsgToCPPA | full | ebMS-#6.4.1 | ( For each reliably received message, if the value of duplicateElimination in the CPPA is "always" AND a DuplicateElimination element is not present in the message ) | RECOMMENDED | An Inconsistent/Error is reported to the From Party. |
r2.1.33.3 | ReportErrorDuplicateEliminationMsgToCPPA | full | ebMS-#6.4.1 | ( For each reliably received message, if the value of duplicateElimination in the CPPA is "never" AND a DuplicateElimination element is present in the message ) | RECOMMENDED | An Inconsistent/Error is reported to the From Party. |
r2.1.35 | RetryIntervalMinLapseTime | full | ebMS-#6.4.4 | ( For each reliably re-sent message, if the RetryInterval is present in the CPPA ) | OPTIONAL | The minimum time elapsed between re-sends of the same message is equal to the RetryInterval.. |
r2.1.36 | SetTimeToLive | full | ebMS-#6.4.5 | ( For each reliably re-sent message, if the RetryInterval element is present in the CPPA AND the Retries element is present in the CPPA ) | REQUIRED | The TimeToLive for the message satisfies the equation: TimeToLive > Timestamp + ((Retries + 1) * RetryInterval) |
r2.1.37.1 | PersistSentMsgLength | full | ebMS-#6.4.6 | ( For each reliably received message, if the PersistDuration parameter is present in the CPPA AND DuplicationElimination element is present in the messsage AND the message is presented once and only once to the application AND the same message is received again by the MSH before PersistDuration expires ) | REQUIRED | The message is presented only once to the application. |
r2.1.37.2 | PersistSentMsgLength | full | ebMS-#6.4.6 | ( For each reliably received message, if the PersistDuration parameter is present in the CPPA AND AckRequested element is present in the messsage AND the message is presented once and only once to the application AND the same message is received again by the MSH before PersistDuration expires ) | REQUIRED | An Acknowledgement message is sent back to the sending MSH. |
r2.1.38 | SendNoMsgWithLapsePersistDurationMsgID | full | ebMS-#6.4.6 | ( For each generated message, if the length of time specified by the PersistDuration parameter in the relevant CPA has passed since a message was first sent ) | OPTIONAL | A message with the same MessageId will not be sent again. |
r2.1.39 | ReptDeliveryFailureIfPersistDurationExpired | full | ebMS-#6.4.6 | ( For each reliably received message, if a message cannot be successfully delivered before expiry of the PersistDuration period ) | OPTIONAL | A delivery failure is reported. |
r2.1.40 | TimestampPersistDurationGreaterThanTimeToLive | full | ebMS-#6.4.4 | ( For each reliably sent message ) | REQUIRED | For each reliably sent message, the message satisfies the equation: PersistDuration > TimeStampe + TimeToLive. |
r2.1.41 | IgnoreSyncReplyMode | none | ebMS-#6.4.7 | ( For each reliablly sent messsage, if the communications protocol is not synchronous ) | REQUIRED | The value of the syncReplyMode in the relevant CPA is ignored. |
r2.1.42.1 | ReturnSyncReplyElementInResponsePayload | full | ebMS-#6.4.7 | ( For each reliably sent message, if ( in the context of the CPPA ) the syncReplyMode is not none ) | REQUIRED | A SyncReply element is present in the message. |
r2.1.42.2 | ReturnSyncReplyResponsePayload | full | ebMS-#6.4.7 | ( For each reliably sent message, if ( in the context of the CPPA ) the syncReplyMode is not none ) | REQUIRED | The MSH returns the response on the same synchronous connection. |
r2.1.44 | GenerateAckWhenAckRequested | full | ebMS-#6.5.3 | ( For each reliably received message, if the AckRequested element that has a SOAP actor URI targeting the MSH ) | REQUIRED | An Acknowledgment Message is generated. |
r2.1.45 | PersistAckWithOriginalMsg | none | ebMS-#6.5.3 | ( For each received Acknowledgment message ) | REQUIRED | The essage is placed in persistent storage with the same PersistDuration as the original message. |
r2.1.46 | DeliverAckWithResponse | full | ebMS-#6.5.3 | ( For each Acknowledgment message ) | REQUIRED | The message can be delivered as part of the normal response to the received message. |
r2.1.47 | ResendMsgOnCommError | none | ebMS-#6.5.4 | ( For each reliably sent message, if there is a communications protocol error during a message send ) | REQUIRED | The message is resent as if the MSH had not received an Acknowledgment Message. |
r2.1.48 | SendOriginalAckOnDuplicateMsg | partial | ebMS-#6.5.5 | ( For each reliably received message, if a duplicate message is received AND the original acknowledgment is still present in the persistent store ) | OPTIONAL | This original Acknowledgment Message is resent. |
r2.1.49 | GenerateSyncResponseOnDuplicateMsg | partial | ebMS-#6.5.5 | ( For each reliably received message, If a duplicate message is received AND the original acknowledgment is not present in the persistent store AND the syncReplyMode is set to none AND the CPA indicates that an application response is included ) | OPTIONAL | response from the application is gathered by the MSH and returned synchronously. |
r2.1.50 | GenerateAckMsgOnNonSyncDuplicateMsg | partial | ebMS-#6.5.5 | ( For each reliably received message, if a duplicate message is received AND the original acknowledgment is not present in the persistent store AND the syncReplyMode is not none ) | OPTIONAL | A new Acknowledgment Message is generated and sent. |
r2.1.51.1 | ReportErrorOnMsgWithAckReqNoTransmit | full | ebMS-#6.5.7 | ( For each reliably received message, if the message contains an AckRequested element AND the message cannot be delivered because the message could not be transmitted ) | STRONGLY RECOMMENDED | An error message is sent to the From Party. The reported error is DeliveryFailure/Error. |
r2.1.51.2 | GenerateWarningErrorOnMsgWithAckRequested | full | ebMS-#6.5.7 | ( For each reliably received message, if the message contains an AckRequested element AND the message was transmitted but no acknowledgement was received ) | STRONGLY RECOMMENDED | An error message is sent to the From Party. The reported error is DeliveryFailure/Warning. |
r2.1.52 | NotifyFailureByAlternateMeans | none | ebMS-#6 | ( For each reliably received message, if an Error Message is generated with an error code set to DeliveryFailure AND an Error Message cannot be delivered successfully ) | REQUIRED | The ultimate destination of the error message is informed of the failure by some undefined means. |
r2.1.53 | GenerateCorrectSOAPHeaderNamespace | full | ebMS-2#2.3.3 | ( For each generated message ) | REQUIRED | A SOAP Header element is namespace qualified as per the SOAP namespace declaration in the SOAP Envelope element. |
r2.2 | MsgOrder | ebMS-2#9 | ||||
r2.2.2 | EnableMsgOrderWithReliableMsg | full | ebMS-2#9 | ( For each received message, if the message contains a MessageOrder element ) | REQUIRED | The DuplicateElimination is present and AckRequested directed to the To Party MSH and absence of a SyncReply element. |
r2.2.3 | ProcessSequenceMsg | full | ebMS-2#9.1.1 | ( For each received message, when two messages are received, each with a MessageOrder element, and the same conversationID ) | REQUIRED | The MSH processes messages only in the sequence indicated by the SequenceNumber element. |
r2.2.4 | PassOrderedMsgToApplication | full | ebMS-2#9.1.1 | ( For each received message, if the message contains a MessageOrder element AND when receiving ordered messages with the same conversationID out of sequence ) | REQUIRED | The message is not passed to the destination application until all messages with a lower (earlier) SequenceNumber have previously been passed. |
r2.2.5 | GenerateDeliveryFailureOnOutOfSequMsg | full | ebMS-2#9.1.1 | ( For each received message, if the message contains a MessageOrder element AND if the maximum number of out-of-sequence ordered messages have been received ) | REQUIRED | The Sending MSH is sent an error and the error code is DeliveryFailure and severity set to Error. |
r2.2.6.1.1 | UseZeroSequenceNoForFirstOrderedMsgForConversation | full | ebMS-2#9.1.1 | ( For each received message, if the message contains a MessageOrder element AND if this is the first ordered message for the ConversationID ) | REQUIRED | The SequenceNumber element has value of 0." |
r2.2.6.1.2 | UseStatusResetForFirstOrderedMsgForConversation | full | ebMS-2#9.1.1 | ( For each received message, if the message contains a MessageOrder element AND if this is the first ordered message for the ConversationID ) | REQUIRED | The status attribute of the message is set to "Reset". |
r2.2.6.2.1 | UseZeroSequenceNoForFirstOrderedMsgAfterReset | full | ebMS-2#9.1.1 | ( For each received message, if the message contains a MessageOrder element AND this is the first ordered message after a reset instruction is sent by the Sending MSH ) | REQUIRED | The SequenceNumber element has value of 0 and the status attribute of the message is set to Reset. |
r2.2.6.2.2 | UseStatusReseetForFirstOrderedMsgAfterReset | full | ebMS-2#9.1.1 | ( For each received message, if the message contains a MessageOrder element AND this is the first ordered message after a reset instruction is sent by the Sending MSH ) | REQUIRED | The message is set to Reset. |
r2.2.6.3 | UseZeroSequenceNoForFirstOrderedMsgAfterWrap | full | ebMS-2#9.1.1 | ( For each received message, if the message contains a MessageOrder element AND this is the first ordered message after the sequence wrapped at value 99999999 ) | REQUIRED | The SequenceNumber element has value of 0. |
r2.2.8 | SetStatusToContinueMsg | full | ebMS-2#9.1.1 | ( For each generated message, if the message contains a MessageOrder element ) | REQUIRED | The sequenceNumber element status attribute has a value of "Reset" or "Continue" |
r2.2.9 | ResetMsgSeqForConversation | partial | ebMS-2#9.1.1 | ( For each generated message, when sending a message with the MessageOrder element AND if the status attribute is set to "Reset" ) | REQUIRED | All previous messages for this conversation must have been accounted for. |
r2.2.10.2 | SyncReplyMsgNotIncludeMsgOrder | full | ebMS-2#9.2 | ( For each received message, if the message contains a SyncReply element ) | REQUIRED | A MessageOrder element is never included in the same message as a SyncReply element. |
r2.2.11 | ReportErrorMsgOrderSyncReply | full | ebMS-2#9.2 | ( If a message is received in which the MessageOrder element is included with a SyncReply element ) | RECOMMENDED | An error is reported. The error is Inconsistent/Error. |
r2.4 | SecurityAndCommunicationChannels | ebMS-2#4 | ||||
r2.4.1.1 | Signature elements SignOutboundMsg | full | ebMS-2#4.1 | ( For each generated message, when one or more Signature elements is present ) | REQUIRED | It is the child of the SOAP Header. |
r2.4.1.2 | Signature elements SignOutboundMsg | full | ebMS-2#4.1 | ( For each generated message, when one or more Signature elements is present ) | REQUIRED | It is namespace qualified witih XML Signature. |
r2.4.1.3 | Signature elements SignOutboundMsg | full | ebMS-2#4.1 | ( For each generated message, when one or more Signature elements is present ) | REQUIRED | Its structure and content conform to the XML Signature specification. |
r2.4.2 | AttributeSignatureElement | none | ebMS-2#4.1 | ( If there is more than one Signature element within the SOAP Header ) | REQUIRED | It is the first signature that represents digital signing of the message by the From Party MSH. |
r2.4.4 | ApplySecurityBasedOnTransportOfCPA | none | ebMS-2#4.1.2.1 | ( For each generated message if,based upon the Transport section of the relevent CPA, a signature is required for the entire message ) | REQUIRED | A Signature element must be present, and its SignedInfo element contains a Reference element to the SOAP envelope which has a URI attribute value of "" |
r2.4.5.1 | GenerateSignToXMLDSIG | full | ebMS-2#4.1.3 | ( For each signed message ) | REQUIRED | Digital signatures are generated and rendered according the XML Signature specification (XMLDSIG). |
r2.4.5.2 | GenerateSignToXMLDSIG | full | ebMS-2#4.1.3 | ( For each signed message ) | REQUIRED | The SignedInfo element has a CanonicalizationMethod, SignatureMethod and one or more Reference elements. |
r2.4.5.3 | GenerateSignToXMLDSIG | full | ebMS-2#4.1.3 | ( For each signed message ) | REQUIRED | The SignatureMethod element is present and has an Algorithm attribute on any generated digitally signed message. |
r2.4.6 | SignCanonicalMethod | none | ebMS-2#4.1.3 | ( For each generated message with one or more Signature elements ) | RECOMMENDED | The canonicalization method applied to the data to be signed is Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315" |
r2.4.8 | SignatureMethodAlgorithmAttribute | full | ebMS-2#4.1.3 | ( For each generated message with one or more Signature elements ) | RECOMMENDED | The value of the Algorithm attribute is Algorithm="http://www.w3.org/2000/09/xmldsig#dsa-sha1" |
r2.4.9 | SupportDSA-SHA1SignAlgorithm | none | ebMS-2#4.1.3 | ( For each generated message with one or more Signature elements, ) | REQUIRED | The MSH supports the signature algorithm DSA-SHA1, validates the signature and passes the message to the application. |
r2.4.11 | AddOptionalReferenceAttribute | full | ebMS-2#4.1.3 | ( For each generated message with one or more Signature elements ) | STRONGLY RECOMMENDED | The MSH supports the optional addition of the informative Type attribute with value "http://www.w3.org/2000/09/xmldsig#Object" on the XML Signature Reference element. |
r2.4.12 | IncludeMandatoryTransformElementToEnvelopedSign | full | ebMS-2#4.1.3 | ( For each generated message with one or more Signature elements ) | REQUIRED | The generated XML Signature Reference element includes a child Transform element which in turn includes a first Transform element with an Algorithm attribute of value "http://www.w3.org/2000/09/xmldsig#enveloped-signature". |
r2.4.13 | GenerateMandatoryTransformWithExcludeSOAPActor | full | ebMS-2#4.1.3 | ( For each generated message with one or more Signature elements, (CPA, Transport section) that requires signature ) | REQUIRED | A second Transform element is generated with the requisite XPath element excluding all elements with SOAP actor attributes targetting the nextMSH or next SOAP node. |
r2.4.15 | CanonicalizationTransformElementAlgorithmAttribute | full | ebMS-2#4.1.3 | ( For each generated message with one or more Signature elements ) | OPTIONAL | The last generated Transfom element has an Algorithm attribute with a value of "http://www.w3.org/TR/2001/REC-xml-c14n-20010315". |
r2.4.16 | XMLSignReferenceURIForPayload | full | ebMS-2#4.1.3 | ( For each generated message with one or more Signature elements ) | REQUIRED | Any payload data requiring digital signature is identified by an XML Signature Reference element that has a URI attribute resolving to the location of that data. |
r2.4.17 | MapSignReferenceURIToManifestPayload | full | ebMS-2#4.1.3 | ( For each generated message with one or more Signature elements ) | RECOMMENDED | The value of the URI attribute of a generated XML Signature Reference element matches the xlink:href URI value present in the Manifest/Reference element corresponding to that same payload. |
r2.4.18 | GenerateSignPriorToTransferEncoding | none | ebMS-2#4.1.3 | ( For each generated message with one or more Signature elements, and with transfer encoding ) | REQUIRED | Signature generation takes place before any transfer encoding (eg base64) is applied to the SOAP Envelope or payload MIME parts. |
r2.4.19 | SignAckReferenceElementList | none | ebMS-2#4.1.3.2 | ( For each received signed message ) | REQUIRED | A digitally signed inbound message may be acknowledged with a digitally signed acknowledgement. Any such acknowledgement message contains an XML Signature Reference element list corresponding to the Reference elements contained in the original message. |
r2.4.20 | AuthenticatePartyByCommunicationChannel | none | ebMS-2#4.1.4.3 | ( For each generated message ) | OPTIONAL | The communication channel used to transport the ebXML message can be used to provide uni or bi-directional party authentication (eg TLS over TCP/IP). |
r2.4.21 | ProvideMsgContentDataIntegrityByCommunicationChannel | none | ebMS-2#4.1.4.4 | ( For each generated message ) | OPTIONAL | The communication channel used to transport the ebXML message can be used to provide data integrity of the message content (eg TLS over TCP/IP). |
r2.4.22 | SignMsgPriorToEncryption | none | ebMS-2#4.1.4.5 | ( For each generated message if,based upon the Transport section of the relevent CPA, a signature is required for the entire message AND if signature and encryption of a message component is requested of the MSH ) | REQUIRED | Signing takes place prior to encryption. |
r2.4.23 | ProvideMsgContentDataConfidentialityByCommunicationChannel | none | ebMS-2#4.1.4.6 | ( For each generated message ) | REQUIRED | The communication channel used to transport the ebXML message can be used to provide data confidentiality for the message content (eg TLS over TCP/IP). |
r2.4.24 | AuthorizeMsgWithBilateralAuthenticationByNetworkProtocol | none | ebMS-2#4.1.4.8 | ( For each generated message ) | OPTIONAL | The source of an ebXML message can be authorised by using a secure network protocol for bilateral authentication of certificates prior to establishing a session (eg TLS over TCP/IP). |
ID | Name | Test Coverage | Specification Ref | Precondition | Requirement Level | Assertion |
---|---|---|---|---|---|---|
r3.1 | MessageStatusService | ebMS-2#7 | ||||
r3.1.1 | GenerateStatusResponseWithReliableMessaging | full | ebMS-2#7 | ( For each received message, if the message contains a StatusRequest element AND the RefToMessageId child element references a previously received message that had been sent reliably and is present in persistent storage ) | RECOMMENDED | A Message Status Response Message is returned. |
r3.1.2 | GenerateStatusResponseWithoutReliableMessaging | full | ebMS-2#7 | ( For each received message, if the message contains a StatusRequest element AND the RefToMessageId child element references a previously received message that had not been sent reliably ) | OPTIONAL | A Message Status Response is returned. |
r3.1.4 | ReportUnsupportedService | full | ebMS-2#7 | ( For each received message, if the message contains a StatusRequest element AND the message is received for a service that is not supported ) | RECOMMENDED | An Error Message is returned with an errorCode of "NotSupported". |
r3.1.5 | GenerateValidStatusRequestMessage | none | ebMS-2#7.1.1 | ( For each received message, If the MessageHeader child Action element is equal to "StatusRequest" ) | REQUIRED | The message consists of no payload and the MessageHeader/StatusRequest elements configured as specified in the Message Service Specification and is not included along with any of the Manifest, StatusResponse, or ErrorList elements. |
r3.1.7 | ProcessUnauthorizedStatusRequest | partial | ebMS-2#7.1.3 | ( For each received message, if the message contains a StatusRequest element AND the StatusRequest child RefToMessageId element value is recognized by the MSH AND the message is received from an party deemed to be unauthorised ) | OPTIONAL | A response is sent with the messageStatus attribute set to "UnAuthorized". |
r3.1.9 | ProvideRefToMessageIdAndMessageIdIntegrity | full | ebMS-2#7.3.1 | ( For each received message, if the message contains a StatusRequest element AND the StatusRequest child RefToMessageId element value is recognized by the MSH AND the message is received from an party deemed to be authorised AND a StatusResponse is generated for this request ) | RECOMMENDED | In the returned StatuResponse element, the RefToMessageId element child of the MessageData element specifies the MessageId of the message containing the associated StatusRequest element. In addition, the RefToMessageId element child of the StatusResponse elements always contains the MessageId of the message whose status is being queried. |
r3.1.10.1 | SetTimestampRecognizedAndAuthorized | full | ebMS-2#7.3.2 | ( For each received message, if the message contains a StatusRequest element AND the the StatusRequest child RefToMessageId element value is recognized AND the message is received from an party deemed to be authorised ) | REQUIRED | In the response message, the Timestamp child element of the StatusResponse element contains the time at which the message being reported on was originally received. |
r3.1.10.2 | SetTimestampNotRecognized | full | ebMS-2#7.3.2 | ( For each received message, if the message contains a StatusRequest element AND the the StatusRequest child RefToMessageId element value is not recognized AND the message is received from an party deemed to be authorised ) | REQUIRED | The Timestamp child element of the StatusResponse element is not present in the response message. |
r3.1.10.3 | SetTimestampNotAuthorized | full | ebMS-2#7.3.2 | ( For each received message, if the message contains a StatusRequest element AND the the StatusRequest child RefToMessageId element value is recognized AND the message is received from an party deemed to be unauthorised ) | REQUIRED | The Timestamp child element of the StatusResponse element is not present in the response message. |
r3.1.13 | GenerateReceivedMessageStatus | full | ebMS-2#7.3.3 | ( For each received message, if the message contains a StatusRequest element AND the the StatusRequest child RefToMessageId element value is recognized AND the message is received from an party deemed to be authorised ) | REQUIRED | The messageStatus attribute in the StatusResponse element has a value of 'Received'. |
r3.1.14 | GenerateProcessedMessageStatus | full | ebMS-2#7.3.3 | ( For each received message, if the message contains a StatusRequest element AND the the StatusRequest child RefToMessageId element value is recognized AND the message is received from an party deemed to be authorised ) | REQUIRED | The messageStatus attribute in the StatusResponse element has a value of 'Processed'. |
r3.1.15 | GenerateReceivedMessageStatus | full | ebMS-2#7.3.3 | ( For each received message, if the message contains a StatusRequest element AND the the StatusRequest child RefToMessageId element value is recognized AND the message is received from an party deemed to be authorised AND the message identified by the RefToMessageId element in the StatusRequest element has been forwarded by the MSH ) | REQUIRED | A StatusResponse with a 'Forwarded' messageStatus is be present in the returned message. |
r3.2 | PingService | ebMS-2#7 | ||||
r3.2.2 | ReportServiceNotSupported | full | ebMS-2#7 | ( For each received message, if the message header Action element contains a child Ping element AND The requested service is not supported ) | RECOMMENDED | A message with an Error element is returned with an errorCode of "NotSupported" and a highestSeverity attribute set to "Error". |
r3.2.3 | GenerateValidPingMessageStructure | none | ebMS-2#7.1 | ( For each generated message, if the message headerAction element contains a child Ping element ) | REQUIRED | The message consists of no payload and the MessageHeader and Signature elements are configured as specified in the Message Service Specification. |
r3.2.4 | GeneratePongResponse | full | ebMS-2#7 | ( For each received message, if the message header Action element contains a child Ping element AND the requested service is supported AND The Ping request is occurring under normally expected conditions, and the the received message is authorized and not interpreted by the Receiving party as part of an attack. ) | OPTIONAL | A message header Action element containing a child Pong element is returned. The message contains no payload and the MessageHeader & Signature elements are configured as specified in the Message Service Specification. |
r3.3 | MultiHopModule | ebMS-2#10 | ||||
r3.3.1 | SetMultiHopIntermediaryNextMSH | full | ebMS-2#10.1 | ( For each generated multi-hop message ) | OPTIONAL | The AckRequested and Acknowledgment elements have the SOAP actor attribute set to NextMSH (urn:oasis:names:tc:ebxml-msg:actor:nextMSH). |
r3.3.2 | RemoveIntermediaryAckRequested | full | ebMS-2#10.1.1 | ( For each received multi-hop message, when a node acts as an intermediary ) | REQUIRED | The node removes any AckRequested element with a SOAP actor attribute of NextMSH. |
r3.3.3 | InsertIntermediaryAckRequested | none | ebMS-2#10.1.1 | ( For each received multi-hop message, when a node acts as an intermediary ) | OPTIONAL | The node can insert a single AckRequested element with a SOAP actor attribute of NextMSH. |
r3.3.4 | GenerateSingleAckRequestedForNextMSH | full | ebMS-2#10.1.1 | ( For each generated multi-hop message with the SOAP actor attribute value targetting the NextMSH ) | REQUIRED | There will not be two AckRequested elements in the same message. |
r3.3.5 | SyncReplyNoAckRequestedForNextMSH | full | ebMS-2#10.1.1 | ( For each generated multi-hop message, if a SyncReply element is present in a message ) | REQUIRED | An AckRequested element with SOAP actor attribute targetting the NextMSH is never included. |
r3.3.6 | GenerateErrorWithSyncReplyAckRequested | full | ebMS-2#10.1.1 | ( For each received multi-hop message, if the SyncReply and AckRequested elements is received in one message AND the AckRequested element is received in the same message ) | REQUIRED | An error is reported with an errorCode of "Inconsistent". |
r3.3.7 | GenerateIntermediaryAckMsgIfNoSyncReply | full | ebMS-2#10.1.1 | ( For each received multi-hop message, when a node acts in the role of intermediary AND no SyncReply element is specified ) | OPTIONAL | A node may synchronously return an intermediate Acknowledgment Message to the Sending MSH. |
r3.3.8 | GenerateAckBasedBasedOnActor | full | ebMS-2#10.1.3 | ( For each received multi-hop message, if an inbound message contains two AckRequested elements where one addresses NextMSH AND another AckRequest addresses the ToParty MSH AND the receving MSH is the ToParty MSH ) | REQUIRED | , the MSH node is in the combined role of Next and ToParty MSH, and will send two Acknowledgments |
r3.3.9 | GenerateIntermediaryAckMsgAtComplete | partial | ebMS-2#10.1.3 | ( For each received multi-hop message, a reliable message received by an MSH node in the role of intermediary ) | REQUIRED | The message is not acknowledged until the message is both persisted and delivered to the Next MSH. |
r3.3.10 | GenerateIntermediarySignedAck | full | ebMS-2#10.1.4 | ( For each received multi-hop message, when a signed Acknowledgment Message is requested by an intermediate node ) | REQUIRED | The message is only generated as a standalone message and is not bundled with any other data (payload). |
r3.3.11 | NoMsgOrderProcessForIntermediary | full | ebMS-2#10.2 | ( For each received multi-hop message, when the MSH acts in the role of intermediary ) | REQUIRED | The MSH does not attempt to participate in Message Order processing. |
r3.3.12.1 | RequestDownstreamAck | full | ebMS-2#6.3.1 | ( For a downstream ( Next ) processor , if an AckRequested element is received ) | REQUIRED | An acknowledgment is returned. |
r3.3.13 | GenerateMultipleAckRequested | full | ebMS-2#6.3.1 | ( For each generated multi-hop message, if there are two AckRequested elements in a generated message Header ) | REQUIRED | The two AckRequested elements do not specify the same value for their respective SOAP actor attributes. |
r3.3.14 | TargetAtMostOneAckNextMSH | full | ebMS-2#6.3.1 | ( For each generated multi-hop message ) | REQUIRED | At most one AckRequested element may be targeted at the actor URI for the Next MSH. |
r3.3.15 | TargetAtMostOneAckToPartyMSH | full | ebMS-2#6.3.1 | ( For each generated multi-hop message ) | REQUIRED | At most one AckRequested element may be targeted at the actor URI for the To Party MSH in a given message. |
r3.3.16 | IgnoreIntermediaryDuplicates | full | ebMS-#6.5.2 | ( For each reliably received message, when the MSH acts as an intermediary node ) | REQUIRED | The MSH does not filter out perceived duplicate messages. |
r3.3.18 | ControlIntermediaryMSHHandling | none | ebMS-2#4.1.3 | ( For each received multi-hop message, when a node acts as an intermediary ) | REQUIRED | The MSH does not change, format or in any way modify any element not targetted at that intermediary MSH. The MSH does not add or delete white space. |
r3.3.19 | TargetAckRequestedToOrNextMSH | full | ebMS-2#6.3.1.1 | ( For each generated multi-hop message ) | REQUIRED | The AckRequested element is targeted at the NextMSH |
NOTE: Test reqs related to Multi-hop module (r2.3) + maybe services, should be moved to Level 3, I think. (I don't recall what distinction we wanted between Level 2 and 3) ---------------------------------------------------------------------------- r2.2.1: not needed as it is subsumed by r2.2.2 [MIKE] - DONE r2.2.2: Just re-formulating: [precondition]: sending M with MessageOrder element [assertion]: REQUIRED:The DuplicateElimination element is present, the AckRequested elet is present, and the SyncReply element is not present.. [MIKE] - DONE ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ [mm1: NOTICED THAT I HAD A CUT-PASTE ERROR IN PRE-CONDITION/CONDITION - R2.2.2-C1 NOT R.2.1.52-C1. SORRY. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ r2.2.3: re-wording: [precondition]: receiving two messages with MessageOrder element, and same conv ID [assertion]: REQUIRED: the MSH passes these messages in the right order to application. [MIKE] - DONE ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ [mm1: JACQUES, I THINK THE REQUIREMENT HAS A BIT OF A DIFFERENT MEANING THAN YOU HAVE HERE... PROCESSING WITHIN A SEQUENCE AND THEN PASSING TO AN APPLICATION ARE TWO DIFFERENT EVENTS/STEPS TO ME. REQUIREMENT IS PROCESSING IN A SPECIFIC SEQUENCE. DO YOU SEE THE DISTINCTION?] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ [MIKE3] - I see Monica's distinction, between how an MSH handles the message, and what the application does with it... however the spec does say: "A MSH that receives a message with a SequenceNumber element MUST NOT pass the message to an application until all the messages with a lower SequenceNumber have been passed to the application.... So the spec does speak of "passing to the application" in a sequence.. so my vote is to use Jacques recommendation. Comments? r2.2.4: re-wording: [precondition]: receiving a message M with MessageOrder element, and some messages for same conv ID have not been received yet or passed to application [assertion]: REQUIRED: MSH does not pass M to application. [MIKE] - DONE ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ [mm1: ON THIS ONE JACQUES, THIS IS CORRECT BECAUSE THE REQT SPECIFIES PASSING TO THE DESTINATION - CONVERSE TO 2.2.3] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ [MIKE3] - Agreed r2.2.5:re-wording: [precondition]: the MSH allows for specifying maximum of received out-of-order messages, and this maximum has been reached for a given conv ID. [assertion]: REQUIRED: MSH notifies sending MSH with delivery failure (with code...). [MIKE] - Spec does not infer that allowing for a maximum # of out-of-order messages is an option.. it infers that this is a requirement.. If it is an option, then it should be specified as such.. therefore the [precondition] is not a condition at all, in my opinion. It is also not testable as a condition. Reworded, but the condition is whether "max" unsequenced message count has been reached. r2.2.6a: re-wording: (consolidated with r2.2.7a) [precondition]: sending message with MessageOrder element, which is first for a conversation ID [assertion]: REQUIRED: the sequenceNumber element must have value 0, and its Status attribute = "Reset". [MIKE] - DONE ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ [mm1: IN THIS CASE (AND OTHERS), DO WE ACTUALLY HAVE TWO ASSERTIONS, JUST LIKE WE HAVE 0...N CONDITIONS/PRECONDITIONS - CAN WE ACCOMMODATE THIS - AND WITH A RESULT TO TRUE OR FALSE?] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ [MIKE3] - This is always the problem with "consolidating"... you "group" assertions ( and expected results ) together, so if there is a failure in one part of the Assertion, you fail the entire test.. So unless the test harness provides a way to uncover exactly which part fails ( the sequence number not equal to "0" or the status not equal to "reset" ) you cannot tell an implementer why their implementation failed, only that it failed one or the other Assertion. I am going to "decouple" the sequence number from the status, and split each test into two tests. It is less "pretty", but it removes any ambiguity. We have to be consistent on this, and I haven't been in all cases. r2.2.7b: could be consolidated with r2.2.6b. [MIKE] - DONE ( also consolidated with r2.2.6a ) r2.2.8: (rewording) [precondition]: sending message with MessageOrder element, [assertion]: REQUIRED: the sequenceNumber element has Status attribute = "Reset" or "Continue". [MIKE3] - I think that we can get away with this one because schema validation will catch either case for every message generated by the candidate MSH. This is really just a message validation test. [MIKE] - DONE ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ [mm1: SIMILAR QUESTION AS 2.2.6a - FOR CHOICE ON ASSERTION AND YET TWO ASSERTION...IF NOT X THEN Y?] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ [MIKE3] - This one I feel is OK, only because this is a "schema validation" test. Not very specific to begin with, only testing that the value of an attribute fall into the enumerated range of either "Reset" or "Continue". This will be picked up in schema validation ( done for all messages generated by the candidate MSH ). If the test were more specific ( in terms of the preconditions ), then it would warrant "splitting" the test requirement into two separate requirements. r2.2.9: not correct as stated: (sounded like reset *must* be done if precondition is true.) [precondition]: when sending message with MessageOrder element, and Status attribute = "Reset" [assertion]: REQUIRED: all previous messages for this conversation must have been accounted for (Acks received). (Note: can only partially be verified... as I believe the reset is controlled by the MSH, not the app) [MIKE- - DONE r2.2.10: we could restate this using precondition: [precondition]: when sending message with MessageOrder element, [assertion]: REQUIRED: no SyncReply element is present. [MIKE] - This is covered in "merged" r2.2.2 [precondition]: when sending message with SyncReply element, [assertion]: REQUIRED: no MessageOrder element is present. [MIKE] - DONE r2.2.11: "RECOMMENDED" instead of OPTIONAL ("should" in spec) [MIKE] - DONE r2.3: assume all these test reqs are moved to Level 3, review later... [MIKE] - DONE r2.4.1:(the current wording makes not clear what is REQUIRED: reword using precondition) [precondition]: when sending message with one or more Signature elements, [assertion]: REQUIRED: Signature element(s) must be child of SOAP Header, must conform to XMLDSIG specification, and be namespaced for XMLDSIG. [MIKE] - DONE ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ [mm1: SAME QUESTION ON MULTIPLE ASSERTIONS.] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ [MIKE3] - Agreed. Split this into 3 test requirements. r2.4.4: maybe we can be more specific here too (here, merging with r2.4.5) (ref spec: 4.1.2.1 + part of 4.1.3) [precondition]: when sending message in the context of a conversation agreement (CPA, Transport section) that requires signature of the entire message, [assertion]: REQUIRED: Signature element must be present, and its SignedInfo element has a Reference element to SOAP envelope, which has a URI attribute value of "". [MIKE] - DONE The Reference element has a child Transforms element. Line 1063 in spec: "This RECOMMENDED value SHALL be supported by all compliant ebXML Message Service software implementations." need be tested: if CPA specifies this, we can test when sending: so r2.4.7 could be "CPA-adpated" like for r2.4.4 above. Or, we could write also the receiver version ("double test"): (that is a rewording of r2.4.9 actually) [precondition]: when receiving message with signature element using sig algo ..., [assertion]: REQUIRED: MSH is able to validate the Signature. NOTE: there should be also a test req for when signature is NOT valid. In that case, the message should not be passed to application, I assume: [precondition]: when receiving message with invalid signature element [assertion]: REQUIRED: MSH does not pass the message to the appl. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ [mm1: THIS ENTIRE SET RAISES THE QUESTION OF "CHAINING" MULTIPLE REQUIREMENTS AND TEST CASES - WHERE A SET OF PREDICTABLE AND EXPECTED EVENTS OCCUR THAT HAVE A RELATIONSHIP. MOVES PAST THE DISCRETE.] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ [MKE3] - My answer is, if these are individual, "standalone", discreet tests.. why can't an application take the test result report and form a more complex evaluation ( i.e. "post-processing" the test results to create a "chained" sequence of test case results that can be evaluated on a higher level)? That way our test suite is nothing more than a collection of requirements and testcases that can be grouped at a "higher level" ( such as profiling... or logical grouping and sequencing ) for evaluation "after the fact"? r2.4.5: use of precondition to be consistent with rest of test reqs, merge with 2.4.7: [precondition]: when sending message with one or more Signature elements, [assertion]: REQUIRED: Signature element(s) have a SignedInfo element that is well formed and contains CanonicalizationMethod, SignatureMethod and references to expected message parts. The SignatureMethod element has an Algorithm attribute. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ [mm1: SAME QUESTION ON MULTIPLE ASSERTIONS.] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ [MIKE3] - DONE ( moved to r2.4.5 ) r2.4.9: in a more testable form: [precondition]: when receiving message with signature element and signed with algorithm DSA-SHA1, [assertion]: REQUIRED: MSH should validate the signature and pass the message to the appl. [MIKE] - DONE r2.4.10: blieve it is REQUIRED, see Spec Line 1065 (I merged it with r2.4.4) [MIKE] - DONE r2.4.12: The "Transforms" element is mandatrory (REQUIRED) under References, and should also have some Transform children. [MIKE] - DONE [precondition]: when sending message with one or more Signature elements, [assertion]: REQUIRED: ... r2.4.13: we could use precondition. [MIKE] - DONE r2.4.14: would move that in Level 3, as multi-hop related. [MIKE] - DONE r2.4.16: would use precondition as in other test reqs. [MIKE] - DONE r2.4.18: [precondition]: when sending message with one or more Signature elements, and with transfer encoding, [assertion]: REQUIRED: ... [MIKE] - DONE r2.4.22: would mention the involvment of CPA: [precondition]: when sending message in the context of a conversation agreement (CPA, Transport section) that requires both signature and encryption of one of its parts, ... [MIKE] - DONE
My comments tagged: [MIKE2] --------------------------- NOTE1: it is useful to specify, for each test item, the context of its verification: Should it be verified: (1) for messages received by the candidate MSH? (2) for messages generated by the candidate MSH? (3) on the callback from application interface (e.g. some errors)? So I'll add this info for each test item. [Jacques]: one way to do that, is to use the "Condition" part of the tset req, which states what are the pre-conditions required for applying this test (more on that below) So for example, we could now have in a COndition part that was empty before: "for each generated message" , in case the test applies to generated messages. Or, we could have "for each received message", in case it applies to received messages. And if Condition used to contain a cond "C1", then well have: "for each received message that satisfies C1", etc. What do you think? .... [MIKE2] - I think that a better place for a "for each generated/received message" ( IF THERE IS NO EXISTING CONDITION ) would be in the Assertion. The <Condition> and <Assertion> elements are going to logically drive the test harness. Each <Condition> must have a corresponding <TestCase> to execute . Each <Assertion> must have a corresponding <TestCase> to execute. Including a "for each" in each "empty" <Condition> means that the test driver must evaluate that <Condition> by executing a corresponding <TestCase>.. for which there is none. The only TestCase that exists in the case of an "empty" <Condition> is the <TestCase> for the <Assertion>. So if you want to add this piece of information to make the requirement more understandable, a better place for it would be in the <Assertion>. I have no problems modifying the requirements to add this to the <Assertion>. Please see below for more explanation of this topic. r1.1.2:(generated M) can we detail a little more what that means? [MIKE] - I believe that the following excerpt from the SOAP With Attachmentts specification provides the requirements for validating SOAP MIME headers: The primary SOAP 1.1 message must be carried in the root body part of the Multipart/Related structure. The type parameter of the Multipart/Related media header will always equal text/xml. Referenced MIME parts must contain either a Content-ID MIME header structured in accordance with RFC 2045, or a Content-Location MIME header structured in accordance with RFC 2557. It is strongly recommended that the root part contain a Content-ID MIME header structured in accordance with RFC 2045, and that in addition to the required parameters for the Multipart/Related media type, the start parameter (optional in RFC 2387) always be present. I added 4 conditions against the above. [Jacques]: That is definitely more precise than before. a bit of editorial job still needed I think: i am still uncomfortable with the way we use condition/assertion, although we improved a lot: even though the distinction Condition / Assertion is not much important from a test execution viewpoint, I believe we could/should use it with the following meaning: [MIKE2]: Because of the way that we are designing the test harness, Condition/Assertion IS important from a test execution standpoint. A Condition <Clause> and its nested <Condition>s must be evaluated as true/false before the <Assertion> is evaluated true/false. Failure of the <Condition> or its enclosing <Clause> prohibits evaluation of the <Assertion>. Conditions are not just provided for human readability.. they must have a matching <TestCase> with real test data associated with them. A parser is going to traverse the test requirements logic tree and execute <TestCase>s based upon the structure of that tree. So if we supply a <Condition> where none exists, we must have a matching <TestCase> to test that condition. So putting something like "for all received messages" as a <Condition> does not really evaluate to a true/false.. it cannot stand alone as testable. A solution for making more readable ( and functional ) test <Conditions> and <Assertions> is: 1) If the requirement is a conditional assertion ( requires certain conditions to be met before the assertion can be evaluated ).. then put the "for each genereated/received message" at the beginning of the <Condition> 2) If the requirement is an unconditional assertion ( no <Condition> element is present ), put the "for each generated/received message" at the beginning of the <Assertion> How does this sound? =============================== [mm1:AGREE WITH THIS STRATEGY.] =============================== o it should help readers understand *in which case* the core of the test requirement applies, i.e. if the COndition is not satisfied, then we cannot say that the test req has been actually verified: just that no matching situation occurred. [MIKE2] - I agree. It would not be hard to modify the requirements using the solution I suggest above to better clarify the context of the requirement. o In case we cannot drive the test harness so that it generates such a situation that satisfies the "Condition" part of a test req, that means we cannot guarantee coverage of this test req (coverage = null). In case we can only generate a subset of the situations, coverage is "partial". [MIKE2] Agreed =============================== [mm1:AGREE WITH THIS STRATEGY.] =============================== [Jacques]: so to get back to our test req item: - in r1.1.2, I think the interpretation should be: [Condition]:"IF the message is multipart MIME (or in any case is NOT a pure SOAP msge) [assertion]: THEN it should satisfy SwA requirements" - So the COndition part (which maybe should be renamed "Pre-condition" to avoid ambiguity) tells that this test verification only applies to multipart messages (because straight SOAP messages are allowed when no payload...). - So the test req item could be: [precondition]: for each generated message, if is multipart MIME, OR is not enveloped like a pure SOAP ("text/xml") [assertion]: the message must then be SwA compliant, which means: (...all you listed previously in "Condition") - if all you listed in Condition part is enough to capture SwA, we don't need to restate what you had before in previous Assertion (as it is just a recap of SwA). - We might have to revisit the content of [Pre]condition/Assertion throughout the other test reqs (I'll try for some). [MIKE2] - Here is how we can restructure and reword this requirement: <Precondition>For each generated mesage, if it is multipart MIME or not text/xml</Precondition> <Assertion>The primary SOAP message is carried in the root body part, the "type" paramater of the Multipart/RElated media header is "text/xml" and MIME parts contain a CID MIME header or a Content-Location MIME header structed in accordance with RFC2557, and the "start" parameter is present.</Assertion> =============================== [mm1:THIS RAISES AN ISSUE THAT MAY BE APPLICABLE TO OTHER LEVELS - OF THE CHOICE (OR) OF PRECONDITIONS - SHOULD WE CONSIDER THIS COMPLEXITY?] =============================== [MIKE3] - My suggestion would be, if we are going to add this complexity, to do it at a "higher level", using a schema that incorporates logical evaluation of "grouped" test case results ( perhaps modifying the "profile" schema that Matt is implementing to incorporate logical grouping and evaluation of test results ). This is yet another layer of complexity for the test framework, and I do not know if there is support within the community for this... perhaps we should discuss this on the next conference call. r1,1,5: (generated M) [Jacques]: A way to state that in a "testable" way is: [precondition]: In case the mesage is NOT a multipart MIME [assertion]: it must have no payload. (Note 1: only "partial" testing can be done here, as we can't force an MSH to satisfy the pre-condition: we have no control on it.) (Note 2: this is also a "double test": according to spec 2.1.2, we should check for Received M as well: [precondition]: For each received message, if it is NOT a multipart and is a well-formed SOAP message without payload, [assertion]: The MSH should accept it.) [MIKE2] - Here's my coding to restructure this ( also "doubled" it): <Precondition>For each receive message, if the message is not multipart MIME AND the message has no payload</Precondition> <Assertion>The receiving MSH should accept it.</Assertion> =============================== [mm1:THE TWO 'AND' SHOULD BE SEPARATED INTO TWO-PRECONDITIONS CONSISTENT WITH LEVEL 2 REQUIREMENTS.] =============================== [MIKE3] - DONE <Precondition>For each generated message, if the message is not multipart MIME</Precondition> <Assertion>The message must have no payload.</Assertion> [Jacques]: r1.1.9: - The way this test req is stated is not wrong, but too restrictive: it does not cover cases where there are SOME payload elts that are identified in Manifest, but SOME are not... (in fact, that is stated in r1.1.10, but I believe r1.1.9 is actually completely subsumed by r1.1.10 as stated, i.e. if 10 is satisfied, then 9 always is...) It needs be rephrased: [MIKE2] - Section RS 2.1.4, paragraph #4 states that "The contents of each Payload Container MUST be identified in the ebXML Message Manifest element within the SOAP body." So, if I read this right, there would not be a case where SOME payload elts are identified in the Manifest, and SOME are not. r1.1.9 tests the case where there are no application payloads, and should therefore have no Manifest References. [MIKE2] On the other hand, r1.1.10 addresses the case where there are payloads present, and tests whether a matching Manifest Reference element is present. So I see both r1.1.9 and 1.1.10 as legitimate tests against the spec. [precondition]: for each generated Message M, if M has some payload containers [assertion]: each container content must be identified in Manifest. - we should state "for generated Message" (generated M) I really believe we should always add this "context", either as a new column, or as part of the "precondition" ("generated Message M has no payload identified in Manifest.) [MIKE2] - As stated above, I will modify all requirements to reflect this context either in the <Condition> or <Assertion> as appropriate. - now, that sums-up previous r1.1.9 and r1.1.10. But another point in 2.1.4 is: "If there is no application payload within the Message Package then a Payload Container MUST NOT be present." So here, I think we need to check that if there are payload containers, then they are not empty. [MIKE2] - Agreed.. will add this requirement r1.1.7: (received M) The "charset = UTF-x" MIME parameter can still - and should - be verified? [MIKE] - Designated test coverage as "partial".... will only veritfy that UTF-X is the value supplied for the MIME content-type r.1.1.11:(received M) Partial check can be done? (by falsification of a message: we put all possible other MIME headers in, and just see that the message is well received.) [MIKE] - Fair enough. Designated test coverage as "partial", based on the assumption that we will be putting in every possible ( but perhaps not all ) MIME headers. [Jacques]: We should interpret a little more what "ignore" means (spec 2.1.5) from a tset req viewpoint: I would say it means no error with code "Error" is generated, and behavior is same as if feature wasn't there. [MIKE2] - I'm a little concerned about the vaguery of this spec statement. It is certinly wide open to interpretation. This is another spec reference that should be brought to the attention of the MS spec writing team. In the meantime, we can check that no "Error" elements are generated... but our interpretation is not certain. We should perhaps "tag" these test requirements as our own interpretation, rather than give the impression that this is a specified requirement. =============================== [mm1:AGREE WITH THIS STRATEGY. PART OF THE VALUE OF OUR EFFORTS IS TO PROVIDE REGRESSION BACK TO THE SPECIFICATION.] =============================== [MIKE3] - I am glad to see that that an "Audit/Issues" list is being created to catch all of the problems that we are finding in interpreting the specification. This is one to add. r1.1.25: (received M) The rejection must be in "accordance with SOAP", I believe there should be a SOAP Error, that should then translate into an MSH error. So by falsification, we should be able to partially check this (i.e. just for one instance of a non-recognizable header.) [MIKE] - Agreed. r1.2.1: (generated M) Actual test is that the "From" and the "To" are present. (identifying the party can't be verified) [MIKE] - Agreed. Since we are now only checking if "From" and "To" are present, I removed this requirement because schema validation ( done for every message generated by the candidate ) will always verify whether both "From" and "To" are present. r1.2.3: (generated M) As worded, I believe we have no way to verify that... however other validation can be done: (uniqueness of type attr value, position of "role" element if any) [MIKE] - Agreed. Substituted partyId attribute "uniqueness" requirement for this requirement. r1.2.5: (received M) Note: this is the kind of test that can be "doubled", i.e. verified for both received and generated messages. In that case, we should make it two test req items: For rceived M: stated as you said (we SHOULD observe an error message) For generated M: (generated M) either the PartyId type attr is present, or if it is not, its value MUST be URI. [MIKE] - "Doubled" this requirement into a "send" and "receive" scenario. r1.2.7: (received M) Is the spec ref wrong (3.2.1)? it seems to refer to XLinks in my version. Anyway the scope of that test seems too broad - too high level - normally should be split in several test items. Can we ignore it and rely on more specific discrepancy checks as they show up later in the spec? [MIKE] - Agreed. Removed this requirement. CPPA/message discrepency tests will have to be done on a lower-level, individual basis. =============================== [mm1:HOW WILL THIS BE DONE?] =============================== I believe that a CPPA "template" will be used, and compared to message content using XPath statements. r1.2.9: (generated M) Only partial verification possible: this requirement actually involves the application behavior: this is a usage guideline for the app... if the app does not set the conv id right, no way can check that: could be another conversation. We may just verify that the conv id set by our test service is well reported in the msg? We should also add a falsification test (generated M, here) for uniqueness of convID within a CPAid, where the app (our test service) on top of candidate MSH try to send two messages with different convID but same CPAid. Normally, the candidate MSH should complain - report error to the app. [MIKE] - If tests are performed serially, then it would seem that this type of a test could be done. You assume that other messages will be sent simultaneously while testing is taking place. Why not require that this test suite be run standalone against candidate ebXML MS implementations? For now, I am following your recommendation and changing the test coverage to "partial" [Jacques]: no, even for a "standalone" conversation: I mean that convID may change under the control of an application, from one message to the other, and we are not authorized to see anything wrong in this... This is an application behavior. Unless we refer to the CPAId: the uniqueness of convID within the same CPAId: [precondition]: for generated messages , if several of them relate to same CPAId [assertion]: these messages should all have same conversation ID. [MIKE2] - Here is my coding of your suggestion: <Precondition>For all generated messages with the same CPAId</Precondition> <Assertion>The message contains the same ConversationId</Assertion> So again this depends on tha app side: can't cover that fully in a testbed. But if we assume an Error must be generated if that is not the case, then we can cover the following test req: [precondition]: for generated messages , if two of them relate to same CPAId but have different conv ID: [assertion]: the MSH should generate an error. (and the test could be doubled with a "Received" version.) =============================== [mm1:HAVE WE CONSIDERED AN OPTIONAL - PERHAPS MORE COLLABORATION BASED SCENARIO TO SHOW A SERIES OF TEST EVENTS THAT ARE LOGICALLY TIED TOGETHER - INFERRED ABOVE.] =============================== [MIKE3] - This will need to be discussed ( adding a higher level of complexity through test grouping ) [MIKE2] - Here is my coding for this: <Precondition>For all received messages with the same CPAId but different ConversationId</Precondition> <Assertion>An Error element element should be generated in the response message.</Assertion> r1.2.10: (generated M) Remind in test description that this is about the Service element. And also, this is a "double" test item: (generated M): "condition C must be satisfied." (received M): "if condition C is not satisfied, an error should/must be generated." [MIKE] - Done. r1.2.11: (received M) probably partial verification: we can only verify for some particular case of bad message we generate (falsification). [MIKE] - Done. =============================== [mm1:SOME OF THESE MAY BE CANDIDATES FOR INTEROPERABILITY WHERE WE HAVE HANDOFFS TO THE APPLICATIONS - WHAT DO YOU GUYS THINK --- THROUGH THE REFLECTOR, FOR EXAMPLE.] =============================== [MIKE3] - I think that this is an excellent "division point" between conformance and interoperability testing... if we cannot verify at the application level, then such a test might be suitable as an interop test. r1.2.14: (generated M) Only partial verification possible: it depends on the application to set the RefToMsgId properly. So we can simulate some unauthorized case, (but that is r1.2.15) by initiating a new conversation with our test service, that sets RefToMsgId (should not), and see an error. (yet, nothing is said about Error generated. Is there an implicit assumption throughout the spec that failed reqs generate errors?) [MIKE] - The spec is unclear as to what type of Error message is generated if the condition is not met ( because the previous message never existed, for example ). I do not find any "implied" assumption that all failed requests generate an error. This should be brought to the attention of the MS spec team as an item to be clarified. For now, I would mark this test coverage as "none", since we cannot write a test for this requirement. [Jacques]: I propose that, when the spec states MUST or SHALL, and does not specify what happens when that is not satisfied, that we interpret this as a RECOMMENDATION for an Error generation. This should be the case for r1.2.9. [MIKE2] - If we are going to treat such requirements as valid, we MSUT provide some kind of additional metadata tagging this requirement as not specified. We are already accumulating a list of spec ambiguities that we must keep track of. We can tag the <Assertion> or <Condition> as RECOMMENDED, as long as we tag the requirement as "interpreted" and not specified. =============================== [mm1:WE HAVE ALREADY SEEN WE NEED METADATA - AND THESE EXPLICIT CONDITIONS SHOULD BE CAPTURED.] =============================== [MIKE3] - This case is another one for the "Issues List". r1.2.18: (received M) The error msg should be returned to the "From" party. Other test items should be derived from (3.1.6.4) , like TimeToLive if present should have valid UTC value (and not local time...) [MIKE] - Correction made.. and an additional derived requirement for UTC created r1.2.19: (received M) Even though this req is in "MS Core features", it really relates to Reliable Messaging (MS additional features). We should handle this in Level 2. [MIKE] - This requirement, along with r1.2.20 were both moved to level 2 requirements r1.2.20: (generated M) (An example of the way r1.2.7 should be split into sub-items.) Yet another "double" test, if in case we send a "bad" message: should there be an error? [MIKE] - Moved to level 2 ( reliable messaging ), also "doubled" the requirement r1.2.24: (received M) Yet another double test: we should also add the (generated M) version of this test req item: For any generated messages from the MSH: "either the xlink:href does NOT contain a cid URI, or if it does, a MIME part with content-id must be present in payload container." [MIKE] - Done r1.4.5: (generated M) I believe we can test that if no error are reported in a message sent by MSH, then the ErrorList elt must not be present. [MIKE] - Done r1.4.6: (generated M) Only partial check, as it may be hard to observe this from candidate MSH: we would need to make it generate such a message with several error elements. [MIKE] - Done r1.4.9: (generated M) reword as: the Error code must be valid. (we can't check more...) [MIKE] - This is done in requirement r1.4.13... so this requirement will be removed. r1.4.12: (generated M) I believe we can at least check the XPointer well-formedness, and in case of payload-related error, this is not app-specific (all these errors are MSH level): we can send a message with a bad payload *container*. so that we can check if the error generated is OK. [MIKE] - Done. r1.4.18: (generated M) I believe its very hard to test that, as it is the case where an error message is bundled with a business message response. Some MSH may simply not support this feature. We may not have any control on that from our test service. So I would say no coverage. [MIKE] - Done. r1.5.1: (generated M) We can test if we got the business response (generated by our test service) as an HTTP response to the previous sending... so I would say coverage is OK (req level: RECOMMENDED, as SHOULD is the same). [MIKE] - Done. Note: spec 4.3.1 also report yet another case of CPA discrepancy that is not allowed: SyncReplyMode. Should be in a separate test req item. [MIKE] - Added this requirement, actually "doubled" it for generated and received
NOTE 0: most of my comments below are about rewording so that we get consistent wording with what we did in Level 1, and also making some test reqs more precise (closer to test cases). Technically, I realize that what we describe here (that we call Test Reqs), are called "Test Assertions" by others (see OASIS / NIST testing literature)... We fit quite well the definition they give - a test description between specification and test case - and our terminology (" assertions" field) is also in sync, so thats good!!! (we just added "precondition", as a way to specify the conditions under which the test need apply). NOTE 1: this Level 2 doc need be in-synch with Level 1 new format ("preconditions", contexts: received M / sending M) [MIKE] - DONE NOTE 2: the "name" field values need probably be renamed due to test req content changes below. I did not touch them below. Ex: r2.1.3 "PersistReliableMsg" must be narrowed. [MIKE] - DONE NOTE 3: When there is an assumed CPA particular set-up, a way to handle it is to mention it in "pre-condition" field. You did it for some, I generalized. (see r2.1.30 below) [MIKE] - DONE NOTE 4: You should use MS spec 2.0 as published on OASIS site: your section numbers are not in sync with it after Section 6. [MIKE] - DONE NOTE 5: my comments below stop just before reviewing r2.2.x (Ordering) Will be done later... ---------------------------------------------------------------------------- r2.1.1: Just re-formulating: (note: I'm surprised the spec uses MAY - optional - for resending behavior!! (line 1433) how can we be reliable if not? So in order to translate that in something testable, I am assuming here that MSH behavior is to resend, when using reliabilty - making this part of the test precondition. So if that is not the case, I'm considering this test simply does not apply... nothing to verify!!! ) [MIKE] - This looks like an oversight in the spec writing, and should be brought to the attention of the MS spec writers. [precondition]: sending M reliably, in case MSH behavior is to resend (per CPA contract?), in case an Ack is not received. [assertion]: REQUIRED: the MSH resends M as many times as needed, as specified in "retries", at the expected time interval. [precondition]: sending M reliably, in case MSH behavior is to resend, if an Ack is received before all retries are expired. [assertion]: REQUIRED: the MSH stops resending M. (two subcases) [MIKE] - DONE r2.1.2: [precondition]: sending M reliably, in case MSH behavior is to resend, in case no Ack is received at all. [assertion]: REQUIRED: the MSH resends M as many times as specified in "retries", then sends back delivery failure notice to application. [MIKE] - DONE r2.1.3: I believe the general "persistent" req should be split in subcases, as you are right it cannot be tested as general statement - but some sub cases may be: we should split in 2 cases illustrating failures that require persistent store: [precondition]: sending M, in case transport connection is broken, then reestablished within the time-to-live value of the message. [assertion]: REQUIRED: the MSH should resend the message as expected. [precondition]: sending M reliably, then in case no Ack is sent back, and MSH system is shutdown, then if MSH restarted within the time-to-live value of M. [assertion]: REQUIRED: the MSH should resend the message as expected, totaling exact number of retries. In addition, we can still restate the overall "persistent" requirement asyou did, but with a "partial" only coverage, as it is hard to create failures that would demonstrate right behavior in all cases, especially for recived messages. [MIKE] - DONE ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ [mm1: DO WE ALSO NEED TO BREAK INTO TWO SUBCASES - ONE FOR SEND AND ONE FOR RECEIVE?] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ [MIKE3] - Yes, DONE r2.1.4: I believe the spec ref is 6.1, not 7.1. (looks like all your refs are shifted: see official MS spec as published on OASIS MS TC site) [MIKE] - DONE r2.1.5: I think this test req is subsumed by future test reqs on duplicate elimination? [MIKE] - DONE r2.1.10: note that this is more a req for multi-hop, as this happens when several MSHs add such elts. Same remark for r2.1.9 and r2.1.11, that we should move in Level 3? [MIKE] - MOVED TO LEVEL 3 Here, At most, we can check that in single-hop situation that never happens (note: we could combine with r2.1.12): [precondition]: sending M reliably with an AckRequested element [assertion]: REQUIRED: only one AckRequested element is present. [MIKE] - DONE r2.1.12: [precondition]: sending M reliably with an AckRequested element [assertion]: REQUIRED: the AckRequested element must be targeted to the toParty (either no (default) SOAP actor, or SOAP actor must have same URI). [MIKE] - DONE ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ [mm1: DO WE BREAK THE NO SOAP ACTOR OR SOAP ACTOR INTO SEPARATE TEST SUBCASES?] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ [MIKE3] - DONE, moved "NextMSH" case into level 3 multi-hop tests r2.1.13: [precondition]: receiving M reliably, with an AckRequested element with Signed attribute set to "true") [assertion]: REQUIRED: MSH is sending a signed Ack. [precondition]: receiving M reliably, with an AckRequested element with Signed attribute set to "false") [assertion]: REQUIRED: MSH is sending a un-signed Ack. [MIKE] - DONE r2.1.18 (rewording) [precondition]: sending an Ack message unbundled with other (no payload) [assertion]: REQUIRED: Message should not contain AckRequested element. [MIKE] - DONE r2.1.19 (rewording) [precondition]: sending an Error message [assertion]: REQUIRED: Message should not contain AckRequested element. [MIKE] - DONE r2.1.21 (rewording) [precondition]: sending an message with an Ack element [assertion]: REQUIRED: .... UTC time ... [MIKE] - DONE r2.1.25: maybe we could combine with r2.1.13a ? ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ [mm1: COULD BE COMBINED OR COMPILED INTO A SUBSET OF SEQUENTIAL TEST STEPS TO MAKE UP A "COORDINATED" CASE??? - WE HAVEN'T GOTTEN INTO THAT BECAUSE WE HAVE BEEN ACTING RATHER DISCRETELY. PERHAPS AS WE MOVE UP IN THE COMPLEXITY OF INTERACTIVITY - IT BECOMES INTEROPERABILITY? IF I'M OFF OK.] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ [MIKE3] - As mentioned earlier, the need for coordinated test cases and how they would be implemented will have to be addressed by the IIC. [MIKE] - I recommend keeping them seperate, as they test two different assertions. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ [mm1: EITHER WAY.] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ r2.1.26:(rewording) [precondition]: sending an message with an Ack element [assertion]: REQUIRED: .... namespace... [MIKE] -DONE r2.1.29:(rewording) [precondition]: sending an Ack message with no payload and no Error elt [assertion]: REQUIRED: Service / Action should be... [MIKE] - DONE r2.1.30:(spec ref 6.4.1) subcases: [precondition]: sending M reliably, in the context of an agreement (CPA) that requires duplicate elimination (CPA DuplicateElimination = always). [assertion]: REQUIRED: the DuplicateElimination element is present in header of M. [MIKE] - DONE ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ [mm1: DO WE ALSO NEED TO BREAK INTO TWO SUBCASES - ONE FOR SEND AND ONE FOR RECEIVE?] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ [MIKE3] - I agree, "doubled" the tests for sending and receiving r2.1.31 is fine (minor rewording:) [precondition]: sending M reliably, in the context of an agreement (CPA) that forbids duplicate elimination (CPA DuplicateElimination = never). [assertion]: REQUIRED: the DuplicateElimination element is not present in header of M. [MIKE] - DONE missing: [precondition]: sending M reliably, in the context of an agreement (CPA) that is neutral about duplicate elimination (CPA DuplicateElimination = per message), in case the party requires duplicate elimination for M. [assertion]: REQUIRED: the DuplicateElimination element is present in header of M. [MIKE] - DONE r2.1.32: we may try to make it more "testable" (sounds too much like the spec itself) Two sub-cases involving persistent store: [precondition]: when receiving twice the same M reliably, with DuplicateElimination element. [assertion]: REQUIRED: the message is presented only once to the application. [precondition]: when receiving once M with DuplicateElimination element, shutting down MSH, restarting MSH, then receiving a second time the same message with same MEssageId. [assertion]: REQUIRED: the message is presented only once to the application. [MIKE] - DONE r2.1.33a: precondition should describe more precisely the context of test. [MIKE] - DONE r2.1.33b: this is the "double" of r2.1.31 and r2.1.30, on Receiving side: [precondition]: when receiving M reliably, in the context of an agreement (CPA) that forbids duplicate elimination (CPA DuplicateElimination = never), with a DuplicateElimination element in header. [assertion]: OPTIONAL: error .... is generated (same for "always and NO DuplicateElimination element ) [MIKE] - DONE r2.1.34: probabbly falls under revised r2.1.1 above. [MIKE] - DONE r2.1.35: can we express a more precise test req for that? [MIKE] - DONE ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ [mm1: SHOULD WE INTRODUCE A PRECONDITION ASSOCIATED WITH THE CPA CONSISTENT WITH HOW JACQUES HAS ASKED FOR IN PREVIOUS TEST REQUIREMENTS? I STILL A BIT MORE CLARITY HOW SEE SEE CPA WHETHER THIS IS A "PRECONDITION" OR SOMETHING ELSE (ANOTHER TEST CASE)? APPLIES TO ALL TEST REQUIREMENTS WHERE THIS APPLIES.] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ [MIKE3] - Added a precondition. r2.1.36: [precondition]: when sending M reliably, with TimeToLive element, [assertion]: the value of TimeToLive satisfies... [MIKE] - DONE r2.1.37: can we express a more precise test req for that? We can verify that by doing duplicate elimination: [precondition]: when receiving M reliably, with a PersistDuration value, and a DuplicateElimination element, and receiving later the same message M before PersistDuration expires. [assertion]: REQUIRED: the message is presented only once to the application. [MIKE] - DONE missing: [precondition]: when receiving M reliably, with a PersistDuration value, and a DuplicateElimination element, and receiving later the same message M before PersistDuration expires. [assertion]: REQUIRED: an Ack similar to first one is sent back to sender. [MIKE] - DONE r2.1.38: can only be partially verified: is stating uniqueness of MessageId. [MIKE] - DONE r2.1.40: use similar wording as r2.1.36 above. [MIKE] - DONE r2.1.41a: (rewording) [precondition]: when sending M , in context of an agreement (CPA) where SynCReplyMode is present and not "none" [assertion]: REQUIRED: the message has a SyncReply element. [MIKE] - DONE r2.1.41b: (rewording) [precondition]: when receiving M , in context of agreement (CPA) where SynCReplyMode is present and not "none", and when responding to M. [assertion]: REQUIRED: the response is returned on same HTTP connection. [MIKE] - DONE r2.1.43: probably belongs to Level 3 (multi-hop) [MIKE] - DONE
Comments from Jacques: -------------------- remaining Level 2 comments ---------------------- r2.1.1.2: should be removed and merged with r2.1.1.1 , as it is redundant somehow (with r2.1.2): (and I'll try to remove this "optional" as well ...) [MIKE3] - The way I see it, r2.1.1.1 tests how a candidate behaves if it must retry sending until it succeds, whereas r2.1.1.2 tests how many times it attempts a retry, given a fail to deliver within the "maximum retry" value. I see these as two discreet tests. I do not see the "redundancy" here, except that both are attempting retries. But they are re-trying under two completely different scenarios. Also, section 6.4.3 of the MS specification states: "The Regries paramater, from the CPA, is an integer value specifying the maximum number of times a Sending MSH SHOULD attempt to redeliver an unacknowledged message using the same communication protocol". Based on this, I believe that the requirement should be changed to "RECOMMENDED". Comments? [MIKE3] - Agreed. Changed to "recommended". r2.1.1.1: [precondition]: sending M reliably, in case MSH behavior is to resend, if an Ack is received before all retries are expired. [assertion]: REQUIRED: the MSH stops resending M and behaves as if M successfully delivered. [MIKE3] This sounds like a new test requirement. Not like r2.1.1.1, which is testing if an MSH will continue to retry sending until it succeeds. This requirement is testing what an MSH will do AFTER it succeeds. I think this is a good test to add, as r2.1.1.3, keeping r2.1.1.1 as it is written, directly from the specification. Comments? r2.1.2: [precondition]: For each reliably generated message, if the MSH is configured to resend AND if the candidate MSH fails to receive any Acknowledgment meesage from a receiving MSH. [assertion]: REQUIRED: The candidate MSH sends successive retries at expected time intervals, then notifies From party of delivery failure. [MIKE3] - DONE. One thing that I would recommend, perhaps is avoiding the use of the word "candidate" in the test Assertions and Conditions, in favor or what the specification says, which is "Sending" or "Receiving" Party. I think that this better clarifies the test requirement ( it is cleaner, and more in line the the spec ) and also distances the test requirements from the underlying testing framework. Comments? r2.1.14: should be split for each subcase: r2.1.14a: [precondition]: For each received message with an AckRequested with the Signed attribute set to True if the Receiving MSH supports signed acknowledgment messages [assertion]: REQUIRED: the MSH sends back a signed acknowledgment. [MIKE3] - DONE r2.1.14b: [precondition]: For each received message with an AckRequested with the Signed attribute set to True consistent with CPA, and if the Receiving MSH does NOT supports signed acknowledgment messages [assertion]: REQUIRED: the MSH generates an Error of type Inconsistent, and severity = Error . [MIKE3] - DONE r2.1.14c: [precondition]: For each received message with an AckRequested with the Signed attribute set to True NOT consistent with CPA, and if the Receiving MSH does NOT supports signed acknowledgment messages [assertion]: REQUIRED: the MSH generates an Error of type Inconsistent, and severity = Warning . [MIKE3] - DONE r2.1.14d: [precondition]: For each received message with an AckRequested with the Signed attribute set to False if the Receiving MSH supports unsigned acknowledgment messages [assertion]: REQUIRED: the MSH sends back an unsigned acknowledgment. [MIKE3] - DONE r2.1.33.X: all these should have RECOMMENDED (as the spec says SHOULD) NOTE: I believe in several places, OPTIONAL should be converted in RECOMMENDED (e.g. r2.1.38, etc.) [MIKE3] - DONE ---------------------------- Level 3 comments --------------------------- r3.1.1: spec ref should be 7, not 8. (need be changed in all other reqs) [assertion]: RECOMMENDED:A Status Response Message is returned, with an Action as StatusResponse. [MIKE3] - DONE r3.1.4: RECOMMENDED [MIKE3] - DONE r3.1.5: (rewording) [precondition]: for each Status Request messages generated, i.e. with an Action element = StatusRequest. [assertion]: REQUIRED: the message consists of no payload and the MessageHeader/StatusRequest elements ...etc. [MIKE3] - DONE r3.1.7: coverage is probably "partial", as it is not clear that we can generate unauthorized calls to the MSH (not specified what it means). (and thsi coverage will now be associated with section 7.1.3 in spec). [MIKE3] - DONE r3.1.9: RECOMMENDED only (see 7.1.2). In fact, although the production of a StatusResponse is NEVER mandatory, when it occurs we should require that all fields are well formed and conforming. One way to do that is to include the generation of StatusResponse in precondition. That is what you want to verify here it seems, not that the response is actually generated (which is handled by r3.1.1, the precond of which is similar or almost). So I suggest: - we let r3.1.1 verify the (recommended) generation of a response, without more checks. - we transfor r3.1.9: [precondition]: For each received message, if the message contains a StatusRequest element AND If the StatusRequest child RefToMessageId element value is recognized by the MSH AND If the message is received from an party deemed to be authorised AND if a Status Response is generated for this Request. [assertion]: REQUIRED: the Response message is consistent with all reqs as in (7.3), and has a RefToMessageId consistent with the Status Request message. [MIKE3] - DONE r3.1.10.1: shouldn't we merge aslo with r3.1.9 ? as the preconds are pretty much the same and the assertions could be consolidated or mayn actually be included in r3.1.9 ? [MIKE3] - I would recommend not to. Merging many tests into one does not help clarify whether an implementation conforms, and often clouds the issue. It is good to be able to "isolate" a particular conformance problem. If the two tests are "merged", the test then can fail for a multitude of reasons. Using "atomic" level tests allows us to pinpoint particular portions of the spec that an implementation is being tested against. Grouping tests together for convenience is not always a good thing, particularly in the case where the tests are actually quite different. r3.1.9 is testing "well formedness" of a StatusResponse, and messageId integrity. r3.1.10 is testing Timestamp integrity, UTC integrity..etc.. so I would suggest not to do this. Comments? r3.1.10.2 - r3.1.10.3: we should use same trick: [precondition]: ...AND if a Status Response is generated for this Request. Only then can we REQUIRE the assertion... [MIKE3] - Again, I see these as two different tests. r3.1.10.2 is a case of "not recognized but authorized"... r3.1.10.3 is a case of "recognized but not authorized"... Both have the same result, but the preconditions are completely different for each case. They both have the same test resutl ( Timestamp not present in returned message ). But how do you evaluate the "merged" test? Was it precondition A true? or was precondition B true? or were both A and B true? And consequently, how do we know the candidate MSH is conformant in both cases? Using simple, atomic tests uncomplicates this issue, in my opinion. Comments? r3.1.12: can't relate to the spec (section 7.1.3?). [MIKE3] - Agreed, removed this requirement. r3.1.13: mostly rewording, but also necessary to have a REQUIRED assertion: [precondition]: ( For each received message, if the message contains a StatusRequest element AND If the the StatusRequest child RefToMessageId element value is recognized <---- remove that one, redundant with (b)? AND If the message is received from an party deemed to be authorised AND If the message identified by the RefToMessageId element <---------------------------- (b) in the StatusRequest element has been received by the MSH AND if a Status Response is generated for this Request) <------------------------- added [assertion]: REQUIRED: messageStatus in StatusResponse has a value 'Received' [MIKE3] - Agreed, removed redundancy, DONE r3.1.14 and r3.1.15: same rework as above. [MIKE3] - Agreed, DONE r3.2.2: OPTIONAL --> RECOMMENDED also: "...an errorCode of "NotSupported" and a highestSeverity attribute set to Error." r3.2.4: this "optional" will spoil the testing... [MIKE3] - I do not believe that changing the test requirement levels to suit our testing purposes is a good idea. We have to have integrity in how we define our testing requirements. If a feature is optional, or recommended, then we have to acknowledge that in the test requirement. Changing the Assertion to a status of "required" means all implementations must pass this test to be conformant, and that is not true. Perhaps a better way to do this is by "filtering" our tests that are run ( by profile or level as Matt is recommending ). This would eliminate "spoiling " test results by including tests against features that are not implemented. Any tests that fail for "optional" or "recommended" assertions would be flagged as failed tests, but would be qualified in the test report for further examination, and possible removal from the test suite. Ping and Pong tests could be filtered based upon a tester's profile. if you haven't implemented the Pong service action, set up your testing profile not to run those tests! That way we would not need to modify a testing requirement to meet our testing needs. (otherwise, there is no way to validate if the Ping service really works when every party agrees it should!) [MIKE3] - Again, I'd suggest to "filter" the tests before running ( or not running ) them. Comments? [precondition]: For each received message, if (a) the message header Action element contains a child Ping element AND (b) The requested service is supported AND (c) the Ping request is occurring under normally expected conditions (unlike those described in 8.3). [assertion]: REQUIRED: <same as before> [MIKE3] - DONE, changed wording. NOTE: when there are such "assumptions" (same for r2.1.2) maybe we can make a list of these, ask the testee to confirm his/her MSH should behave that way for the tests. and make that part of our CPA-like MSH configueration. I.e. in case the testee cannot/will not confirm an assumption, then the Test Reqs that use it as pre-condition will not be satisfied and the Test Case will simply not apply. [MIKE3] - I think that perhaps we are talking about the same thing here, "filtering" the tests based upon a testee's "profile". r3.3.4: we should move some of the assertion in precond: [precondition]: For each generated multi-hop message, with a SOAP actor attribute value targetting the NextMSH, if reliability is used [assertion]: there will not be two AckRequested elements in the message . [MIKE3] - DONE r3.3.8: reword: (note the multi-hop MSH MUST differentiate the Ack requests, as in section 10.1.3) [precondition]: ( For each received multi-hop message, if it contains two AckRequested elements where one addresses NextMSH and another addresses ToPartyMSH, AND the receiver - candidate - MSH is the ToParty MSH. ) [assertion]: REQUIRED: the candidate MSH is in the combined role of Next and ToParty MSH, and will send two Acknowledgments. [MIKE3] - DONE r3.3.9: use precondition: [precondition]: For each multi-hop message received reliably, if the candidate MSH node is in the role of intermediary, [assertion]: REQUIRED: the message is not acknowledged until both persisted and delivered to the Next MSH. [MIKE3] - DONE r3.3.14 and r3.3.15: use precondition too. (in fact, to be consistent, every req item should have a precond, as it relates to either sending or receiving a multi-hop message). [MIKE3] - Jacques, I've come to agree on this point. All requirements should have some precondition. I have modified all level 1,2 and 3 requirements to reflect that. Each precondition is testable, so each Assertion has integrity.
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [Elist Home]
Powered by eList eXpress LLC