[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: RE: [ebxml-msg] Bundling concerns, continued.
From: Moberg Dale [mailto:email@example.com]
Sent: 01 October 2009 17:54
To: Pim van der Eijk; firstname.lastname@example.org
Subject: RE: [ebxml-msg] Bundling concerns, continued.
The different processing in the bundling case is contrasted with individual message: When the message is not bundled, its P-mode as determined by its user message metadata is used. In bundling, the processing ignores how the message would be processed by its P-mode retrieved by its metadata, but must instead apply a different retrieval method for P-mode, and different processing. It makes a difference in what is done whether it is in a bundle or not. Even the presence of metadata that would govern processing in one case, has to be ignored when bundling. That violates Modularity, which I would like to see work. So I am not a fan of the RootConvention at all, even in its current lexical ordering version.
It is not one Pmode when bundled, and another when unbundled. The current draft says: "A message bundle is processed based on the processing mode of the primary message unit for the following processing mode sections: Pmode.Protocol, Pmode.ErrorHandling, Pmode.Reliability, and Pmode.Security with the exception of parameters for non-repudiation of receipt." The effect is that the Pmode of the primary message is used in the security and reliability modules, and the Pmode of the user message itself in the ebMS module. (Receipts are different in that they are in the security section in the PMode, but are not handled in the regular WS-Security module). So Service/Action of the user message are set based on that user message's Pmode. But WS-Security is configured by the primary unit's security Pmode.
MSHDiscretion means to me that bundling is just supposed to be a transmission option, and if it were that, then a fallback to an alternative transmission mode would work, just as in ebMS 2 when two delivery channels were accepted for a given action. So that motivates trying to see how a fallback would work. It doesn’t. So I agree that we should not really think of bundling as a transmission option. If it fails, it is logged and manual resubmission is probably needed.
I don't get this. The MSH has submit() and deliver() operations for user message units. If you submit a sequence of related messages and one fails, it is also safer for the sending MSH to not (re)send the units that follow the units in error, just as the receiving MSH should not deliver (and the sending MSH should not just resend) units in a bundle with a dependency on a failed unit in that bundle. If there are no dependencies among units in a bundle, they can be processed as if they were all sent separately.
I think it is still a very complex proposal. I will study the sections you mention before the next meeting to see whether they simplify but I am hoping we can reduce complexity and not add features and checks and processing overhead.
The intention was to make the
spec easy to implement, and unbundling in particular should require little
changes to an existing ebMS3 product (assuming it has a reasonably
modular/layered architecture). The transport, security and
reliability modules should be reusable with little or no changes. Some
features may seem complex, but e.g. the ordering and partitioning
features really just add a few lines of program code/Xpaths for
sorting/grouping-by. Also, decisions on what to bundle and which
units impacts which other units in cases of failure is fully driven by
configuration rather than depending on any intelligence in the
The intention was to make the spec easy to implement, and unbundling in particular should require little changes to an existing ebMS3 product (assuming it has a reasonably modular/layered architecture). The transport, security and reliability modules should be reusable with little or no changes. Some features may seem complex, but e.g. the ordering and partitioning features really just add a few lines of program code/Xpaths for sorting/grouping-by. Also, decisions on what to bundle and which units impacts which other units in cases of failure is fully driven by configuration rather than depending on any intelligence in the MSH.
From: Pim van
der Eijk [mailto:email@example.com]
There are several design principles and conventions at work in our current approach to bundling that don’t quite hang together cleanly -- or so it seems to me. Let me describe some of the major ideas and then discuss some of the tensions in getting them to work together.
Modularity: Each user message should behave in a bundle just as it would by being sent on its own.
Partly true. For routing, transport and security (except receipts) and reliability the secondary units are being processed like the primary unit.
MSHDiscretion: A MSH should be free to be statically or dynamically configured to send messages in a bundle or individually.
RootConvention: The P-Mode of the SOAP body user message (href attribute omitted in PartInfo) is primary.
No, the primary message is the first eb3:UserMessage in the eb3:Messaging container, which is a SOAP header. It doesn't matter where the payloads of that primary message are stored (in the SOAP Body, as an attachment). And perhaps the eb3:UserMessage doesn't even have payloads, as the minOccurs of eb3:Payloadinfo is 0.
SingleSignature: When a signed message is indicated by the P-Mode of the Root the signature must cover all user messages.
The PMode of the primary unit specifies what (if anything) is signed. If it says: sign the eb3:Messaging unit, the SOAP Body, any other SOAP headers per WS-I RSP, and all payloads, then the effect is that all user messages are signed.
OneSharedReliabilityId (sequence number in WS-RX)
Tensions in Modularity Root Convention, and Single Signature
Suppose one non-root body part has a P-mode with enveloped encryption. The assembly of parts is to proceed as it would for individually sent messages even when in a bundle. But the P-mode of the root is signed only. Should the non-root part be encrypted and then signed? Or should encryption be omitted and the receiving end indicate a Value-Inconsistent error? See section 3.2.2 of the latest draft. Whether or not to encrypt payloads is determined by the primary unit's PMode, since this is specified in the Pmode.Security PMode section. This is true for both sending and receiving MSH, so there is no Value Inconsistency error. Or when bundled should each non-root part be processed by ignoring the defined P-mode for this user message?
On the face of it Modularity can easily conflict with RootConvention when SingleSignature is operative. In fact whenever the root P-mode conflicts with a P-mode of a non-root user-message part, then the normal message receiver-side error checks must be disengaged. Likewise I think during sender message assembly, the access to the P-mode associated with the user message’s AgreementRef will need to be overridden. This is a lot of special coding and almost amounts to a new mode of operation. Bundling is functionality for the ebMS module, the security and reliability modules don't need to know much if anything about it. Suppose you have an existing ebMS3 implementation that you want to adapt to support bundling. The only thing is to check that it picks the right (=first) UserMessage for PMode setting (agreement) validation, since the only new thing is that there may be more of them. The security and reliability modules just use a different key to the agreement/pmode store than the ebMS module. Please explain why this would be a lot of special coding.
So I agree with Sander that it might be better to restrict user message bundling to messages with the “same” P-mode, assuming we can explain what that means. I doubt that this constraint will be sufficient to solve all the tensions, however.
If trading partners want to restrict bundling this way, then the parameters in section 3.3 give them the configuration parameters they need to do so. But if partner P1 exchanges 30 types of messages with a trading partner P2 (hence 30 PModes), it may be perfectly acceptable to use a single set of security/reliability parameters for all 30 message types and to allow all of them to be bundled together, and this will increase the benefit on bundling. Many current ebMS user communities use configurations based on a handful of profiles (or CPA templates). If they exchange messages regularly, they immediately benefit from bundling.
Tensions in MSHDiscretion and OneSharedReliabilityId
Here is a case I was struggling with describing today. Supposedly the MSH can “just decide” to bundle messages together. And if the bundle does not work for some reason (errors come back about message parts) it could decide to “back off” and send messages separately. In the case of a signature failure, I think we agree that it might be reasonable to see whether each part could be sent individually. Presumably this makes sense because no part “should” be delivered, given that we have SingleSignature.
This is certainly an option, but in practice I suspect some system manager will want to inspect this message and see what went wrong, instead of having a policy to automatically sending the units unbundled. We could define a PMode parameter to do this, or leave it as an implementation choice.
If the error is due to e.g. the use of an untrusted certificate, then the primary unit will also be rejected when sent unbundled. There could be a benefit in sending a secondary unit whose PMode does not specify signing (or references another -trusted- certificate) unbundled, since they would not suffer from the certificate errors of the primary unit's PMode. But this doesn't help if, as you and Sander want, only units that have the same PMode are bundled: then you are certain that they will all fail when sent unbundled, since the PMode specifies the certificate and it is the same for all. So then, when a bundle fails, all units in it fail permanently.
If there is an error in calculating the hash for some cid: MIME attachments and not for other, this looks like a product error that would probably also occur with standalone units. Resending the units is unlikely to solve anything .
Sending each part individually would at least be preferable to sending the same bundle again. This is because the message processing failed before the bundle was acked, and so retrying the same bundle with its security defect could go into an endless loop of retries as long as acks might not be sent.
Yes, just as you presumably do not resend a failed unbundled message unit.
Now suppose for a different bundle that its security checks out, and reliability is processed, and we arrive at the separate message parts. Resending the whole bundle would probably just be acked as a duplicate, so that will not be an option for a transparent recovery. So if we do get error messages, should we trust that the parts for which we did not receive an error message were ok and delivered? What happens when you send an unbundled user message unit, you get an RM acknowledgement and no error. Why would you not question whether it was delivered in that case, whereas you have doubts about it being delivered if it is sent in a bundle? You can specify that you want an NRR receipt. Those are specified per message unit, so you would get separate eb:Receipts for each user message unit that didn't fail. What if (some of ) the error messages get lost? What if an error for an unbundled message is lost? With WS-ReliableMessaging, an acknowledgment does not mean that no error occurred, even when using InOrder delivery. There is no difference when bundling applies, so why is this raised as an issue with bundling? Not sure how to specify it in PModes, but an MSH could send errors back over a reliable sequence, to make sure they don't get lost.Or to be on the safe side should we send all the parts again, individually, adhering to MSHDiscretion and the hope that a more determinate outcome of errors and/or acks comes back? If our received error messages were infallible indicators of exactly what went wrong, then we might try to send each payload with an error and see whether that remedies the error message. Otherwise, we retry all the body parts separately. The problem here is that if some of the parts had been ok and had been delivered optimistically, then retrying individually will get duplicates delivered. Yes, that's why I think resending a message unit that you did not get an error for separately, if you sent them before in a bundle that you got an acknowledgment for, is a questionable idea. You have to build a second level of duplicate elimination at the ebMS level that you otherwise wouldn't need. And that violates our modularity assumption because when each part has its own reliability id value, duplicates would be eliminated. It could be that we just give up and do nothing about these ebMS level errors, just logging them. I think the receiving MSH should generate errors for the units that have errors and no errors (or receipts, if NRR is requested) for the units that are fine at ebMS level. We could mandata the errors for units are always bundled, so that the sending MSH receives them together, providing strong evidence that some units failed but others not (since that bundle would contain explicit errors for those if they were). But that would violate modularity too, because in that case, we would not know that any of our parts got delivered successfully as we could with an acked, errorless single message sent individually. Not true, with WS-ReliableMessaging, an acknowledgment does not mean that no error occurred, even when using InOrder delivery, also with unbundled messages. Being part of the bundle has made our state alignment a bit foggier in the case of some errors at the ebMS header level. Or so it seems to me. Have a look at 3.4 and 3.9.3.
Let me stop with these two issues, and ask for simplifications that would diminish the interactions threatening Modularity. I think that Sander had a good suggestion to disallow P-modes that were “different” but I would like a cleaner way of stating this requirement. I personally would not mind dropping the single signature idea because that definitely makes sending a message in a bundle different from sending it on its own. I think that if signatures are per unit, can use different certificates or algorithms, that some units use encryption and others not we are loosing the biggest efficiency benefit for bundling. In that case, I would not use bundling at all. Or define a batching protocol (multiple SOAP envelopes in distinct MIME parts) if saving a separate HTTP connection set-up is worth the effort. With batching the various SOAP envelopes could also have separate reliabilty sequence numbers.
(One security header can have many separate signatures within it, although I need to check WSI BSP to see that they have not profiled that away.)
As far as ebMS level errors go, either we should state that nothing is to be done in response to an error by the sender as part of our specified protocol. Or we might state that each part is sent again individually (but I think this is bad because the receiver will have to support a second layer of duplicate detection by ebMS message id). Or we state that a receiver MAY retry a message with an error received when it was sent in a bundle. My preference is now headed toward saying that the specified protocol does not say what to do when an ebMS error is received for one part of a bundled message. Maybe we even say the same thing about the failed-signature error message when received by the sender. Please review 3.4 and 3.9.3.
I do not think that going into the dependency graph direction is going to make this situation simpler or better. I think that we should say that when choosing bundling, that is the mode of transmission, and when it fails, each message payload must be resubmitted by operations (but with a do-not-bundle flag set! ) Please review 3.4 and 3.9.3.
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]