[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [Elist Home]
Subject: Re: reliable messaging - hop by hop
Guys,
Reading through this discussion on RM reinforces the idea that we haven't
really addressed the issue from a layering point of vew. I remember many
happy :-) hours spent at the whiteboard actively discussing this over the
last 18 months. We need to really understand what is RM and what is a
reliable business process. I offer this as a starting point:
1. Reliable messaging over a single hop ensures that a message (thought of
as an opaque buffer) is delivered once and once only from a sender to a
receiver. This corresponds to delivery from a sending MSH to a receiving
MSH with no intermediaries in the ebXML context. This is similar to the
MQSeries and HTTPR approach - somple point-to-point reliable delivery. Note
that there is no talk about Acknowledgement receipts, NRR or whatever -
this is simply getting a buffer from one point to another.
2. Sitting above this single hop is a distributed layer which handles the
intermediaries. This is where the routing, LDAP (or whatever directories
are needed), SOAP-RP is used. This layer relies on the lower, reliable
delivery, single hop capabilities but inspects attributes of the message
(mainly the ultimate recipient) to route the message. I believe from a
performance point of view, the message must be as opaque as possible for
routing to be as efficient as possible. Decrypting, encrypting and
signatures should be kept out of this layer.
3. We then get to the end-points - the applications that send and receive
the message. This is where the business process sits and I view things like
NRR and anything to do with signatures as sitting here. Anything in the
lower layers may alter the syntax of the message but it is only at the
application level that the semantics (meaning) of the message can be
altered.
To answer David's question on MQSeries, it does not issue NRRs in the sense
that has been discussed. There are programming exits that can be used to
add those features, but the base product does not support it. It provides
the services described in layers 1 and 2 above.
Comments ?
John
XML Technology and Messaging,
IBM UK Ltd, Hursley Park,
Winchester, SO21 2JN
Tel: (work) +44 (0)1962 815188 (home) +44 (0)1722 781271
Fax: +44 (0)1962 816898
Notes Id: John Ibbotson/UK/IBM
email: john_ibbotson@uk.ibm.com
christopher
ferris To: ebxml-cppa@lists.oasis-open.org,
<chris.ferris@eas ebxml-msg@lists.oasis-open.org
t.sun.com> cc:
Subject: Re: reliable messaging - hop by hop
08/29/2001 02:54
PM
Please respond to
christopher
ferris
David,
A NRR *may* not be meaningful in a legal sense unless it is
signed by an authorized signer. The private key for the certificate
that is authorized *may* not be available to the MSH for security
reasons.
This is why it is my strong belief that NRR is an "application"
function. It does not seem reasonable to me to impose a requirement
that the private keys that carry a weight of authority be made
available to an MSH.
This is not to say that an MSH does not have certificates that
can be used to sign and/or encrypt message traffic. It is just that
these certificates are potentially more vulnerable and hence they
may only be accorded marginal trust (they may be used to sign and
encrypt data such that message traffic confidentiality and integrity
is preserved, but they cannot necessarily be used for authentication
of anything other than the software that transmits the message).
NRR is most assuredly an important aspect of electronic business
message exchange. Of that there is no doubt. However, its function
is somewhat orthogonal to a reliable messaging protocol.
Cheers,
Chris
David Fischer wrote:
>
> I guess I take more of the end view. I need an Acknowledgement/Receipt
from my
> trading partner at the other end saying they received the message (not
> application processing). Often, I need it to have a MIC and be signed by
the
> end for legal reasons (NRR). An Acknowledgement from some other
intermediary
> tells me nothing. How it gets there is of little or no consequence to me
(Black
> Box). If I don't get the Acknowledgement/Receipt then I need to be able
to send
> the exact message again and not have someone in the middle stop my retry
> attempt.
>
> However, if the folks that handle the intermediaries tell me they need
certain
> mechanisms to make this transfer happen, then I am happy to oblige --
provided I
> still get end-to-end NRR.
>
> Regards,
>
> David Fischer
> Drummond Group.
>
> -----Original Message-----
> From: John Ibbotson [mailto:john_ibbotson@uk.ibm.com]
> Sent: Tuesday, August 28, 2001 11:08 AM
> To: Martin W Sachs
> Cc: 'christopher ferris'; Burdett, David;
> ebxml-cppa@lists.oasis-open.org; ebxml-msg@lists.oasis-open.org; jacques
> durand
> Subject: RE: reliable messaging - hop by hop
>
> Hi Guys,
> My apologies for not responding in detail to all the RM discussions over
> the last few weeks, I'm catching up on >1000 emails after vacation :-)
>
> This whole topic of end-to-end versus single hop is one that will run and
> run. From the MQSeries perspective (which also influenced the HTTP-R
> specification), reliability is from point-to-point single hop. The HTTP-R
> specification gives an indication of the interchanges between the end
> points to give the reliable delivery - MQSeries does a similar thing but
in
> a proprietary way. To achieve the end-to-end delivery, MQSeries has the
> concept of a Queue Manager whose role is to coordinate queues and their
> message persistence management at intermediate nodes thereby providing
> end-to-end reliability but implemented as a set of single hops.
>
> The sending application PUTs the mesage to a queue. The receiving
> application GETs the message from its input queue. The reliable delivery
is
> within the middleware but there is the advantage that an API is defined
> between the endpoint applications and MQSeries. One example is the time
to
> live mechanism. This assigns a time to live to a message. Once this time
> expires, then the message is purged from the wherever in the MQSeries
> infrastructure it has ended up. This can include it sitting on a queue
> waiting for the receiving application to Get it. The sending queue
manager
> is informed of the purging and can ultimately inform the sending
> application.
>
> I'll try and catch up with the other discussions.
> John
>
> XML Technology and Messaging,
> IBM UK Ltd, Hursley Park,
> Winchester, SO21 2JN
>
> Tel: (work) +44 (0)1962 815188 (home) +44 (0)1722 781271
> Fax: +44 (0)1962 816898
> Notes Id: John Ibbotson/UK/IBM
> email: john_ibbotson@uk.ibm.com
>
> Martin W Sachs
> <mwsachs@us.ib To: "Burdett, David"
> <david.burdett@commerceone.com>
> m.com> cc: jacques durand
> <jacques@savvion.com>, "'christopher ferris'"
> <chris.ferris@east.sun.com>,
> ebxml-msg@lists.oasis-open.org,
> 08/28/2001 ebxml-cppa@lists.oasis-open.org
> 03:09 PM Subject: RE: reliable
messaging -
> hop by hop
> Please respond
> to Martin W
> Sachs
>
> I am starting to look in detail at HTTPR. I am only through chapter 1 so
> far. It offers exactlyOnce semantics. HTTPR does reliable messaging hop
by
> hop. It explicitly states that it assumes that the intermediaries store
> and forward reliably. My point for mentioning this is, of course, that
if
> we can focus on stating in the specification what reliable messaging
> promises and what assumptions it makes on endpoint and intermediary
> implementations, we might converge a lot more quickly than we are doing.
>
> Regards,
> Marty
>
>
********************************************************************************
> *****
>
> Martin W. Sachs
> IBM T. J. Watson Research Center
> P. O. B. 704
> Yorktown Hts, NY 10598
> 914-784-7287; IBM tie line 863-7287
> Notes address: Martin W Sachs/Watson/IBM
> Internet address: mwsachs @ us.ibm.com
>
********************************************************************************
> *****
>
> "Burdett, David" <david.burdett@commerceone.com> on 08/27/2001 09:52:00
PM
>
> To: Martin W Sachs/Watson/IBM@IBMUS
> cc: jacques durand <jacques@savvion.com>, "'christopher ferris'"
> <chris.ferris@east.sun.com>, ebxml-msg@lists.oasis-open.org,
> ebxml-cppa@lists.oasis-open.org
> Subject: RE: reliable messaging
>
> I don't think that we are that far apart the critical difference in view
is
> around the "requirement" that the From Application is informed of the
> delivery failure.
>
> For example, what should an implementation do if the application has no
> existing reasonable method of being notified of the delivery failure. In
> this case, one reasonable approach might be for the MSH to log the
failure
> and then provide a GUI which allows a user to browse the log and decide
> what
> to do. I think that if we use a "SHALL" we would preclude the second
> option.
>
> I think that if we take your view literally it would mean that if an
> application could not accept this type of notification then the
application
> MUST be changed before ebXML reliable messaging could be used? I don't
> think
> this is reasonable. Thoughts?
>
> I also think that there are two types of delivery notification:
> 1. The From MSH reporting delivery failure to the From Application, and
> 2. Another MSH (not the From MSH) reporting a delivery failure to the
From
> MSH.
> I think you are focusing on the first. I'm thinking of both.
>
> More detail below marked with <db></db>
>
> Best wishes.
>
> David
> -----Original Message-----
> From: Martin W Sachs [mailto:mwsachs@us.ibm.com]
> Sent: Monday, August 27, 2001 10:31 AM
> To: Burdett, David
> Cc: jacques durand; 'christopher ferris';
> ebxml-msg@lists.oasis-open.org; ebxml-cppa@lists.oasis-open.org
> Subject: RE: reliable messaging
>
> My replies below.
>
> Regards,
> Marty
>
>
****************************************************************************
>
> *********
>
> Martin W. Sachs
> IBM T. J. Watson Research Center
> P. O. B. 704
> Yorktown Hts, NY 10598
> 914-784-7287; IBM tie line 863-7287
> Notes address: Martin W Sachs/Watson/IBM
> Internet address: mwsachs @ us.ibm.com
>
****************************************************************************
>
> *********
>
> "Burdett, David" <david.burdett@commerceone.com> on 08/27/2001 12:55:22
PM
>
> To: Martin W Sachs/Watson/IBM@IBMUS, jacques durand
<jacques@savvion.com>
> cc: "'christopher ferris'" <chris.ferris@east.sun.com>,
> ebxml-msg@lists.oasis-open.org, ebxml-cppa@lists.oasis-open.org
> Subject: RE: reliable messaging
>
> Marty/Jacques
>
> I think I agree with both of you.
> 1. We need to make much stronger statements about the From Party MSH
> notifying the From Party Application that a message was not delivered.
> 2. We can't make it a "SHALL" or "MUST" as we have not specified the API
> and
> we can't check compliance.
>
> MWS: We can make it a SHALL and worry about defining the API later.
This
> is
> no different than every other function in the MS spec.
> <db>See discussion above.</db>
>
> So let's try and agree some words. How about the following for the
> replacement first and last paragraphs in section 10.4 ...
>
> Current first para (lines 1874-6) ...
> >>If a message sent with deliverySemantics set to OnceAndOnlyOnce cannot
be
> delivered, the MSH or process SHOULD send a delivery failure notification
> to
> the From Party. The delivery failure notification message contains: ...<<
>
> Revised first para ...
> >>A MSH that is not the From Party MSH might receive a message sent with
> deliverySemantics set to OnceAndOnlyOnce that it determines cannot be
> delivered to the To Party application or other process that is the final
> destination of the message. In this case, the MSH MUST send a delivery
> failure notification to the From Party that contains: ...<<
>
> MWS: This is not sufficient. A delivery failure notification sent over
> the
> network is inherently unreliable. The From MSH will learn, from failure
to
> receive an ACK after the specified number of retries, that the message
was
> not delivered. The From MSH can reliably send a delivery failure
> notification
> to the From application.
> <db>This is paragraph is saying what a MSH which *NOT* the From MSH
should
> do. I think your answer is saying what the From MSH should do which is
> covered later.</db>
>
> We should also add to the bulleted list ...
> >>. a deliverySemantics attribute set to OnceAndOnlyOnce<<
> ... as the message should be sent reliably.
>
> Current last para (lines 1886-9) ...
> >>It is possible that an error message with an Error element with an
> ErrorCode set to DeliveryFailure cannot be delivered successfully for
some
> reason. If this occurs, then the From party that is the ultimate
> destination
> for the error message SHOULD be informed of the problem by other means.
How
> this is done is outside the scope of this specification.<<
>
> Revised last para, its now two and has sub headings ...
> >>10.4.1 From Party MSH Behavior
> The From Party MSH that sent a message with deliverySemantics set to
> OnceAndOnlyOnce might determine that the message could not delivered. In
> this case it is strongly RECOMMENDED that the From Party MSH notify the
> application or other process that requested the message be sent of the
> delivery failure. This should indicate whether the failure was certain,
for
> example, there was a communications failure that meant the message could
> not
> be sent, or probable, for example, although the message was sent, no
> acknowledgement or delivery receipt was received.
>
> MWS: As indicated above, I do not agree to anything weaker than SHALL.
The
> word
> "probable" in the next to last line is too weak. Exhaustion of retries
> with no
> Acknowledgment is certain except for the unlikely case that the message
was
> delivered but the From party is continuously unable to receive ACKs. We
do
> need to think more about handling that case.
> <db>I was "thinking more about handling that case" which is why I came up
> with the wording. We also need to agree about the feasability of
> "requiring"
> a from MSH to notify the "application". One way around this is to
fromally
> define in the spec what is meant by the "application" and allow this to
> include real software applications, notifying appropriate users via log
(as
> described earlier). But that still leaves the use case where the
> implementer
> want's the message sent reliably but as it increases the chances of
success
> but doesn't want to know if it doesn't work. I know that this is not
> recommended, but sometimes implementers want to do things they really
> shouldn't and we *can't* stop them.</db>
>
> 10.4.2 Failure to deliver a DeliveryFailure message
> It is also possible that an MSH that sent an error message with an Error
> element with an ErrorCode set to DeliveryFailure determines that the
> message
> was not delivered successfully even though it was sent with
> deliverySemantics set to OnceAndOnlyOnce. If this occurs, then it is
> strongly RECOMMENDED that the party that is operating the MSH notifies
the
> From party that is the ultimate destination for the error message by
other
> means. How this is done is outside the scope of this specification.<<
>
> MWS: Same comments as above (10.4.1) apply. In addition, as others
pointed
> out, a message which is part of the RM protocol for another message
should
> not itself be sent reliably because that can lead to a never-ending
series
> of messages.
> <db>I think your concern is covered by the text above which says that if
> the
> sending of a delivery failure fails then solve the problem by other
> means.</db>
> As noted above, I do not agree to sending DeliveryFailure
> over the network. DeliveryFailure is an indication from the From MSH to
> the
> From application as a result of exhaustion of retries without receiving
an
> acknowlegment.
> <db>I think there is possibly a misunderstanding here. Consider the
> following use case:
> 1. An intermediate MSH receives a message and sends and acknowledgment
back
> to the "From MSH".
> 2. The intermediate MSH then determines that it cannot forward the
message
> to the "To MSH" as it is down.
>
> What should the intermediate MSH do the options as I see it are:
> 1. Do nothing. The From MSH will then deduce that the delivery failed as
no
> Delivery Receipt was received. Note that there is an edge case as
> previously
> disccussed that the From MSH must assume that there is a small
probability
> that it was delivered.
> 2. Send a "Delivery Failure" message to the From MSH. This means that the
> From MSH is then positively informed that the message was not delivered.
I
> prefer this option and it is what the current spec says.
>
> Neither of these options preclude the From MSH informing the From
> Application of the results of sending the message.
> </db>
>
> We also need to change lines 1849-53 in section 10.3.4 as this is now
> covered in section 10.4 (see above). We also could extend it to describe
> the
> idea of checking that MSHs are up and running. Currently these lines
> contain
> ...
> >>. If the Sending MSH does not receive an Acknowledgment Message after
the
> maximum number of retries, the Sending MSH SHOULD notify the application
> and/or system administrator function of the failure to receive an
> acknowledgement.
>
> MWS: SHALL
> <db> Marty!! This is what the current spec says!! I know you object to it
> ;)
> </db>
> . If the Sending MSH detects an unrecoverable communications protocol
error
> at the transport protocol level, the Sending MSH SHOULD resend the
> message.<<
>
> We could replace it with ...
> >>. If the Sending MSH does not receive an Acknowledgmemt Message after
the
> maximum number of retries then the Sending MSH SHOULD:
> a) Send a Message Service Handler Ping Message to the same MSH one or
> more times as the Sending MSH determines.
> b) If no Message Service Handler Pong Messages are received then the
> Sending MSH MUST carry out the Failed Message Delivery behavior as
> described
> in section 10.4<<
>
> MWS: If this ping/pong is new for me.
> <db>It's in version 1.0 of the spec.</db>
> If it is useful in resolving doubt,
> the SHOULD probably should be changed to SHALL. However given that the
> maximum number of retries has failed, it isn't obvious what value there
is
> to
> performing ping/pong. If there is value to it, the (b) need also to
state
> what
> do do if ping/poing succeeds.
> <db>Good point. If the ping succeeds then they should retry sending the
> message again. Agreed?</db>
> Thoughts?
>
> David
> -----Original Message-----
> From: Martin W Sachs [mailto:mwsachs@us.ibm.com]
> Sent: Sunday, August 26, 2001 2:44 PM
> To: jacques durand
> Cc: Burdett, David; 'christopher ferris';
> ebxml-msg@lists.oasis-open.org; ebxml-cppa@lists.oasis-open.org
> Subject: Re: reliable messaging
>
> Jacques,
>
> I have to disagree. With or without an API definition, the
> reliable-messaging must include the sending application, receiving
> application, and both MSHs. A contract that is just between the MSHs is
> worthless because the beneficiaries of the contract are the From and To
> applications. We can state the essentials of the contract and the
> assumptions on the implementations with or without an API definition.
Once
> we have the API definition, we can go back and improve the description of
> the contract and the assumptions on the implementations to take the API
> into account but the assumptions and contract do not change.
>
> Regards,
> Marty
>
>
****************************************************************************
>
> *********
>
> Martin W. Sachs
> IBM T. J. Watson Research Center
> P. O. B. 704
> Yorktown Hts, NY 10598
> 914-784-7287; IBM tie line 863-7287
> Notes address: Martin W Sachs/Watson/IBM
> Internet address: mwsachs @ us.ibm.com
>
****************************************************************************
>
> *********
>
> jacques durand <jacques@savvion.com> on 08/24/2001 07:59:31 PM
>
> To: "Burdett, David" <david.burdett@commerceone.com>
> cc: "'christopher ferris'" <chris.ferris@east.sun.com>,
> ebxml-msg@lists.oasis-open.org, ebxml-cppa@lists.oasis-open.org
> Subject: Re: reliable messaging
>
> "Burdett, David" wrote:
>
> > Chris said ...
> >
> > >>>If no acknowledgment has been received, the sender continues to
retry
> > delivery, using the Retries and RetryInterval to govern processing.
When
> the
> > number of retries identified by Retries is exceeded, the sending MSH
> > SHOULD notify the sending "party" by some means that is unspecified
> > (e.g. notify the application through some API that it provides, log
> > something
> > useful in an error log, etc.)<<<
> >
> > Note that there is an edge case where all the acknowledgements that
were
> > sent failed to be delivered, e.g. maybe a MSH can receive messages but
> not
> > send them. This means that even though no acknowledgement was received,
> the
> > message was actually delivered.
>
> That is indeed a point we have demonstrated in past POC.
> Clearly, RM cannot be substituted to a message-based transaction service,
> which
> is the right
> level to guarantee consistency across parties' apps. But it can be the
> basis for
> such a service.
> By NOT receiving an ack, the sender should not infer that the receiver
has
> not
> received the message:
> only that the reception has not been confirmed, and that it is OK to
resend
> it
> (the duplication check doing the cleanup job on receiver side).
>
> Regardless of what RM can or can't do, the question raise dby Martin W
> Sachs
> ( the requirement to notify sending party) is interesting in that it
> depends on
> the definition
> of RM:
> (1) if RM is a contract between sending party, receiving party, and MSH
> transport layer, then
> these sender notifications (as well as elimination of duplicate for
> receiver)
> are part of the contract.
> (2) if RM is a contract between two end-point MSHs, then these
> notifications
> have no normative
> value.
>
> My understanding is that (2) is currently applies (so SHOULD should
remain
> SHOULD...)
> However, once an formal MS API is specified, the MS spec will have to
> address
> the
> "contract" value of such API, with regard to sender and receiver...
>
> My two cents...
>
> Jacques Durand
> Savvion
>
> >
> >
> > David
> > PS Catching up on emails and logging them into the change request
> database
> > ;)
> >
> > -----Original Message-----
> > From: christopher ferris [mailto:chris.ferris@east.sun.com]
> > Sent: Friday, August 03, 2001 7:36 AM
> > To: ebxml-msg@lists.oasis-open.org; ebxml-cppa@lists.oasis-open.org
> > Subject: Re: reliable messaging
> >
> > Marty,
> >
> > Please see below.
> >
> > Chris
> >
> > Martin W Sachs wrote:
> > >
> > > Chris,
> > >
> > > I think I may have been unclear. I specifically am not after an
> > > application-level response for this purpose.
> > >
> > > The question is: when can a sending party conclude that his message
> > either
> > > was or wasn't delivered? That time is not relevant to the
performance
> of
> > > the application function. If the service provider site goes down
> before
> > > processing the message, but the message has been persisted ( a key
> > > requirement of reliable messaging), knowing that the message was
> persisted
> > > at the application is important information because it tells the
> sending
> > > party not to resend.
> >
> > Then receipt of the reliable messaging acknowledgment is the answer to
> > your question. That is the point at which the sender knows that the
> message
> > has been received and persisted.
> >
> > >
> > > Yes, receipt of the RM acknowledgment tells the party that the
message
> got
> > > there but how long does the sending party wait to decide that it
won't
> be
> > > receiving a guaranteed delivery failure notification? The answer, in
> my
> >
> > If the sender receives an acknowledgment, it won't be receiving a
> guaranteed
> > delivery failure notification because the message HAS been received.
Once
> > this acknowledgment has been received it SHOULD cease all reliable
> messaging
> > retries, etc. as any subsequent retries would place an unnecessary
burden
> > on both party's MSHs.
> >
> > If no acknowledgment has been received, the sender continues to retry
> > delivery, using the Retries and RetryInterval to govern processing.
When
> the
> > number of retries identified by Retries is exceeded, the sending MSH
> > SHOULD notify the sending "party" by some means that is unspecified
> > (e.g. notify the application through some API that it provides, log
> > something
> > useful in an error log, etc.)
> >
> > It isn't at all clear to me that the sender needs anything more than
> > Retries and RetryInterval to achieve its mission. Again,
persistDuration
> > is NOT a sending MSH parameter, it is a receiving MSH parameter.
> >
> > > mind, is long enough for all the allowable reliable-messaging retries
> to
> > be
> > > completed. I believe that persistDuration is the right answer as
long
> as
> > > it is prescribed that it be set long enough to cover the time to
> complete
> > > the allowed number of retries plus a little for propagating the
> delivery
> > > failure notification back to where the sending application can find
it.
> > > Alternately, a worst case time to recognize a delivery failure could
be
> > > defined.
> > >
> > > The sending application cannot determine if the message is relevant
> unless
> > > it knows that delivery did or did not succeed. Receiving or not
> receiving
> > > a delivery failure notification within a defined time is crucial.
> > >
> > > Yes, what I described covers several layers in the stack and maybe
> several
> > > middleware "modules". However, unless all the reliable messaging
rules
> > are
> > > set down in one place, they will never be understood.
> > >
> > > ...and let me reiterate again: The messaging service must guarantee
> that
> > a
> > > delivery failure notification will be sent by the sending MSH to the
> > > sending application in all cases where delivery could not be made.
> > Without
> > > this, reliable messaging is utterly broken because the key
requirement
> of
> > > reliable messaging is that the state of the business transaction not
be
> in
> > > doubt if the application-level acknowledgment is not received. If
the
> > > message sender is not notified of delivery failure, reliable
messaging
> > > fails because the sending application does not know if the message
got
> to
> > > the other party and therefore doesn't know how to recover. People
> outside
> > > of the ebXML teams are starting to notice this failure and conclude
> that
> > > reliable messaging is no good. Changing those SHOULDs to SHALLs is
> > > essential to the business future of the ebXML specifications because
> > > reliable messaging is a major component of the value of the ebXML
> message
> > > service.
> >
> > The MS specification cannot dictate to implementation vendors anything
> > of this nature. How they notify the sending "party" (application or
> > person) is strictly within their prerogative. The MS spec deals
> exclusively
> > with the details of the wire protocol, not the implementation details
> > of how an MSH is integrated with some application.
> >
> > I don't see how this can be perceived as a failure of the specification
> > when it is clearly (IMO) outside the scope of our work.
> >
> > If we change all of these SHOULDs to SHALLs then everyone would be
> > asking "how?" to which there is no possible answer that covers all
> possible
> > cases.
> >
> > >
> > > Regards,
> > > Marty
> > >
> > >
>
> ----------------------------------------------------------------
> To subscribe or unsubscribe from this elist use the subscription
> manager: <http://lists.oasis-open.org/ob/adm.pl>
>
> ----------------------------------------------------------------
> To subscribe or unsubscribe from this elist use the subscription
> manager: <http://lists.oasis-open.org/ob/adm.pl>
>
> ----------------------------------------------------------------
> To subscribe or unsubscribe from this elist use the subscription
> manager: <http://lists.oasis-open.org/ob/adm.pl>
----------------------------------------------------------------
To subscribe or unsubscribe from this elist use the subscription
manager: <http://lists.oasis-open.org/ob/adm.pl>
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [Elist Home]
Powered by eList eXpress LLC