[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [wsrm] R: [wsrm] Rel YY
One clarification to my previous message: > In this case, since the messages are un-ordered, the > SequenceNumber doesn't have to indicate the order of the message, > but it helps to identify the message within the GroupId. > And it is much simpler than identifying message with > long GroupId, especially for multiple Acking. The above sentence is describing about case#2. Thanks, Iwasa ----- Original Message ----- From: "iwasa" <kiwasa@jp.fujitsu.com> To: "Sunil Kunisetty" <sunil.kunisetty@oracle.com> Cc: "Doug Bunting" <Doug.Bunting@Sun.COM>; "Paolo Romano" <paolo.romano@dis.uniroma1.it>; "'Jacques Durand'" <JDurand@fsw.fujitsu.com>; <wsrm@lists.oasis-open.org> Sent: Thursday, September 25, 2003 2:37 PM Subject: Re: [wsrm] R: [wsrm] Rel YY > Sunil, > > The case I mentioned was un-ordered case. > When GoupId is different one by one, > the case I described would not be used, as you mentioned. > > However if we allow to use sequence number for > un-ordered messages, sender can send un-ordered > messages in one GroupId with unique SequenceNumber. > In detail, there are two ways to send un-ordered messages: > > 1. Different GroupId + SequenceNumber "0" > 1st message: > GroupId = aaaaa.bbbbb.cccccc/xxxxxxx/yyyyyy/zzzzzz/group1 > SequenceNmbr = 0 > 2nd message: > GroupId = aaaaa.bbbbb.cccccc/xxxxxxx/yyyyyy/zzzzzz/group2 > SequenceNmbr = 0 > 3rd message: > GroupId = aaaaa.bbbbb.cccccc/xxxxxxx/yyyyyy/zzzzzz/group3 > SequenceNmbr = 0 > > 2.One GroupId + unique SequeceNumber > 1st message: > GroupId = aaaaa.bbbbb.cccccc/xxxxxxx/yyyyyy/zzzzzz/group1 > SequenceNmbr = 1 > 2nd message: > GroupId = aaaaa.bbbbb.cccccc/xxxxxxx/yyyyyy/zzzzzz/group1 > SequenceNmbr = 2 > 3rd message: > GroupId = aaaaa.bbbbb.cccccc/xxxxxxx/yyyyyy/zzzzzz/group1 > SequenceNmbr = 3 > > For the case#1 above, we can't use efficient multiple Acking, > but for case#2, we can. I believe we should allow both cases > above, since grouping is up to sender and receiver's policy. > > And we can omit SequenceNumber for the case#1, and > spec can mention that it must be considered SequceNumber is 0, > if it is omitted. I think it helps those who don't want > to use SequceNumber. > > In this case, since the messages are un-ordered, the > SequenceNumber doesn't have to indicate the order of the message, > but it helps to identify the message within the GroupId. > And it is much simpler than identifying message with > long GroupId, especially for multiple Acking. > > Thanks, > > Iwasa > > > > > > > Iwasa, > > > > I'm bit confused. Could you clarify whether the options you have > > mentioned are for ordered or un-ordered case? > > > > If it is for un-ordered case, their GroupIds will be different, > > so you can't club them as you have mentioned in Option 2. > > If it is for ordered case, we do have SequenceNo and the current > > proposal has 2 sub-elements (RefToGroupId and RefToSequenceNo) > > under Response Header. For multiple acks., we will have multiple > > Response Header elements. > > > > Yes, it is in-efficient for multiple ack case, but we really haven't > solved > > or visited the multiple-ack case so far. We should definitely optimize > > it for such cases. > > > > Infact we still can use a modified option 2 for multiple acks efficiently > > by saying that RefToSequenceNumberStart and RefToSequenceNumberEnd > > as optional attributes? > > > > Current Response Header: > > <Response> > > <RefToGroupId> > > <RefToSequenceNumber> > > </Response> > > > > Modified version of your Option 2 would be: > > <Response> > > <RefToGroupId > > RefToSequenceNumberStart="1" > > RefToSequenceNumberEnd="100"> > > aaaa.bbbbb.cccccc/xxxxxxx/yyyyyy/zzzzzz/group > > </RefToGroupId> > > </Response> > > > > RefToSequenceNumberStart and RefToSequenceNumberEnd will be > > optional attributes. > > > > For un-ordered msgs., they won't be needed. > > For single ordered ack, they will be the same. > > For multiple ordered acks, they will represent the range. > > > > So I don't see a reason to have SequenceNumfer for un-ordered case. > > > > -Sunil > > > > iwasa wrote: > > > > > I also agree with Jacques proposal, > > > since I was considering a resolution and struggling > > > to find out a way to send multiple acks in a single Ack message, > > > which is our requirement. > > > > > > See the following example of five Acks in a single > > > Ack message: > > > 1) Without SequeceNumber: > > > <Response> > > > <RefToGroupId> > > > aaaaa.bbbbb.cccccc/xxxxxxx/yyyyyy/zzzzzz/group1 > > > </RefToGroupId> > > > <RefToGroupId> > > > aaaaa.bbbbb.cccccc/xxxxxxx/yyyyyy/zzzzzz/group2 > > > </RefToGroupId> > > > <RefToGroupId> > > > aaaaa.bbbbb.cccccc/xxxxxxx/yyyyyy/zzzzzz/group3 > > > </RefToGroupId> > > > <RefToGroupId> > > > aaaaa.bbbbb.cccccc/xxxxxxx/yyyyyy/zzzzzz/group4 > > > </RefToGroupId> > > > <RefToGroupId> > > > aaaaa.bbbbb.cccccc/xxxxxxx/yyyyyy/zzzzzz/group5 > > > </RefToGroupId> > > > </Response> > > > > > > 2) With SequceNumber > > > <Response> > > > <RefToGroupId SeqNmbrStart=1, SeqNumbrEnd=5> > > > aaaaa.bbbbb.cccccc/xxxxxxx/yyyyyy/zzzzzz/group1 > > > </RefToGroupId> > > > </Response> > > > > > > In conclusion, SequenceNumber is reasonably > > > desired to realize multiple Acks in a single Ack message. > > > Since we should not preclude a possibility of efficient and scalable > > > way of Duplicate Checking, and Multiple Acking. > > > > > > We may want to define default value of "0" as SequenceNumber for > > > non-ordered delivery, when it was ommitted. > > > > > > Thanks, > > > > > > Iwasa > > > > > > ----- Original Message ----- > > > From: "Doug Bunting" <Doug.Bunting@Sun.COM> > > > To: "Paolo Romano" <paolo.romano@dis.uniroma1.it> > > > Cc: "'Jacques Durand'" <JDurand@fsw.fujitsu.com>; > > > <wsrm@lists.oasis-open.org> > > > Sent: Tuesday, September 23, 2003 9:28 AM > > > Subject: Re: [wsrm] R: [wsrm] Rel YY > > > > > > > Paulo and Jacques, > > > > > > > > This appears to be a standard performance / storage trade off. The > > > > direction chosen at our last face to face meeting emphasizes > simplicity > > > > of the specification and implementations. The suggestions below > > > > optimize on the other side, toward reduced storage requirements. The > > > > included reasoning depends upon the number of messages received > overall > > > > (in some time period) and how many of those messages come from the > same > > > > origin. Without clear numbers applicable to all WS-Reliability use > > > > cases (something we are not going to get), I find it hard to choose. > > > > > > > > Absent concrete numbers, I would probably lean toward the simpler > > > > solution already chosen by the group. > > > > > > > > thanx, > > > > doug > > > > > > > > On 22-Sep-03 16:49, Paolo Romano wrote: > > > > > > > > > I like Jacques proposal. We should address both the problems of (1) > > > > > minimizing the state information that needs to be persistently > stored > > > > > and of (2) how to efficiently acknowledge batches of messages. The > key > > > > > idea of Jacques suggestion is to have a sequential, strictly > > > > > increasing identifier. Such a property could then be exploited to > > > > > record/acknowledge continous sequences of messages (i.e. messages > with > > > > > sequential MessageIDs) by referincing only the lower and upper bound > of > > > > > the sequence. > > > > > With this approach, it is possible to reduce the amount of memory > > > > > required at the purpose of duplicates elimination even furtherly:in > fact > > > > > if, for a given GroupID, a set of messages with > > > > > MessageID={0,1,..i..,n-1,n} has been received, then it is only > necessary > > > > > to store the MessageID "n", to indicate that messages up to n have > > > > > already been received. > > > > > As an example, consider a receiver RMP which has caught the messages > > > > > carrying GroupID:X and MessageID={1,2,3,4,5,6,10,15,16}. It should > only > > > > > record that "messageIDs up to 6" and MessageID={10,15,16} have > > > > > been received. In general, if a sequence of > MessageIDs={j,j+1,..,k-1,k} > > > > > with k>j is received, it is sufficient to store just j and k. Note > that > > > > > this property is independent on having enabled the ordering > > > > > functionality at the RMP level. > > > > > On the other hand, GroupID is essential (and sufficient) to allow > > > > > univoque identification of the "conversation". > > > > > > > > > > Paolo > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Here is a more precise wording of the issue I see with limiting > > > > > the use of sequence numbers to ordering only. > > > > > > > > > > Jacques > > > > > > > > > > > > > > > A new issue, (related to Rel-36, Rel-88) > > > > > ------------ > > > > > > > > > > Precluding the use of sequence numbers, when message ordering is > NOT > > > > > required, > > > > > will pose serious scalability issues for duplicate elimination > > > > > algorithms. > > > > > Indeed, in doing so the GroupID would actually behave as a > Message > > > ID, > > > > > and will require to be stored for each past message for the > > > > > duplicate look up. > > > > > Consider the two following deployment cases of WS-R: > > > > > > > > > > Case 1: Assume a messaging Hub, that must guarantee exactly once > > > > > delivery under > > > > > the following conditions: > > > > > Throughput: 1000 messages/sec > > > > > Size of GroupID values (approximately): 30 bytes > > > > > Scope of duplicate checks: messages over last 5 days > > > > > Using a messageID-based duplicate check, this Hub must keep a > > > > > database of GroupIDs able to store: > > > > > 1000 mesg * 432,000 (Number of seconds in 5 days) = 432,000,000 > > > > > GroupIDs to store. > > > > > Database size: 12.9 GB. > > > > > Besides the significant resource investment needed, (which may > > > > > please database vendors!) > > > > > this solution may simply not be feasible or at least cause quite > an > > > > > additional headache and cost: > > > > > the database may not keep up with the speed required for > duplicate > > > > > checks: > > > > > fast retrieval would require indexing. But the high rate of > updates > > > > > of such an index > > > > > (2000/sec counting additions and removals) offsets the > performance > > > > > gain in indexed search, > > > > > as we know indexes are costly to update, especially on a large > > > table. > > > > > One way or the other, the overhead of duplicate checking may > simply > > > > > be overwhelming. > > > > > > > > > > Case 2: Assume a messaging end-point, receiving 10 messages/sec > > > > > average. > > > > > Duplicate search is required over the last 30 days. We would > still > > > need > > > > > a full-fledged database (or B-tree) engine, with data size close > to > > > > > 1 GB. > > > > > > > > > > > > > > > Proposal: (P1 + P2 + P3) > > > > > --------- > > > > > > > > > > P1: Make it possible for Senders to use GroupID + SequenceNo to > do > > > > > any grouping they want, > > > > > even when ordering is not required. The only requirement is, > these > > > > > elements > > > > > should be used as they are for ordering, i.e.: > > > > > - (GroupID + SequenceNo) must be globally unique, > > > > > - the sender must generate contiguous sequence numbers within a > > > group. > > > > > > > > > > P2:SequenceNo element is mandatory in the header, even when NOT > > > > > requiring ordering. > > > > > In both cases, a Sender can at discretion: > > > > > - (a) send messages with a different GroupID each time (1 > message > > > > > per group, with smallest > > > > > SequenceNo) > > > > > - (b) send longer sequences of messages within a group. > > > > > > > > > > P3: To signal the Receiver to do ordered delivery on a sequence, > the > > > > > Sender will add > > > > > messageOrder element in the header (close to option (3) in Rel > 88), > > > > > and is required to > > > > > do so at least in the first message(s) of a group. > > > > > > > > > > NOTE1: On the receiver side, duplicate message elimination would > use > > > > > SequenceNo > > > > > for fast duplicate elimination within a group, and use an > indexed > > > > > search over a store > > > > > of GroupIDs for all groups currently active. Clearly, in the > extreme > > > > > case (a) above, > > > > > duplicate elimination would be as costly as over conventional > > > > > message IDs. > > > > > But if Case 1 above actually has only about 1000 "active groups" > at > > > > > any time > > > > > (e.g. 1000 concurrent senders generating each a single long > sequence > > > > > at rate of 1 mesg/sec), > > > > > then the GroupID store and seq number info associated with it, > can > > > > > hold in the memory > > > > > of a small device (estimate: 1 Mb) and does not need a DBMS. > > > > > > > > > > NOTE2: The requirement (Rel 26) to allow for "multiple-Ack" > > > > > messages, can be fulfilled > > > > > in a simple way when using sequences of messages, by the use of > an > > > > > interval notation to signal > > > > > groups of messages that are acknowledged. Or, use an upper > bound: > > > > > One Ack message could state > > > > > that all messages below SequenceNo N are acknowledged. > > > > > > > > > > > > > > > > To unsubscribe from this mailing list (and be removed from the roster > of > > > the OASIS TC), go to > > > > http://www.oasis-open.org/apps/org/workgroup/wsrm/members/leave_workgroup.php. > > > > > > > > > > To unsubscribe from this mailing list (and be removed from the roster of > the OASIS TC), go to > http://www.oasis-open.org/apps/org/workgroup/wsrm/members/leave_workgroup.php. > > > > > To unsubscribe from this mailing list (and be removed from the roster of the OASIS TC), go to http://www.oasis-open.org/apps/org/workgroup/wsrm/members/leave_workgroup.php. >
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]