RE: [wsn] Use cases: Durable Subscription

["Rick Cobb" <rcobb@KnowNow.com>]

Well, here are my not-particularly-well-organized thoughts.  Dave, Lily, perhaps we should caucus sometime this evening?
 
Thanks --
-- Rick

________________________________

From: David A. Chappell [mailto:chappell@sonicsoftware.com]
Sent: Mon 1/31/2005 7:44 AM
To: Lily Liu; wsn-oasis
Subject: RE: [wsn] Use cases: Durable Subscription



-----Original Message-----
From: Lily Liu [mailto:lily.liu@webmethods.com]
Sent: Friday, January 14, 2005 11:38 AM
To: wsn-oasis
Subject: RE: [wsn] Use cases: Durable Subscription

[snip]

> - What is the duration of the durability?  (How long do saved messages
> need to be saved?).  We could use the termination time of the message
> and do it on a per message basis. 
<lily>
It can make things easier if the durability of the subscription is the
same
as the lifetime of the subscription. Message expiration is a separate
concept. Do we have a concept of message lifetime in WSN? Should we
discuss
that too?
</lily>

Good point.  That would be in the queueing/replay spec that we never
fleshed out. :)  I think we should have a way of doing expiring it, even
if it is to rely on the underlying RM protocol to expire messages.
Dave





To support the two use cases, we need to handle the following situations:

0. Discovery of whether durable subscriptions are supported

1. Reconnection of the consumer

2. Establishment of the delivery state

3. "Replay" (really, "play") to reach the current NP state

Assume:

1. Consumer retains delivery state

2. Consumer EPR has changed

3. Subscription EPR may no longer be available at the producer 
   (removed after fault)

Discovery
---------

An NP may support durability  via 

    (a) Durability of the subscription itself (i.e., the consumer
        may "resume" a subscription that has reached a fault state).

    (b) Durability of the topic & message data (i.e., the consumer
        may "replay" "old" information by presenting a historical 
        query [like 'send everything after this <timestamp> or <message
        identifier>']

Case (a) is primarily germain to use case (1), and (b) to use case (2).

The primary distinction is in the cost to store consumer state; in use
case (1), the time frames considered make storing consumer state at the
NP potentially possible, whereas in use case (2), system scalability
demands that the consumer maintain more state information.

So it seems like the NP must support policy declarations that can express
things like:

    (a) Subscription durability is supported with the following limits:
        a.1  Subscription reconnect time no longer than <foo>

    (b) Topics may have properties identifying their retention policies
        a.1  Retained until acknowledged or expired
        a.2  Retained until deleted or expired
        a.3  No expiration
        a.4  [Brokered case]: Maximum expiration

    (c) Identification of the information that can be used in historical
        queries (e.g., <timestamp>, <message_id>, <sequence_number>). I'll
        call these <identification_methods> below. 

Connection/Reconnection
-----------------------

* A subscription request may request a session reconnect time (time failure is allowed) 
    * If greater than the policy, <fault>
    * If out of resources, <fault>

* Reconnection is a subscription modification request.  

Establishment of delivery state
-------------------------------

In use case (b), we assume that the NP has lost the subscription state
of the original subscription.  OTOH, the consumer or subscriber has
retained timestamp, sequence, or message identification information from
the topics it previously subscribed to.

* A subscription request may identify in its <filter> (<selection>?) that
  it is interested in all events since a given historical point.

* If the state information given in the subscription request pre-dates the earliest retained message, the subscription should (fault?, play from earliest?)

Non-duplication and missed events
---------------------------------

In many cases, it will be impossible to prevent both duplicate
deliveries and missed events in the case of historical queries.
For each <identification_method> supported by the NP, the policy must
state whether it will deliver duplicates or miss events.

Other considerations
--------------------

In JMS, an individual message may be durable or non-durable.  In this case, 
the assumption is that non-durable messages may not be delivered across any interruption in service (whether use case (a) or (b)).

An alternative is to distinguish between NP/Broker failure and
NC/Subscriber failure. Non-durable messages are not delivered across
NP/Broker failures, but may be across NC/Subscriber failures.  This would
require another layer of error porcessing; I'm not sure it's worth
the effort.