Pub/Sub and decoupling

[David Hull <dmh@tibco.com>]

One of the major wins -- probably the major win -- of the pub/sub paradigm is the decoupling of sources (publishers) from sinks (consumers).  There can be an M:N relationship between publishers and subscribers and no party has to know about any of the others.  This is an obvious win over MxN point-to-point connections, where each publisher maintains a connection to N consumers and each consumer maintains a connection to M publishers.

There are many ways of looking at this, depending on ones focus.  The mental model I've been using in this context relies on an intermediary -- the usual technique for decoupling M:N relations in general.

In this view, each publisher has an outbound point-to-point connection to the intermediary, to which it may send messages from time to time.  Each consumer has an inbound point-to-point connection to the intermediary, from which it may receive messages from time to time.  The semantic is that (modulo filtering) every message sent to the intermediary is received by every consumer connected to the intermediary.  In this view, 1-of-N delivery would be effected by a different kind of intermediary.

WS-BaseNotification deals mainly with intermediary-consumer connections, while WS-BrokeredNotification tends more toward publisher-intermediary connections.

Each connection from intermediary to consumer (i.e., each subscription) is independent of the others -- each may have its own QoS, security, buffering, pause/resume capability or whatever.  Some may be pull-based, some may be push-based.  It might or might not be possible to redirect an active connection to a new destination endpoint.

I've been trying to capture the idea that all these stream-handling issues are orthogonal to the idea of decoupling publishers from consumers.  The multicast pub/sub I describe, the 1-of-N delivery variant and direct connection from source to sink all share the same concerns.

There is a certain duality here:  With the aid of an intermediary, you can view pub/sub as a particular setup using point-to-point connections.  On the other hand, you can view a point-to-point connection as a special case of multicast pub/sub (or of 1-of-N delivery).  Neither view is more correct or more implementation-friendly.  LANs are inherently biased toward multicast, WANs are inherently biased toward point-to-point.

I think it's important not to get too entangled in the issue of what the intermediate plumbing actually looks like.  The part we can nail down is the local view:  Each publisher sees a pipe to throw messages into, each consumer sees a pipe to get messages out of.  My intuition is that those views are independent of our model of how sources get connected to sinks.  Subscription via topics is one view, but others are possible.