RE: [obix-xml] Asynchronous Web Services Summary

From http://weblogs.cs.cornell.edu/AllThingsDistributed/archives/000388.html

Events, Eventing and Notifcation

Three web-service specifications have been published in the past 6 months, that deal with event notification, a fundamental paradigm in building distributed applications. HP published ws-events 2.0 in July of last year, and in the past month we have seen the publication of ws-eventing (MS, et al) and ws-notifcation (IBM, et al). I have made my first past over these specs and here are my initial thoughts. Most of these are just notes trying to distinguish the different systems from each other.

Event notification, and its big brother publish-subscribe, are technologies that are part of every distributed systems developers toolbox, and they been been for many years. The technology predates web-services by two decades and there is a wide range of free and commercial software systems available. Event notification and publish-subscribe are often used in inter-exchangeable manner, but me for they are different:

Event notification is a contract between two components where the event-sink requests the a-synchronous delivery of specific events from an event-source.
Publish-subscribe is a message (event) routing paradigm, where messages are being delivered to receivers, not based on specific addresses but on a set of criteria related to message metadata or to the message content. The publishers are in general unaware of the receivers, and the selective forwarding and delivery is performed by extensive middleware systems.

I know this is a very simplistic view, but I am trying to make a clear distinction between the two technologies. Event based programming is an essential tool in building a-synchronous applications, which helps to build scalable systems in the distributed and in the non-distributed case. One can see event notification as a restricted form of the publish-subscribe paradigm, where it appears as if there is a direct interaction between the publisher and the subscriber. This simple form is not a bad thing as a large collection of applications can be build using the simple notification pattern, and they do not need the overhead of a full blown publish-subscribe system. Of course a simple event based notification system can be used as a building block in a more elaborate pub-sub system.

There is a class of middleware systems, that includes broker technology to provide routing functionality, yet still claim to be event notification system. One of the litmus tests I often use for making the distinction between the two paradigms is whether there are dedicated interface specifications for publishers, and for the broker management system. In a simple event notification system for example there are no facilities to indicate that events need to be persisted, that there may be causal relations between events, how to handle periodic events with temporal expiration, etc.

If we have a look at these three specifications that all describe very different event systems, we see the following:

ws-eventing

ws-eventing is minimalist interface to an event system. It builds on the ws-addressing specification and relies on the wsa:EndpointReference construct and specifically uses the wsa:ReferenceProperties to identify the subscriptions. There are only a few interactions defined: subscribe, renew and unsubscribe, to which the event-source needs to respond. These requests do not need to be generated by the service that will consume the events, it can be done by a 3rd party that controls the source-sink associations. The subscriptions are leased, and need to be periodically renewed. When a subscription ends, the event-source needs to generate a subscription-end message, which is send to an endpoint specified in the original subscription.

The specification only deals with setting up and tearing down of the subscriptions, but does not specify what format the events should be in. If any reference properties were specified in the subscription request, then to will show up again as properties in the wsa:To section of the notification. Also the format of the event filtering specification, if used, is completely free.

ws-events

ws-events is a more extensive specification of an event notification system. It does not use ws-addressing, and is http specific. It has the subscribe/renew/unsubscribe interaction we already saw at ws-eventing, although they are named subscribe/extend/cancel. The notification message is encapsulated in a notification envelope.

An important part of ws-events appears to be the event-type discovery mechanism. A subscriber can query the system for the available types, for the definition of specific types and for which of these types have been instantiated. It can also subscribe to be notified when new types become available. Subscriptions are made to particular event types and there is no facility for the specification of subscription filters that would work over meta data or the event content.

The events can be delivered a-synchronously to a URL specified at subscribe time or they can be retrieved synchronously through a pull mechanism. In the pull operation the request can contain constraints such as all messages-since-messageID, messages-since-date, or message-range. The result of a pull operation is a notification list.

ws-notification

ws-notification is the more extensive of the three. But even the 70 pages that the specs now runs are actually on the low side, I expect that this specification will easily grow to 150-200 once more detailed examples of the different scenarios are added. The spec uses some of the idiom from the ws-resource frameworkk but is relatively self contained, and I would not be surprised if some of the idioms get replaced over time to make the spec completely independent. At 30,000 feet a ws-notifcation based system has a remarkable resemblance to the Corba Notification Service, but this is not surprising as there are only a few general accepted pub-sub/eventing interface strategies.

According to my earlier classification ws-notification describes a full-blown publish-subscribe system, not just an event notification interface. It clearly identifies publishers, brokers and subscribers, even promises support for federated brokers (although there is no specific mention in the spec on the intra-broker interaction), and allows for complex interaction between the different components (e.g. on-demand activation of publishers). The abstractions are chosen such that direction interaction between notification producers and consumers is possible, or can be accomplished with the help of a broker.

ws-notification uses a topic based publishing mechanism. This means that notifications are organized into topics and these topics can be organized into hierarchies where a parent topic is the aggregation of all child topics. A subscription to /system/motherboard will receive notifications published to /system/motherboard/fans/CPUfan.

A subscription request issued by a consumer can contain 3 constraints:

A topic expression. This expresses which topics you are interested in. There is a specific topic path expression language you can use to specify the topics.
A precondition query. This is an Boolean query that must evaluate to true when executed over the properties of the notification publishers. The language is independent of the spec and could be XPath. You can use this to select a subset of the notification publishers for your purposes. For example in a datacenter control system you could use this to select all quad processor machines.
A selector query. This is a Boolean expression that is run over the notification message, and must evaluate to true if the individual message is to be delivered to the consumer. Can be any selection language.

For example a subscription could request to receive notifications to be that are posted to both the /system/network/tcp-connections and /notifications/alarms/critical topics, with a precondition of 'server type == Tier #1' and a selector of "#syn-sends > 30,000". Of course these constraints require the knowledge of the topic spaces in use, the properties of the producers and the structure and content of the messages.

This spec provides for notifications to be delivered 'raw', similar to ws-eventing or encapsulated in a notification message, similar to ws-events. The interaction is always a-synchronous, even though there is a facility to retrieve the last notification produced for a particular subscription from the producer. A consumer can pause and resume a subscription, but there is no leased renewal based mechanism, nor have I been able to find the equivalent of an unsubscribe interface.

Overall the spec reads OK, but I feel that the second part of the spec, section 7 and higher which deals with the interfaces and the messages, can benefit from a more coherent discussion, and good examples. This part feels as if it was rushed to the printer.

Some final remarks

ws-eventing and ws-notification describe interfaces to systems that appear to be at opposite end of the spectrum. ws-eventing describes a simple mechanism for a-synchronous event delivery, ws-notifcation describes the interface to a system that has to potential be a full-blown pub-sub system. I see little overlap between the two systems, asa they are likely to target different audiences. ws-events seems to be caught in the middle of this, and the question is whether it will survive the corporate politics, and whether it should. The participation of HP in ws-notifcation could mean that ws-events might be replaced.

I do feel that these specs are very preliminary. They will change significantly before they are cast in stone. The feedback cycle with workshops that both groups have started should be interesting. I am looking forward to the moment when interface issues such as flow-control on notification delivery will be discussed. I will go to the ws-eventing workshop this week at Tibco but have to miss next weeks ws-notifcation workshop at HP, as there is a ws-transaction feedback workshop at the same time.

I do have some criticisms on both specs but will keep them to myself until these workshops are over and i have had some direct feedback.

One issue that I have not touched on at all is the interaction between these specs and WSDL. You should read Steve Vinoski's integration column in the March/April issue of IEEE Internet Computing for more insights in that issue, and for his views on some of these specs.

obix-xml message

From http://weblogs.cs.cornell.edu/AllThingsDistributed/archives/000388.html

Events, Eventing and Notifcation