[OASIS Issue Tracker] (MQTT-197) Support request/reply as a new MEP

[OASIS Issues Tracker <workgroup_mailer@lists.oasis-open.org>]

    [ https://issues.oasis-open.org/browse/MQTT-197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=59327#comment-59327 ] 

Raphael Cohen commented on MQTT-197:
------------------------------------

If we can do this without having to have brokers understand it, that'd be perfect. Optional understanding would be good, so gateways, routers, aggregators, etc could be effective.

If we did this as a committee note, then we could do it by a special message format. This has come up in the discussion of content-type and topic meta data, and I'd encourage us to align the two if possible. A publisher sends a message formatted appropriately to a topic where a service is listening. The service understands the format, and sends a reply to a reply-to topic. This topic can be named by convention (it could include a client id, or correlator id, or it could be just /my/topic because we want anyone to handle replies and they all share some publication state). The broker can know diddly. If we want to hint to the broker, so it could do processing, we use the $ namespace.

As a message format, I'd actually propose something very simple: HTTP/1.0 (actually MIME). In other words:-

- Key-Value headers, terminated with /r/n (and line continuation not allowed; it's deprecated anyway)
- Followed by the remaining payload
- A header for Reply-To or Correlator, which ever folks prefer (personally, I'd probably prefer a Reply-To with a topic name of /path/to/my/service/UUUID-As-Correlator, and the publisher subscribes to /path/to/my/service/#)
- If adopting this format, Content-Length would be handy, but therein lies the rub - all of a sudden, everyone will want to add their favourite header, or abuse those from HTTP (chunked encoding, anyone)? Or think they can pass HTTP GET requests directly. And we end up with all the horrible things that MIME has been plagued with over the years.

Whilst there is no doubt that parsing such a format is inefficient (compared to say ProtocolBuffers or a bespoke binary encoding), it is simple to debug, inspect and work with - there are many, many client libraries out there, and one can even parse this using shell script in a few lines (eg by wrapping an awk script that produces a character-by-character output). And products like nginx have shown that it isn't difficult to maintain good performance even with parsing this format. If folks are really worried, then we could encourage this format to be used on a $ namespace.

If the broker doesn't need to understand it, then the burden of parsing then resides with the 'services' receiving the messages, and these can be easily scaled out, simply by adding more processing nodes and the proposed shared subscriptions. The broker remains stateless (which is important, as it's the bottleneck in the flow). If one wants to convert to AMQP, one can, although there is no correlation id that can be used (but there is a ReplyTo IIRC).

There is a problem with security of replies, as Paul alludes to, with this design. Soluble, but unpleasant.

My suggestion is that we write up a draft committee note, try it out with some real use cases and see what works and what doesn't, and use that to move forward.


> Support request/reply as a new MEP 
> -----------------------------------
>
>                 Key: MQTT-197
>                 URL: https://issues.oasis-open.org/browse/MQTT-197
>             Project: OASIS Message Queuing Telemetry Transport (MQTT) TC
>          Issue Type: New Feature
>          Components: candidateCN
>            Reporter: Shawn McAllister
>
> This JIRA requests that the version of the MQTT protocol after 3.1.1 be augmented to support the request/reply message exchange pattern.  Minimally what is required is the ability to (optionally) carry a "reply-to" topic in a published "request" message that would allow the receiver to publish a "reply" message addressed to the contained reply-to topic.
> Hopefully the need to support request/reply is not in dispute as it is a very common message exchange pattern supported by HTTP, JMS and all major messaging products because of its usefulness and for which there are many applications in M2M use cases.
> The only way now with MQTT3.1.1 (that I am aware of)  to return a reply to the sender of a request message is for the requester and replier to coordinate on the reply-to address outside of any specific services provided by MQTT.  For example, by embedding the reply-to address in the payload of the message.   The main functional reason this is insufficient is that it does not allow protocol interworking functions(MQTT to some other messaging protocol)  that are not (and should not be) aware of application payloads to create an end-to-end request/reply flow.
> Therefore I would like to request that the TC consider and implement the protocol changes required to support this functionality.



--
This message was sent by Atlassian JIRA
(v6.2.2#6258)