MQTT-145 - MAY vs may

[Peter Niblett <peter_niblett@uk.ibm.com>]

Issue MQTT-145 is about the use of may
vs MAY in PRD1.  I have done a review of all the instances of the
words "may" and "MAY" in the document.

I would appreciate it if anyone who has had experience with may and MAY in other specifications could take a look at this and provide their views.

The most pressing issue I found is the question about whether a server is required to support wild cards in topic filters or not (this was explicitly called out in MQTT-145), but I have listed a number of other things that need changing in Section 3 below. In Section 4 I have included a number of statements that are currently MAY, but which could be replaced by may (and in general you are supposed to  be sparing in your use of MAY).  I've also listed the places where "may" appears in non-normative text which appear to be describing normative choices.

1. Introduction

RFC 2119 says that the word "MAY" is meant to be used only for items that are truly optional.  It says "One vendor may choose to include the item because a particular marketplace requires it or because the vendor feels that it enhances the product while another vendor may omit the same item. An implementation which does not include a particular option MUST be prepared to interoperate with another implementation which does include the option, though perhaps with reduced functionality. In the same vein an implementation which does include a particular option MUST be prepared to interoperate with another implementation which does not include the option (except, of course, for the feature the option provides.)"

- The best example I could find of "MAY" being used in this way is in line 849: "A Server MAY allow a Client to supply a ClientId that has a length of zero bytes."

In contrast lower case "may" should be used when we are making an assertion about something that is permitted to happen within the protocol,

- As an example, in line 1548 we say "Topic level separators may appear anywhere in a Topic Filter or Topic Name".  

This kind of "may" is really making a kind of MUST statement in that an implementation is required to expect that Topic Filters or Names can have separators anywhere.  

I notice that we also use lower case "may" in non-normative text, even where we are talking about items that are truly optional.

2. Instances of may or MAY that are ok

 I found the following occurrences of lower-case "may" that look ok to me - none of them are meant to have the RFC2119 meaning

34 "At least once", where messages are assured to arrive but duplicates may occur.

47. A TC may approve a Working Draft

66 - 82  Several occurrences of "may" in the standard OASIS text at the beginning of the document

1394 Normal operation of the Client of Server may mean that stored state is lost or corrupted because of administrator action, hardware failure or software failure.

1397 For example the server may determine that based on external knowledge, a message or messages can no longer be delivered to any current or future client.

1404 For example, a user wishing to gather electricity meter readings may decide that they need to use QoS 1 messages because they need to protect the readings against loss over the network, however they may decide that the power supply is sufficiently reliable that the data in the Client and Server can be stored in volatile memory without too much risk of its loss.

1548 Topic level separators may appear anywhere in a Topic Filter or Topic Name

1666 Devices may be compromised

1667 Data at rest in Clients and Servers may be accessible

1668 Protocol behaviors may have side effects (e.g., 'timing attacks')

1670 Communications may be intercepted, altered, re-routed or disclosed

1688 In addition to technical security issues there may also be geographic...

1696 An implementation may want to provide conformance with specific industry security standards such as NIST Cyber Security Framework [NISTCSF] ...

1728 Implementations passing authentication data in clear text, obfuscating such data elements or requiring no authentication data should be aware this may give rise to Man-in-the-Middle and replay attacks.

1758 An implementation may allow for authentication where the credentials are flowed in an Application Message from the Server to the Client.

1780 An application may independently encrypt the contents of its Application Messages.

1784 Client and Server implementations may provide encrypted storage for data at rest

1801 Client and Server implementations using TLS [RFC5246] may choose to provide capabilities to check Certificate Revocation Lists (CRLs [RFC5280])...

1855 Some MQTT implementations may make use of alternative secured tunnels (e.g. SSH) through the use of SOCKS.

1858 implementations should be aware that SOCKS authentication may occur in plain-text

1863 Implementers and solution designers may wish to consider security as a set of profiles

1883 TLS [RFC5246] Client authentication may be used in addition to – or in place of – MQTT Client authentication as provided by the Username and Password fields.

I found the following uses of "MAY" which seem to fit the description in RFC 2119:

444 The data SHOULD NOT include encodings of the Unicode [Unicode63] code points listed below. If a receiver (Server or Client) receives a control packet containing any of them it MAY close the network  connection.

679 If the protocol name is incorrect the Server MAY disconnect the Client, or it MAY continue processing the CONNECT packet in accordance with some other specification.

842 The Server MAY restrict the ClientId it allows in terms of their lengths and the characters they contain

849 A Server MAY allow a Client to supply a ClientId that has a length of zero bytes.

888 Note that a Server MAY support multiple protocols (including earlier versions of this protocol) on the same TCP port or other network endpoint.

898 The Server MAY check that the contents of the CONNECT Packet meet any further restrictions and MAY perform authentication and authorization checks.

1022 In addition, the Server MAY deliver further copies of the message, one for each additional matching subscription and respecting the subscription's QoS in each case.

1516 It MAY provide an administrative or other mechanism to allow one or more Topics to be treated as an "Unordered Topic"

1594 MQTT Server implementations MAY define Topic Names that start with a leading $ character

3. Instances which need fixing

a) Topic Filter wild cards.  The question here is whether a server is required to support wildcards or not. At the moment there are two statements with lower case may (which implies that the server is required to support wild cards) and one with a MAY which implies that it does not have to support them. I think our intention is that a server is expected to support them, so the MAY in 1153 should be a may.

240/1 A Topic Filter may include wildcard characters.

1153 The Topic Filters are UTF-8 encoded strings, which MAY contain special wildcard characters to represent a set of topics, see Section 4.7.1.

1541 A subscription’s Topic Filter may contain special wildcard characters, which allow you to subscribe to multiple topics at once.

b) Dynamic Topics. As written this seems to require a Server to offer a choice of pre-defined or dynamic topics. I think this is a case where we could use some MAYs

1650 The topic resource may be either predefined in the Server by an administrator or it may be dynamically created by the Server when it receives the first subscription or publication with that Topic Name. The Server may also use a security component to selectively authorize actions on the topic resource for a given Client.

c) WebSocket.  I think this particular statement is ok as written, but issue mqtt-194 is about rewriting the whole section to include some MUSTs...

1902 WebSocket binary frames are used. A single frame may contain multiple or partial MQTT Control Packets; they are not required to be aligned.

d) The following occurrences of "MAY" are not meant to have the RFC2119 meaning so should be replaced by lower case may.

526.  It SHOULD store the new QoS 0 message as the new retained message for that topic, but MAY discard it at any time.

816  Note that a Server MAY choose to disconnect a Client that it determines to be inactive or non-responsive at any time, regardless of the Keep Alive value provided by that Client.

1178 The Server MAY start sending PUBLISH packets matching the Subscription before the Server sends the SUBACK Packet.

1299 It MAY continue to deliver any existing messages buffered for delivery to the Client.

1913 A single entity MAY conform as both an MQTT Client and MQTT Server implementation.

4 Instances which are ok, but we could consider changing

a) I found the following occurrences of "MAY" which could be interpreted to have RFC2119 meaning, but I think could be replaced with lower case may, as I don't think they directly affect interoperability

708 It MAY also store QoS 0 messages that meet the same criteria.

716 The Client MAY store QoS 0 messages for later transmission.

722 The Server MAY store QoS 0 messages for which transmission to the Client has not yet been started.

1483 Clients MAY resend SUBSCRIBE and UNSUBSCRIBE Packets on reconnect but are not REQUIRED to do this.

b). Places where lower case "may" is used to describe an optional feature or behaviour in the following places, but this is text that is marked as non-normative:

864 A Client implementation may provide a convenience method to generate a random ClientId.

1723 Implementations can choose how to make use of the content of these fields. They may provide their own authentication mechanism, use an external authentication system such as LDAP [RFC4511] or OAuth [RFC6749] tokens, or leverage operating system authentication mechanisms.

1743 An implementation may restrict access to Server resources based on information provided by the Client such as User Name, Client Identifier...

1844 Servers may disconnect Clients and require them to re-authenticate with new credentials.

I'm wondering whether some of these (e.g. 1844) should be included in normative sections of the spec.

Regards

Peter Niblett
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU