[OASIS Issue Tracker] (MQTT-329) Remove the ID/Value Pair length field

[OASIS Issues Tracker <workgroup_mailer@lists.oasis-open.org>]

    [ https://issues.oasis-open.org/browse/MQTT-329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=64690#comment-64690 ] 

Ken Borgendale commented on MQTT-329:
-------------------------------------

The field you are talking about removing is called "Property Length".  I presume that you are not proposing just removing it but replacing it with some other means of determining the extent of the properties such as a delimiter.

The encoding of properties was discussed for 8 months under MQTT-256 including a major amount of time in several of the face to face meetings.  Two of the finalist encodings were to specify a length for the metadata (now Property Length) or to end the metadata with an end-of-metadata Identifier.  The proposal to use a length was selected.  I do not think we should reopen some of the wounds from that discussion unless we think it is a major error, not just an inconvenience. 

The obvious problem in using a Variable Byte Integer as you state is that you cannot just leave some pre-allocated space for the length and fill it in later.  A similar problem exists for the Variable Byte Integer used for the length of the packet.  I presume that you have found methods for handling that case.  In my implementations of MQTTv5 for clients, proxies, and servers I used several strategies for this.
1. Assume the properties are less than 127 bytes and leave one byte of length.  If this fails leave two bytes and repeat.
2. Write the properties into a separate buffer and then copy them into the main buffer when the length is known.  Using a stack buffer this is commonly quite efficient
3. Move the data in the buffer left or right as required to put in the length.

None of these take very much code and it really depends on the expectation and the exact operation.  For instance in a proxy I generally maintain the properties but I need to add or remove them in some cases so move bytes.  For PUBLISH I create the properties in a stack buffer, and for the other packets I tend to loop the writing of the properties.  For cases where there is a need to edit the properties either the length or delimiter encodings has its problems.

Compared to the amount of code needed to correctly parse and check properties, the code needed to write them is pretty small.

To go back to the reason we went with the length based rather than delimiter encoding, this was a split decision which helped move us to consensus.  The primary value of the property length is that:
1. It is possible to get to the payload by simply skipping over the properties.  This will most likely be done by an embedded client or a wire format parser.  
2. The length adds redundancy which the delimiter does not have.    With the length.we can check that the properties really do stop at a property boundary.  


> Remove the ID/Value Pair length field
> -------------------------------------
>
>                 Key: MQTT-329
>                 URL: https://issues.oasis-open.org/browse/MQTT-329
>             Project: OASIS Message Queuing Telemetry Transport (MQTT) TC
>          Issue Type: Improvement
>            Reporter: Allan Stockdill-Mander
>            Priority: Minor
>
> The current spec requires an id/value pair length field indicating how many bytes constitute the id/value pairs in the controlpacket, prior to any payload that may be in the controlpacket as well. From an implementer's point of view this means allocating and building a buffer to hold this data so it can be counted and the value inserted logically prior to this buffer. This is particularly problematic as the value is a Variable Byte Integer so space in a larger control packet buffer cannot be preallocated then filled afterwards (same annoyance occurs with the remaining length field earlier in the control packet).



--
This message was sent by Atlassian JIRA
(v6.2.2#6258)