[OASIS Issue Tracker] (ODATA-868) Describe HTTP encoding for streamed requests and responses

[OASIS Issues Tracker <workgroup_mailer@lists.oasis-open.org>]

    [ https://issues.oasis-open.org/browse/ODATA-868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=67910#comment-67910 ] 

Evan Ireland commented on ODATA-868:
------------------------------------

(edited transcript of SAP internal discussion on this topic)

From: Ireland, Evan 
...
Subject: RE: In-stream errors with OData

Ralf,

Thanks for the document link. I was looking for it in ATOM and JSON specs 

First of all, for clients, we don’t need to distinguish JSON (or ATOM for V2) that is transferred in chunks (e.g. HTTP chunked transfer encoding). Client libraries (e.g. Java, C#) usually provide a byte/char stream interface to HTTP response content (whether or not the server uses chunking), so we implement all client-side parsing stream-wise – with multiple layers of streaming. For example, with JSON (see http://xscript.mo.sap.corp:8080/server/xscript.json.html), for streamed parsing of a delta response, we can:

(1)	Get hold of a CharStream from HTTP layer.
(2)	Wrap that with a JsonTokenStream.
(3)	Wrap that with a JsonElementStream.
(4)	Wrap that with a JsonDeltaStream.

Any non-streamed client-side parsing (e.g. to get a complete collection of entities) is just a wrapper over the appropriate level of streamed parsing.

...

Now you wrote: “I’d especially be interested in ways that allow appending an error response object after signaling the premature end of the “streamed” original response.”

Well with HTTP chunked transfer encoding, there is facility for “trailers” (https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Trailer#Chunked_transfer_encoding_using_a_trailing_header) but apparently the processing of such trailers isn’t well supported by client libraries. So let’s not go there.

So I would like to suggest instead “appending an error response object before signaling the premature end of the “streamed” original response”.

Consider for example the following delta response:

    {"@odata.context":"...",”@odata.deltaLink”:”…”,"value":[xxx,yyy,zzz]}

where “xxx”, “yyy” and “zzz” are shorthands for embedded JSON objects (e.g. changed/deleted entities/links).

Now in relation to the spec text you referenced: “In the case that the service encounters an error after sending a success status to the client, the service MUST generate an error within the payload, which may leave the response malformed. Clients MUST treat the entire response as being in error. This specification does not prescribe a particular format for generating errors within a payload.”

Since the specification does not prescribe a particular format for generating errors within a payload, but clients must treat the entire response as being in error, I would suggest that the server MUST leave the response as malformed (otherwise there is no way to portably assume the client can detect the error). So, invalid JSON must be returned.

The easiest way to handle this is for the server to omit the trailing right brace ‘}’.

But how to give the client some more useful error information?

Suppose the production of ‘zzz’ in the above example breaks due to a connection to the database from the app server being broken. Assume that at that point, we have already streamed (via HTTP chunked transfer encoding):

    {"@odata.context":"...",”@odata.deltaLink”:”…”,"value":[xxx,yyy

Now what the server could do is to transmit:

    ],”@error”:{…}

where “]” terminates the incomplete collection, “@error” is a new (e.g. 4.01) instance annotation for in-stream errors, and “…” is the appropriate encoding of a JSON error response.

Note in particular the final ‘}’ to terminate the overall response is intentionally missing. So therefore the response is clearly malformed (as per 4.0 requirements).

Additionally, clients that don’t completely ignore the “@error” annotation (before tripping up with the missing ‘}’) can either:

(1)	Process it, providing extra error information, or
(2)	Treat it as a malformed response.

For 4.0 (and prior), we could use “@odata.error” as the annotation name. We can consider this backwards-compatible since the (4.0) specification “does not prescribe a particular format for generating errors within a payload”, and clients that want to treat it as a malformed response are free to do so.

So the basic approach here is to assume that any meaningful streamed responses contain (potentially-large) collections and that if we trip on a certain collection item, we terminate the collection with ‘]’, add an “@error” instance annotation (which is client-parseable for in-the-know streaming clients) and subsequently do not include any of the remaining required array/object terminators.

For V2 with Atom, we could include an Atom “error” response (http://docs.oasis-open.org/odata/odata-atom-format/v4.0/odata-atom-format-v4.0.html ) in the feed (where an “entry” would normally be expected), and omit the terminating </feed>.

How does this sound for a way forward?

Thanks


> Describe HTTP encoding for streamed requests and responses
> ----------------------------------------------------------
>
>                 Key: ODATA-868
>                 URL: https://issues.oasis-open.org/browse/ODATA-868
>             Project: OASIS Open Data Protocol (OData) TC
>          Issue Type: Bug
>          Components: JSON Format
>    Affects Versions: V4.0_ERRATA02
>            Reporter: Ralf Handl
>            Assignee: Hubert Heijkers
>              Labels: Clarification
>             Fix For: V4.02_WD01
>
>
> OData JSON Format has a format parameter for streaming scenarios but does not describe how these messages should be encoded.
> Part 1: Protocol, section 9.4 talks about in-stream errors in a format-independent way. Unfortunately we don't define how these should be represented in the JSON format.
> Situations where in-stream errors can occur include
> - metadata requests in XML or JSON
> - data request in JSON or XML
> - media resource requests with any content-type



--
This message was sent by Atlassian JIRA
(v6.2.2#6258)