RE: [wsrp] WSRP url parameters and URL Encoding questions

[Andre Kramer <andre.kramer@eu.citrix.com>]

see below,

 

regards,

Andre

-----Original Message-----
From: Rich Thompson [mailto:richt2@us.ibm.com]
Sent: 31 March 2003 15:06
To: wsrp@lists.oasis-open.org
Subject: RE: [wsrp] WSRP url parameters and URL Encoding questions

The way the spec reads now, it is the Consumer's responsibility to return to the Producer whatever was specified. So for the two cases:
 1. Consumer rewriting: Consumer's task to encode/decode parameters for round trip through user agent so that it can resupply them correctly to the Producer.
[Andre Kramer] producer must also not include any "&" in parameter values for consumer writing.

 
 2. Producer writing: Producer's job to encode/decode/interpret properly when parameters make round trip through user agent (Consumer just reads from the activated URL and places the value into the appropriate field).
[Andre Kramer] The consumer's Web Server is likely to do a "URL decode" as the URL params come in to it. Best to forward unencoded to the producer, I think.

 
The discussion below sounds to me like some testing may be in order and then a section of the primer can discuss guidance in this area. 
 
[Andre Kramer] The question is how much up front work we put in to make inter-op possible (even if common Web Servers URL decode and recognize the '/' url path separator) and probable.

 

 Not sure we need to be modifying the spec to say what all the issues might be, but do agree with the general comment that started this thread that the spec say that all parameters need to be URL encoded by the Producer. 

[Andre Kramer] If not the spec, we need a document with some ability to make interoperability conformance statements (and a process to allow us to agree on them). Something that does not have to be frozen at 1.0. An Implementer's best practice?

 
Rich Thompson

Andre Kramer <andre.kramer@eu.citrix.com> 03/31/2003 04:01 AM

To:        wsrp@lists.oasis-open.org         cc:                 Subject:        RE: [wsrp] WSRP url parameters and URL Encoding questions

Some <ak> comments below.

-----Original Message-----
From: Eilon Reshef [mailto:Eilon.Reshef@webcollage.com]
Sent: 28 March 2003 22:07
To: wsrp@lists.oasis-open.org
Subject: RE: [wsrp] WSRP url parameters and URL Encoding questions


Rethinking about it - in the end of the day, URL encoding can be done by two
actors:

<ak> A special case is wsrp-url where the consumer must be able to get the
(producer's) actual parameter value. The other parameters hold data that is
opaque to the consumer or data that does not really need to contain any %aa.
</ak>

When the Producer writes the full URLs using the provided templates, the
question is what happens when the Producer encodes a navigational state of
e.g., a%20b. I believe (and may be wrong here) is that the standard states
that the Producer should expect to receive the navigational state "a b" in
the getMarkup() call. If the Producer so wishes, it may encode information
further but that should be transparent to the protocol. (Eventually portlets
will use an auxiliary function provided by the Producer to encode the
various parameters and the Consumer is oblivious to the content, so it's
mostly a Producer decision.)

When the Consumer writes the full URLS, it parses the "original" URL
provided by the Producer to find the WSRP-specific parameters. The Producer
must thus encode illegal characters to ensure that the Consumer can parse
the URL, and to that end the encoding mechanism must be defined. (It is then
up to the Consumer to rewrite the URL in a way that will "survive" across
the interaction with the user - the Consumer may decide to re-escape
elements or to encode them in other ways, but this decision is made solely
by the Consumer and does not seem to affect WSRP.)

<ak> The consecutive '/'s problem can be addressed by the consumer in the
second case but we should have the same rule for "//" for both schemes. So
that the producer should be able to treat incoming param values in the same
way for both schemes. <ak>


I believe (and may be wrong here again) that this is the situation captured
in the spec.

<ak> We could clarify editorially (as Rich suggests) by stating the need to
URL encode all (producer's) parameter values (that may contain special
chars) and that this layer of encoding is removed by the consumer or Web in
both schemes (on the way back in to the producer). </ak>

Now, it seems to me that if a Producer generates a Consumer URL that has
wsrp-navigationalState=x%aa%aay, where %aa is the URL encoding for the
character '/', and the Consumer incorrectly passes back a navigation state
of 'x/y', then this is an incorrect Consumer behavior. If that behavior is
done before the Consumer gets control (e.g., by the Web server or by a
gateway) then we may need to prohibit the %aa as a parameter, for otherwise
we can expect to see many erroneous Consumers.

<ak> We only need to prohibit "//", "///" etc (i.e. runs of %2f in (singly)
encoded form). However, this formalizes a work-around and still means the
consumer must still carefully handle "/" (only a minor implementation issue
for some consumers).

We could ban all %xy from parameter values (i.e. no % chars at all). Very
draconian, but may prevent interoperability problems down the road.

We could just ban "/" and give no advice (apart from clarify producer encode
required but no producer decode required) [as you suggest.]

We could also point out that a double url encode will hide any "/" (and
other special chars but %) from all but the producer (who must then also do
a decode). But if we don't require this, then the consumer must still be
careful about parameter value contents (e.g. it may care about all '/'s) and
wsrp-url remains a problem.

Requiring a double encode may be a step to far. So, I agree a ban on "/" is
simplest. A required double encode (by the producer for both URL schemes so
we have one set of rules) and a single producer decode would give some added
protection but may be more than we should be specifying.

With a '/' ban, we still need to be clear on wsrp-url. We could just allow
'/' in wsrp-url proxy urls, as the consumer can have simple heuristics to
deal with the "://" issue (the "//" is redundant anyway!).
</ak>

Eilon


-----Original Message-----
From: Subbu Allamaraju [mailto:subbu@bea.com]
Sent: Friday, March 28, 2003 8:37 PM
To: wsrp@lists.oasis-open.org
Subject: Re: [wsrp] WSRP url parameters and URL Encoding questions



Eilon Reshef wrote:
> A somewhat different approach would be to disallow "special"
> characters
> in the portlet-provided values, e.g., navigational state, etc.
>  
> This would mean that a portlet can only use, say, URL-allowed
> characters
> (alphanumeric+), but not /, #, :, ?, =, ;, %, etc. (the full list is
> documented in the corresponding RFC).

I prefer encoding. Applications/portlets may use special characters
(say, "Smith & Co") in state, and it is safer to encode those characters
 instead of disallowing such strings.

> This will make it easy for the Consumer to process (no special
> handling,
> no dependency on Web server), and will also make it easy for portlets
> and Producers (easy rules, the portlet can always do one level of
> escaping and replace the % with, say a dash (-).

Is this not somewhat similar to requiring portlets to URL-encode
parameters? So, why not requre the consumer do this? This would be less
restritive for developers.

Subbu



>  
> Escaping seems to be one of the more common sources for
> incompatibility
> and it may just be safer to sidestep rather than spend a lot of time to
> ensure that compatibility across Producers Consumers and Web servers.
>  
> My two cents,
> Eilon
>
>     -----Original Message-----
>     *From:* Andre Kramer [mailto:andre.kramer@eu.citrix.com]
>     *Sent:* Friday, March 28, 2003 7:41 PM
>     *To:* wsrp@lists.oasis-open.org
>     *Subject:* RE: [wsrp] WSRP url parameters and URL Encoding
> questions
>
>     Rich,
>      
>     Unfortunately, the collapsing of // to / also happens within
>     parameters not just to the separators, e.g. the navigationalState
>     parameter value written into the template by the producer is "a//b"
>     but is modified to "a/b" following normal url path processing
> rules.
>      
>     The whole path is subjected to this collapsing. The consumer can
>     protect itself from this for its own parameter separators as you
>     suggest (and even have special smarts for wsrp-url values), but the
>     producer should not need to special case any "//" in its URL
>     parameter values (wsrp-navigationalState and wsrp-interactionState
>     are the only difficult parameters). That is why I'm suggesting doing
>     a double encoding. We could instead suggest no consecutive '/'s
>     allowed in navigationalState and interactionState but that seems
>     very arbitrary.
>      
>     regards,
>     Andre
>
>         -----Original Message-----
>         *From:* Rich Thompson [mailto:richt2@us.ibm.com]
>         *Sent:* 28 March 2003 16:42
>         *To:* wsrp@lists.oasis-open.org
>         *Subject:* Re: [wsrp] WSRP url parameters and URL Encoding
> questions
>
>
>         It would be a good editorial clarification to comment that all
>         parameter values should be URL encoded since they will appear in
>         the URL activated by an End-User interaction. Does anyone object
>         to adding such a comment?
>
>         As to the particular question about collapsing // into /, the
>         Consumer can easily prevent this by using a construct such as /_
>         to separate parameters (or "/ns_" before the navigational state
>         if so desired). I think we can leave it the Consumer's
>         responsibility to properly construct its templates.
>
>         Rich Thompson
>
>
>                          *Andre Kramer <andre.kramer@eu.citrix.com>*
>
>         03/28/2003 10:48 AM
>
>                                
>                 To:        wsrp@lists.oasis-open.org
>                 cc:        
>                 Subject:        [wsrp] WSRP url parameters and URL
>         Encoding questions
>
>
>
>
>         In 10.2.1.1.4.1 we advice "wsrp-url" values to be URL Encoded.
>
>         However, we are silent on the remaining consumer rewriting
>         tokens and their
>         producer URL writing counterparts. But the obvious thing to do
>         is to URL
>         encode them all (i.e. if {wsrp-navigationalState} contains an
>         "&" or a "/"
>         etc then URL Encode it. Or encode it anyway just to be safe).
>
>         Having URL Encoded all parameters, for producer template URL
>         activation ,
>         the web server may even help out and do the URL decode for
> you.
>
>         Furthermore, in order to support method GET, URL templates must
>         avoid query
>         strings. One strategy is to use a path ("/") instead, but I have
>         found that
>         (after the above helpful URL decode) some Web Servers will
>         replace any "//"
>         with a "/"!
>
>         [A valid transformation for file paths as, e.g. file path a///b
>         == file path
>         a/b, but not great if one is encoding data as a path. We should
>         not force
>         consumers to use "#" or ";" instead of "?", as these also have
>         issues.]
>
>
>         Obviously this corrupts any (URL template) parameters that contain
>         consecutive back slashes, and we can not expect producers to
>         know what URL
>         structure the consumer is using for it's templates. Both the
>         decode and
>         collapsing consecutive "/"s seem valid things for the Web server
>         to do but
>         they are interacting with our method=GET work around. What could
>         we do?
>
>
>         The simplest solution seems to be to *double* URL Encode values
when
>         replacing a  {wsp-someparameter} in templates. By double encode
>         I mean
>         {wasp-paramValue} =
>         HttpUtility.UrlEncode(HttpUtility.UrlEncode(RawParamValue)) or
>         {wsrp-paramValue} =
>         URLEncoder.encode(URLEncoder.encode(rawParamValue));
>
>         [A single URL encode should be enough for consumer rewriting.
>         The consumer
>         can apply a second encode on re-writing (if required).]
>
>         This double encode does seem onerous at first, but has the
>         advantage of
>         being always safe and independent of template schemes and
>         usesMethodGet (as
>         well as constant for a Web Server environment & avoids us
>         inventing a new
>         encoding scheme).
>
>         What do people think?
>
>         regards,
>         Andre
>
>