Re: [amqp] Groups - AMQP Addressing Working Draft 6 uploaded

On Fri, 16 Nov 2018 at 22:48, Clemens Vasters <clemensv@microsoft.com> wrote:

My proposal already covers all of these cases in the ABNF. I have a relative node/terminus address and one that just covers scope and node.

If you seeÂ

amqp://onramp/(scope)/fooÂ

and you are already inside the AMQP network, you can completely ignore the scheme and authority.Â

Thinking of a reply-to... if the above is the case, then we are saying that scopes are globally unique (the on-ramp is unimportant) and globally reachable.Â In this case all on-ramps must be equivalent and thus as long as you know of one on-ramp then you don't really need to know any others.Â If this is the case I'm not sure why we would ever send on-ramp info over the network.Â Conversely if scopes may only be available over certain on-ramps, or there are non-connected networks where the same scope name may be used for different actual domains, then the on-ramp is important and can never be ignored.

If the URI pops out of the network, you need those again.

As above - do you need an on-ramp, or does it need to be that particular one?

-- Rob

If you need an abstraction for the on-ramp address because that looks different in different places, DNS offers all the tricks you need, IMO.

Clemens Vasters // Architect // Microsoft Azure Messaging

Von: Alan Conway <aconway@redhat.com>
Gesendet: Freitag, November 16, 2018 10:35 PM
An: Clemens Vasters
Cc: oasis-amqp-list
Betreff: Re: [amqp] Groups - AMQP Addressing Working Draft 6 uploaded
Â

On Fri, Nov 16, 2018 at 11:10 AM Clemens Vasters <clemensv@microsoft.com> wrote:

It appears that you are now merely objecting to my syntax proposal and not to the addressing elements.

No I'm not pushed about the syntax - I want to distinguish between "on-ramp" and "scope" such that:

1. They can be the same if they are both the same IP/DNS authority - "ampq://service.com/foo" means just what it looks like - on-ramp=service.com, scope=service.com

2. They can be different, in which case scope is not required to be DNS.

3. Relative URI references are allowed as AMQP addresses: //scope/path, /path, path, and are resolved in a system-specific way.

4. On-ramp information may not be globally meaningful (localhost:5672, my-firewall-proxy:8888) so it MUST NOT be included in AMQP addresses or confused with scope. However the solution MUST allow both on-ramp and scope in an AMQP URL.

Consequence of 2 and 4 is that AMQP addresses are not always URLs, and AMQP URLs are not always addresses. In the simple case of authority==scope they overlap.

YouÂ brought in the interesting notion of multi-scheme AMQP addresses - not sure what to do with it but I like it so I"ll keep it.

You still have an overlap in your model because I canât tell what

//example.com/fooÂ

is and what concept of âdomainâ that reflects. Do I look at DNS or do I look at something else?

Deliberate - it is determined by the system. IÂ think we need both global/absolute addresses and "relativeÂ URI references""Â that are interpreted relative to some system-defined or local defaults, e.g.:

//example.com/queues/bob

/queues/bob

bob

What the are relative to, and whether or not they refer to anything is up to the container you are immediately connected to. This reflects current practice for direct client/broker connections (address == queue name or somesuch)Â and allows "fan-out" routing where a router distributes work transparently to different back-ends based on path matching rules. Note the // form does have a clear "message scope", as the URI authority, but doesn't tie it to DNS or any specific technology (and doesn't exclude any such technology either). If you want it tied to DNS then use amqp://example.com - or whatever other schemes we might choose.

Better examples, simple cases first: on-ramp == IP/DNS authority, or no on-ramp:

amqp://domain:port/path - is both a URL and an AMQP address. As URL it means connect to amqp://domain:port. As an address the verbatim string "amqp://domain:port/path"Â can be used in to/reply-to/link address. It means use the messaging scope "domain:port" for routing, and let the final destination interpret the string "amqp://domain:port/path" as a node. Typically the final destination would just use the /path, but it can't be prevented from looking at the rest of the address.

//scope/foo - is an AMQP address (relative URI reference), but not a URL. . It can be routed using the "scope" information, but can't be used to make a direct connection without external on-ramp info. Probably the message will circulate in an all-amqp system where all participants already know their on-ramp or can deduce it from the scope. To pass it out of that system would require forming an amqp-via:// address with an externally visible on-ramp -see below

/foo or foo - is an AMQP address, not a URL. Any entity that processes such an address can decide whether it names a local node, should be resolved in terms of a default scope, should be routed to another entity etc.

Now for the amqp-via scheme: URLs whic are NOT amqp addresses but contain an AMQP address (with some URL-required encoding). They include on-ramp information in the usual way, as the URL scheme://host:port.

The optional scheme://scope:port parts of the contained address are encoded asÂ (scheme$scope$port) in the first path component.

/(scheme$scope$port) => scheme://scope:port/

/(scheme$scope) => scheme://scope/

/(scheme) => scheme://

/($scope$port) => //scope:port/

/($port) => //:port/

/foo => /foo - if the first component doesn't start with (

Note if we wanted to reduce the number of special chars we could also use /$$scheme$scope$port - I'm happy with either.

Examples:

amqp-via://localhost/($scope)/foo - AMQP to localhost:5672, use address //scope/foo in to/link address. This address isn't a URL but can be used to form one by adding locally-valid on-ramp information, e.g. amqp-via://192.168.1.1:888/$(scope)/foo

amqps-via://localhost:8888/(amqp$domain$port)/foo - AMQPS to localhost:8888, use amqp://domain:port as AMQP as message/link address.Â This address IS a URL and can be used to reconnect to the system - not via the localhost:888 on-ramp, but via some internet wide amqp://domain.port on-ramp. Could be a common scenario for systems that use local on-ramps to evade firewalls or benefit from routing, but where messages will eventually end up on the open internet - where there is now a simple way to reply.

amqp://localhost/foo - this is a trick but possibly a useful one. This is a URL and an address, it assumes that every participant in the system is deployed with an AMQP sidecar at localhost:5672 - not uncommon in HTTP cloud deployments. O for r not.

http://authority/foo, mailto://foo@bar etc.... any address that does not start with an amqp:, amqps:, amqp-via: amqps-via, or does not look like a relative URI reference (which I think basically means it has an unknown scheme: prefix)

are taken verbatim as string addresses. An AMQP endpoint MAY match themÂ as simple strings, MAY reject them or MAY know how to perform scheme-specific actions. AMQP routers that don't recognize the scheme SHOULD pass them along verbatim or reject them according to their configuration.

Â

There are basically 3 modes of operation:

1. All internet (not necessarily public): scope==DNS domain, URL="" Every address can be AMQP-routed by or used to connect.

2. All AMQP: clients know their own on-ramps, all AMQP addresses are //scope/path. Scope can be interpreted any way the AMQP routing system wants.

3. Mixed - need to form/decompose AMQP-via URLs at the boundaries.

Â

Von: Alan Conway <aconway@redhat.com>
Gesendet: Freitag, November 16, 2018 4:29 PM
An: Clemens Vasters
Cc: oasis-amqp-list
Betreff: Re: [amqp] Groups - AMQP Addressing Working Draft 6 uploaded
Â

On Fri, Nov 16, 2018 at 5:38 AM Clemens Vasters <clemensv@microsoft.com> wrote:

The scheme is always relevant because it distinguishes the kinds of URIs you may be looking at.

Â

Since we have no constraint today on what âaddressâ means, a reply-to may quite well contain âmailto:clemensv@microsoft.comâ or âhttps://example.com/foobarâ, meaning the ultimateÂ

consumer is asked to use a different reply path that doesnât use AMQP, and that may be so because the sender sends its messages via an intermediary and only that intermediary speaks AMQP downstream.

Interesting idea but I'm not clear how it works.ÂYou're saying an AMQP "address" (the string that goes in to/replyto/link address) is an *arbitrary* URI, and that the recipient of an AMQP message is expected to be able to speak the required protocol of a reply-to message in order to reply?

Even if everything is AMQP, you might have situations where youâll want to force a message consumer to establish a connection for the reply-path with a particular container through the external IP interface and you would therefore provide an absolute URI, complete with scheme, in the reply-to.

Â

What youâre doing below is to overlay the exact same construct, the authority portion, with two entirely different concepts: Internet global and private global. Thereâs no way to tell what is what by looking at it and you can also not tell by doing a DNS lookup because a âprivate globalâ address may just clash with an âInternet globalâ address by accident.

Â

Youâre also scoping out the concept of âonrampâ for the private global addresses to some external âconfigurationâ â which is exactly one of the core problems that needs a concrete solution. I need to be able to give someone an AMQP address (target container and node) along with a suitable onramp gateway, and a URI is an appropriate container for all that information.

Â

You canât collapse gateway and scope, because the gateway and the scope you want to reach might be two completely different places (different systems, one addresses via IP, one addresses logically) and you need to be able to identify both separately, but in the same _expression_.

This comes down to the question of "address" vs. URL. An address is carried on a message or link addresses. A URL includes an address and the means to connect to the resource or some IP host that can forward to the resource. In the case of a URL, I'd like URL == address (I do feel that the distinction amqp: vs amqs: is a local configuration issue and NOT part of the routable address. Someone using such an address from reply-to should not be constrained by the choice of TLS or not made by the originator. That's why I proposed omitting the scheme - but I'm intrigued by your http:/mailto notions so I can rethink that.)

ÂIn the case of a non-connectable URI, the address *cannot* contain the on-ramp because the correct on ramp depends on who you are. My on-ramp might be127.0.0.1:8888, yours might be 192.126.1.1.

Neither of those can be part of a routable, protocol-neutral "address". They could be part of a URL that *contains* an address, but not part of the address itself that is propagated in a message to: field. Hence I characterize that as "configuration". Now of course we can design a URL scheme where we can embed such configuration BUT the client has to know that such details must be removed before before embedding the address in an immutable message field.

So how about this:

amqp[s]://domain/path - is a URL and an AMQP address. Processes using this URL to connect MUST use the designated scheme (amqp/amqps)

//domain/path - is an AMQP address, but not URL. It MAY be possible to use it as a functional URL by pre-pending amqp: and/or amqps:. Alternatively the domain may be a unique messaging "scope" without any IP mapping. Note it is trivially easy to manufacture such scopes in any organization that owns a domain - just reserve one non-domain name (e.g.amqp-sopces.qpid.apache.org) and then use whatever uniqueness authority you want to generate unique scope namesfoo.amqp-scopes.qpid.apache.org etc. In a system that never uses AMQP addresses as URLs, the issue of clash never arises. If you want to assert that an address can be used as a URL, you MUST use the amqp[s]://domain/path format.

/path, path - is an AMQP address. It does not imply any domain. A broker or server can interpret it directly as a queue, topic etc. A router mayÂresolve it relative to a default URI, relative to a the virtual-host of the incoming connection, using path-matching rules or by any other means it chooses.

amqp[s]-via://domain/(scheme2:domain2)/path - this is a URL, but NOT an AMQP address. The user of such an a URL SHOULD connect to amqp[s]://domain as normal. It MUST transform the URL into the following address [scheme2:][//domain2]//path and use that as the AMQP address in to/reply-to/link address. This allows an "on-ramp" IP-connectable URL to contain a complete AMQP address, possibly using a different scheme and/or domain.

Other URI formats (mailto: http: etc.Â can be used as AMQP addresses, and can be embedded in amqp-via; URLs - the meaning of such URIs to an AMQP client/server to be expanded on.

Note we can syntactically chain embedded addresses amqp:((foo:bar/baz)/x:y)/path, I'm not sure if that is desirable or not.Â

Â

Â

Â

Von: Alan Conway <aconway@redhat.com>
Gesendet: Freitag, 16. November 2018 00:14
An: Clemens Vasters <clemensv@microsoft.com>
Cc: oasis-amqp-list <amqp@lists.oasis-open.org>
Betreff: Re: [amqp] Groups - AMQP Addressing Working Draft 6 uploaded

Â

Here's a short write-up of my thoughts on merging "authority" and "scope". Essentially a non-DNS Clemens scope corresponds to a "private global address" as outlined below. This is very abbreviated, not counter-proposal just thoughts for discussion.

Â

# AMQP URIs and addresses

Â

An AMQP URI is a [hierarchical URI](https://tools.ietf.org/html/rfc3986#section-3) with one of the following schemes: amqp, amqps, ... (others from draft 6)

Â

An AMQP *address* is a [relative URI reference](https://tools.ietf.org/html/rfc3986#section-4.2) that MUST NOT have a scheme part. The RFC defines 3 kinds of reference:

Â

1. "Network-path": //authority/path[?query]

2. "Absolute-path": /path[?query]

3. "Relative-path": path[?query] (no leading /)

Â

AMQP addresses are used in message `to` and `reply-to` fields, link source and target `address` fields, and any other place where the AMQP spec requires an "address". Addresses MUST NOT start with a scheme prefix like "amqp:" or "amqps:".

Â

Network-path addresses (//authority/path[?query]) are termed "global" addresses". Absolute-path and relative-path addresses are termed "local addresses".

Â

An AMQP URL is an AMQP URI where the authority part can be resolved to an IP address for example a DNS name or IP literal, with or without a :port.

Â

## Local addresses

Â

A local address is of the form `/path[?query]` or `path[?query]`

Â

AMQP applications within a single organization typically will use local addresses. Local addresses can be routed by AMQP routers for flexible deployment, load balancing and so on.

Â

An AMQP URL can be decomposed into connection information and a local AMQP address. The local address is simply the "/path" part of the URL.

Â

Local addresses do not by themselves allow you to make a connection, you need to connect to a route or broker that can process the address.Â Typically this will be to a co-located "sidecar" or a locally-configured default router URL.

Â

## Internet-global addresses

Â

An internet-global address is of the form `//authority/path[?query]" where the authority can be resolved as an IP address. Such an address can be converted to or from a URL by adding or removing an "amqps:", "amqp:" or other suitable scheme.

Â

An internet-global address can be used to route messages within an AMPQ network, since the authority is available to the AMQP routers to make routing decisions. It can also be converted to URL, passed to non-AMQP applications, and used to re-create a connection to the AMQP network later.

Â

Internet-global addresses can be used for public internet-scale applications, but also for private applications that use internet tools like DNS or IP load-balancing, virtual host routing and so on.

Â

## Private global addresses

Â

A private global address is of the form `//scope/path[?query]` where the authority part does not correspond to a DNS or IP address, but to some privately-controlled unique "scope".

Â

In this case, all endpoints must be configured with suitable "on-ramp" DNS or IP addresses, or co-located with a suitable sidecar to make an initial connection.

Â

Once a connection is made, AMQP routers can use the authority information to route messages. There's no need to convert private global addresses to URLs since all participants in the system know how to connect to an "on-ramp".

Â

# JustificatioMy claim is that there are 3 cases to consider:

Â

1. Existing AMQP systems manage fine with just a free-form path-style address, including routed systems like dispatch router.Â They should be able to continue using URLs of the form amqp://connection-info/amqp-address, with free-form paths for routing, and using virtual-host for orthogonal scoping/tenancy etc.

Â

2. Global addresses that are both connectable AND routable have obvious appeal.Â In this case scope==authority==DNS/IP. This fits with virtual-host based routers since a router can qualify a local addresses by using the virtual host as the authority in a global address, thus a client that connects toÂ vhostfoo.net and uses addres /x and a client that connects to //foo.net and uses address //foo.net/x can both be routed to the same global address.Â (This is for link addresses, immutable message addresses have to be global from the get-go if they need to be, as they can't be modified however read coming rant on the proper use of link and message addresses :)

Â

3. Global addresses that are routable but NOT connectable have obvious appeal for systems that don't use DNS/IP/virtual-host approaches to identify an "authority" as part of their routing strategy. For them the authority (aka scope) can mean anything they want.

Â

This proposal covers all 3.

Â

_____________________________________________________________________________

Red Hat GmbH,Âwww.de.redhat.com,
Registered seat: Grasbrunn, Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Paul Argiry, Charles Cachera, Michael Cunningham, Michael O'Neill

amqp message