Re: [tosca] TOSCA key_schema must derive from string

Thanks for the feedback, Denes!

It's true that if a map key is an _expression_ it could lead to an invalid map during runtime. But ... that is true for any _expression_ in TOSCA. Data types have constraints, and those constrained might not be met when the _expression_ is evaluated. In other words, you can say that a map has an implicit constraint: no duplicate keys. A proper TOSCA processor should emit an error in such a case. Alternatively, we can decide that subsequent identical keys override previous ones or similar. But it's true we need to be clear in the spec about it.

It's also true that we could allow keys to be any valid value, just as they are in YAML. But ... some of us (myself and Adam) have argued, from experience, that this is a can of worms. Many, many systems with which we interact, from databases to messaging brokers to other cloud languages do not support such maps. There is always the workaround of emitting a map as a list of tuples or translating map keys into a string (JSON or YAML), but obviously it has cons. But we can definitely reopen this issue and discuss with more feedback from the community.

By the way, duplicate keys in YAML are ... a bit subtle. From the 1.2 spec:

JSON's RFC4627 requires that mappings keys merely âSHOULDâ be unique, while YAML insists they âMUSTâ be. Technically, YAML therefore complies with the JSON spec, choosing to treat duplicates as an error. In practice, since JSON is silent on the semantics of such duplicates, the only portable JSON files are those with unique keys, which are therefore valid YAML files.

However, in practice I've found popular YAML parsers that do not error on duplicate keys! So this is something that anybody writing a TOSCA processor should be aware of.

On Fri, May 7, 2021 at 3:03 AM Nemeth, Denes (Nokia - HU/Budapest) <denes.nemeth@nokia.com> wrote:

Dear All

Â

I think we should not introduce dynamic values for keys in Tosca maps, because in that case one could define a map that has the same key twice and from the static analysis of a template one would not be able to determine if the template is valid or not.
Technically it would be nice not to force the values of the map to be string, but other primitive values (specified in YAML [int, float, â] ) should be supported.
If we would opt for complex keys, than precise definition shall be available when two keys are considered to be the same.

Â

Cheers Denes

Â

From: tosca@lists.oasis-open.org <tosca@lists.oasis-open.org> on behalf of Tal Liron <tliron@redhat.com>
Date: Thursday, 2021. May 6. 23:08
To: tosca@lists.oasis-open.org <tosca@lists.oasis-open.org>, adam souzis <adam@souzis.com>
Subject: [tosca] TOSCA key_schema must derive from string

Adam and all,

Â

I looked up the issue and just want to clarify what we decided in the past:

Â

1) A map key_schema must derive from string. It's a decision we made based on the same arguments you made, Adam, that non-string keys can be very problematic in many environments. We accepted that the main reason key_schema exists is to allow for constraints, not for supporting complex types.

Â

2) That said, YAML itself does allow for complex map keys. That's a feature independent of TOSCA so there's not much point in us arguing about its merits. We've already decided that TOSCA aligns with YAML 1.2. So, this means that we could, potentially, allow for function calls in keys by using our map-and-single-key notation. That's entirely with what's possible with YAML.

Â

Still, this is not so trivial -- the fact that the YAML spec allows for complex keys doesn't mean that all YAML parsers would do the right thing when decoding arbitrary YAML to native data structures. In fact, I suspect that most would choke on such an input. So, there's a programming challenge here if not a spec challenge. I can point to two specific solutions:

Â

1) For Go's go-yaml parser I created yamlkeys.

2) For Python's ruamel.yaml parser I created ard.

tosca message