RE: [tosca] Implementing the node/relationship template "copy" keyword

Hi Tal,

These are very good questions:

First, we should debate the usefulness of this feature. This feels like yet one more mechanism to derive/refine entities that are described elsewhere. Do we really need it?
If weâre going to keep it, then we should discuss it in the context of the current discussion about entity refinements, to make sure the same rules apply. For example, your question about lists: if a derived type defines a requirement called âhostâ that is already defined in the parent type, does this mean that the derived type wants to ârefineâ the host requirement, or add a second âhostâ requirement. My interpretation is the former, but your rules would suggest the latter. Having different rules will be confusing.

Thanks,

Chris

From: tosca@lists.oasis-open.org <tosca@lists.oasis-open.org> On Behalf Of Tal Liron
Sent: Sunday, February 16, 2020 9:55 AM
To: tosca@lists.oasis-open.org
Subject: [tosca] Implementing the node/relationship template "copy" keyword

This is what we have in the spec:

"The optional (symbolic) name of another node template to copy into (all keynames and values) and use as a basis for this node template."

Actually, this deceptively simple description raises many questions:

1. What is included in "all keynames and values"? "Keynames" is fairly clear, but I understand "values" to also mean nested values. For example, if we have a complex nested data type, do we completely overwrite the basis or perform a nested merge of the map values? E.g.:

server1:

type: MyType

properties:

mydata:

field1: hello!

field2: world!

server2:

copy: server1

properties:

mydata:

field2: other world!

The question is: should server2 have "mydata.field1 = hello!"? Neither "mydata" nor "field1" are keywords, but they are structural components of the value. So I think, yes, it is expected that this nested value be copied over. Moreover, this should happen at all nested levels:

server1:

type: MyType

properties:

mydata:

myfield:

subfield1: hello!

server2:

copy: server1

properties:

mydata:

myfield:

subfield2: world!

I think it's understood here that server2 should have both "subfield1" and "subfield2" values.

2. What to do with lists? Specifically let's look at the "requirements" keyword:

server1:

type: MyType

requirements:

- host: mymachine

server2:

copy: server1

requirements:

- db: mydb

The question is: should server2 have the "host" requirements from server1? In other words: when overriding a list, are we replacing it or appending to it? I think server2 should not have that requirement. The reason is that an element in a list is not a structural component of that value. When we specify the "requirements" keyword we do not mean "and also these requirements" but instead mean "these are the requirements". So, I think lists as a rule should be replaced, not merged together, when overriding.

Note that lists can also occur in other places, such as complex data types.

3. What about recursion? E.g.:

server1:

type: MyType

properties:

mydata1: hello

server2:

copy: server1

properties:

mydata2: world

server3:

copy: server2

It seems fairly clear to me that server3 should have the "mydata1" property from server1 and the "mydata2" keyword from server2. That means that we're not just copying from another node template, we are also making sure that node template is "resolved" in terms of what it needs to copy before we copy from it. Recursion is necessary. And that also means that a good parser should be able to detect loops and fail nicely rather than getting stuck in that endless loop (and possibly running out of stack memory depending on your implementation).

So, what I am suggesting here that "copy" is a fairly low-level mechanism. What is meant is:

"copy" is a recursive YAML copy-and-merge, whereby: 1) map values are recursively merged into existing ones, and 2) new lists replace existing lists.

That, or a more detailed variation of it, should be the description. To me it is the most comprehensible, straightforward, and least confusing way to understand what the "copy" keyword intends. This talk about "keywords and values" raises more questions than it answers.

Finally, not sure why "(symbolic)" is there in the original description. We just mean a node template name, right? Best to remove that, it's confusing.

One more note about implementation -- depending on how you architected your parser, this could be a very challenging feature to implement! For example, if you use some kind of YAML mapping parser to move directly from YAML to your programming language data structures (classes, structs, etc.) then it might be non-trivial to copy that data from another instance and then merge it recursively. You might need to do reflection.

Probably the "cleanest" way to handle this feature is in the pre-reading phase, in raw YAML. That way you can set up all the YAML in the right way and have it read as-is into your data structures. Again, depending on how you architected your parser it might be challenging to implement such a phase. (This "pre-read" phase how I've handled it in Puccini.)

tosca message