RE: [tosca] TOSCA orchestration and tooling directives

[Chris Lauwers <lauwers@ubicity.com>]

Hi Tal,

 

In our meeting on Tuesday of this past week, we discussed whether âdirectivesâ really belong in the standard, since in theory all the keynames in the language are âinstructions for orchestration and toolingâ. Perhaps we should repurpose the concept of âdirectivesâ explicitly for those instructions that are non-standard, and use keynames exclusively for standard features. If you agree with this, then we should introduce a new keyname to capture whether a node is âselectableâ or âsubstitutableâ.

 

What do you think?

 

Chris

 

From: tosca@lists.oasis-open.org <tosca@lists.oasis-open.org> On Behalf Of Tal Liron
Sent: Wednesday, February 23, 2022 1:03 PM
To: tosca@lists.oasis-open.org
Subject: [tosca] TOSCA orchestration and tooling directives

 

Thank you everyone for helping us move forward.

 

To follow up on our discussion, I'd like to attempt to consolidate the conventional directives.

 

There's not a lot in the spec about directives, but here's how they are described in [3.8.3.1]:

An optional list of directive values to provide processing instructions to orchestrators and tooling.

 

That seems right to me. The "and tooling" addition is important, as directives could potentially be processed by various components along a toolchain that eventually reaches an orchestrator (or not). But we need to add some more information:

1. We need to clearly state that order could matter. For example, if there are multiple directives related to node representation association and processing one of them has a satisfactory result then the subsequent ones could be ignored. Or, perhaps two contradictory directives could create ambiguity and would result in an error. The actual semantics are implementation- and directive-specific.
2. We should state that "directives" can appear not only in a node template but also in a requirement (a new place for them, as we've suggested) where they are considered to be applicable to the target of the requirement. The question then arises: what if we have explicit directives both in the requirement and in its target node template? The answer is that it depends on the directive's semantics. For example, a contradiction may lead to an orchestration error. Relatedly, some directives may not be allowed to be used in conjunction with others.
3. Finally, there's the question of default behavior for orchestrators and tools. If a user does not specify a directive, does that mean that its functionality should be considered "turned off"? I'm very happy with Calin's proposal that we do not specify default behavior. All we're doing with directives is suggesting conventions. This unburdens the TOSCA grammar from having and enforcing a potentially unwelcome opinion.

In TOSCA 1.3 there are exactly two directives:

• "substitute"
• "select"

I think we need to change these conventions, considering our recent discussions:

 

"optional"

 

Specifies that node representation(s) are not absolutely necessary. Note that this covers a lot of ground. The directive specifies neither priority ("nice to have"?) nor conditions for making the decision. It really conveys just one thing: if node representation(s) are not created then it should not be considered an error, and deployment should continue. There might, of course, be other errors due to the loss of these node representation(s). It's just this specific error that would be skipped.

 

For requirement targets the logic is quite clear: if there are no target node representation(s) then no relationships will be created. They are thus "optional". But I don't see why it wouldn't make sense for any node template, not just a requirement target. In fact, I've had multiple use cases where I've needed something like this and I've solved it with a boolean property on the node template. The nice thing about using a directive instead is that it doesn't clutter the model with orchestration-specific properties and indeed it can be applied to any node template (or requirement) from any imported profile.

 

I see no issue in different combinations of "optional" directive usage in both the requirement and the target node template. Example:

 

node1:

  type: Cat

  requirements:

  - req:

      node: Fish

      directives: [ optional ]

 

node2:

  type: Fish

  directives: [ optional ]

 

For "node1" we are saying: don't emit an error if we can't find target representation(s). For "node2" we are saying: don't emit an error if there are no representation(s). Let's assume that in this case we don't have a source for "Fish" representation(s) other than "node2". This means that if "node2" doesn't end up having representation(s) then no errors would be emitted at all.

 

"external"

 

Specifies that the node representation(s) would come from a source other than this service. Again, this covers a lot of ground: the source could be an inventory, a pre-existing (used or unused) resource in the cloud, another orchestrator, and also -- in my opinion; see below -- registered substituting services (a "catalog"). Again, we're not telling the orchestrator from where exactly these representation(s) should come. What we are conveying here is that it is impossible to determine the existence of such representation(s) without access to those external sources.

 

The impact I'm interested in here is for Day 0 validation tools, like Cloudnet and Puccini (when used without an orchestrator). This directive, when in a requirement, lets these tools know to skip trying to determine the validity of this requirement. Because they are not connected to external sources (at least not in Day 0) they cannot know if the requirement is satisfiable or not.

 

When used on node templates this directive replaces both the "select" and "substitute" directives from TOSCA 1.3 (see example [2.9.2]). My opinion is that substitution should not be singled out with its own directive, at least not conventionally. Otherwise we get into the impossible task of enumerating all the diverse kinds of external sources that would be available. That said, implementations can add their own custom directives if they want to allow designers to select a source. I would suggest that they should be used in conjunction with "external". For example, let's specify a list of potential sources in order of priority:

 

directives: [ external, _catalog, _pool, _cloud ]

 

(Note: Chris, you said that directives cannot be user-defined. However, I looked through the spec at different versions but can't find any mention of such a restriction. The closest is section [3.4.3], which lists which directives are defined. But it doesn't say that others cannot be defined. I think we should clarify this language -- to explicitly allow custom directives. I see no reason why we shouldn't. An unsupported directive can be easily ignored by a tool. Also, I am here marking the custom directives with an underscore prefix. This is just a proposed convention.)

 

The suggested semantics for combining the "external" directive in both the requirement and target node template is that they must match. For example:

 

node1:

  type: Cat

  requirements:

  - req:

      node: node2 # node template name

      directives: [ external ]

 

node2:

  type: Fish

  directives: [ external ]

 

Here "node1" requires a specific target to have external representation(s). Thus, if "node2" did not have the "external" directive, I would expect the orchestrator to emit an error.

 

Relatedly, Chris would like "external" to be the default for all requirement targets in Ubicity, unless they specify a node template by name. That's fine. As stated above, not specifying a directive explicitly doesn't mean that its functionality is turned off. Behavior depends on the implementation. But actually I think Ubicity should do something more sophisticated. For example, what if the named target has an "external" directive? Example:

 

# In Ubicity

node1:

  type: Cat

  requirements:

  - req:

      node: node2 # node template name

 

node2:

  type: Fish

  directives: [ external ]

 

In this case, perhaps Ubicity should consider the requirement to have an implied "external" directive despite having a named target. In other words, if a target is named then its directives would be used (and perhaps requirement directives should not be allowed if they conflict).

 

Special note for substitution mapping: it could be that all mapped requirements should be assumed to have an implied "external" directive. There are some other issues here, so I'd rather we discuss that separately.

 

"internal"

 

This directive has a special semantic relationship to "external". If only one of these directives is present, it implies that the other is "turned off". If both are present, the order in which they appear determines the order of preference. Here are the possible combinations:

 

directives: [ external ] = use external representation(s) only.

 

directives: [ internal ] = use internal representation(s) only.

 

directives: [ external, internal ] = use external representation(s), and if not found use internal representation(s). This one is actually a bit subtle. Depending on the implementation, "external" might never conceivably fail. For example, in a cloud environment with potentially endless resources a new VM can always be spun up, so we would never get to using "internal".

 

directives: [ internal, external ] = use internal representation(s), and if not found use external representation(s).

 

directives: [] = no stated preference, so it depends entirely on the implementation.