Re: [tosca] More on dangling requirements

We remain in disagreement over this.

On Tue, Nov 17, 2020 at 1:16 PM Chris Lauwers <lauwers@ubicity.com> wrote:

Yes, and presumably the Application node type already defines the âdataâ requirement to ârequireâ a node of type âDatabaseâ, in which case you donât even have to specify the âDatabaseâ type here. In addition, you will likely also want to specify a node filter to âfilterâ the set of Database nodes that will be considered to fulfill this particular requirement. The node filter (together with the node and capability types specified in the requirement definitions) define the âqueryâ that you will run against your inventory to find the set of suitable nodes for fulfilling this requirement.

How could a node filter satisfy the complex provisioning logic I detailed below, e.g. in the policy?

In the past we saw proposals for very complex branching-logic additions to node filters. Those would not only be very challenging to parse, but also they are very partial, because they only indeed apply to branching-logic. I don't see how we could ever create a grammar that would reflect how all orchestrators work. Some orchestrators would be able to do more (and we won't be able to express those in TOSCA) while other orchestrators do less and will have to be non-compliant.

ÂIf the requirement is mandatory (i.e. the âoccurrencesâ keyword in the requirement definition has a lower bound that is greater than zero), then the orchestrator will find a suitable node during the requirement fulfillment phase, and create an edge between the node that has the dangling requirement and the node from inventory that was used to fulfill the requirement.

A single number is insufficiently simplistic. In any case, I am confused as to what you intend "occurrences" to mean. If you are agreeing that it has only to do with the instance model, but aren't we talking about runtime provisioning?

You can absolutely do a âget_propertyâ to refer to the database. Since your template specifies that the target node for the âdataâ requirement is of type Database, your parser can validate that âvalidâ property values are being retrieved.

How would we do this? What is the entity name? I.e. { get_property: [ ???, ip_address ] }

I donât think there is confusion either way: if a dangling requirement is not mandatory, it will not result in an edge in the instance model. If it is mandatory, it will result in an edge in the instance model (or the orchestration will fail if a suitable target node cannot be found).

Whether or not it's mandatory could be entirely up to the runtime environment. Moreover the exact number of occurrences might not be known during design time. And then how many "occurrences" are mandatory or optional or preferred might depend on runtime factors.

TOSCA doesnât specify how the orchestrator is supposed to provide the (inventory) database, so Iâm not sure what additional flexibility is needed?

If not in TOSCA, how do you see this configured? How can I say that for a specific node in a specific service template I have specific (and conditionally complex) requirements?

The problem with policies as you use them is that they have absolutely no (language) semantics associated with them. All the semantics are encoded in the properties, which means that they can only be processed by an external domain-specific entity that knows what these properties mean.

That is absolutely correct, and is the core of our disagreement. I'm afraid that the dangling requirements dictates a runtime "selection" policy for orchestrators. I can assure you that it won't work for any of my use cases. If it ends up being a TOSCA feature, I will simply not implement it and will have be non-compliant.

The idea of a "pure" TOSCA orchestrator is being taken too far here for me, because you seem to be dictating a very specific "selection" algorithm for all orchestrators. Would it really be impossible for you to have a "ubicity" profile which users can import and then gain access to specific behaviors of your orchestrator? I don't see how one size can fit all.

ÂAs I stated earlier, the âoccurrencesâ keyword in the requirement definition specifies whether the requirement is mandatory or optional. Do youÂ have example where âconditionally-optionalâ should be used?

Example: "Require databases if running on an x86 platform. Require two databases in redundancy mode if running with 16 GB RAM or higher. On all other platforms consider databases a nice-to-have soft requirement." Real world examples tend to be much more complex than this.

And it can get more complex: perhaps later on in the runtime lifecycle (day 2) a database becomes available, and because it's a "nice to have" suddenly there would be a non-zero correspondence between the instance model and the world.

Â

Iâm not sure I understand what you mean here.

Imagine that initially the cloud cluster does not have capacity for a database. But, a week later it is expanded and suddenly has capacity. So, now the node template can be instantiated. (The requirement has already been resolved in the design phase.)

If youâre talking about âselectionâ (i.e. requirement fulfillment i.e. find a node from inventory), then this is exactly what the âexpandedâ node filter syntax is for that we proposed several months ago. However, you use the word âprovisionâ in your example, which is different from âselectionâ. Assuming you âprovisionâ using substitution, then a substituting template for a virtual machine presumably will be different from a substituting template for a bare-metal server, and each substituting template will specify how much memory it needs.

"Selection" is simply too minimal a description for what happens. The process of turning a node template into something real (zero or more "things") will always go beyond what TOSCA grammar could provide.

I think TOSCA currently supports all of this: a TOSCA orchestrator performs requirement fulfillment, or substitution. Both of these functions require âdecision logicâ to find the best node to fulfill a dangling requirement, or the best template to substitute an abstract node. If a TOSCA orchestrator wants to use AI/ML to help with this decision logic, it should feel free to do so.

OK, but how would you model this logic in TOSCA?

However, based on your examples in this email, Iâm not sure we need to make any changes to the current âmodelâ for how a TOSCA orchestrator is expected to work.

Hard "no" from me on this.

Of course, I can do all the stuff I mentioned in the email already, and do. However, if TOSCA grammar allows for dangling requirements -- this is something that I will have to explicitly not support. My policy for "selection" looks nothing like node filters, and is indeed an area of constant innovation where I try new approaches and discover new orchestration challenges that require new thinking. E.g. I have one selection where I actually run a whole Ansible playbook just to gather the data needed to make the selection. I think it's not a good move for TOSCA to dictate orchestration behavior and will continue to argue against such attempts.

tosca message