Re: [tosca] Proposal for requirement "occurrences"

Thanks Paul, this is helpful.

I'm convinced that you can achieve what you need using the 2.9.2 example, without breaking the graph.

In your topology you will have a node template representing the instance of the compute platform that you require. The node type of this template can be as abstract or as concrete as you would like. You would specify a node filter on it (again, per the 2.9.2 example).

(Note that 2.9.2 does require more work by us. For example, what happens if a node template has a required property, but then you don't want to give it a value? Instead you want to use the node filter to constrain the value. 2.9.2 has never been well thought out, which is why I proposed a different syntax. But unless we start talking about this together there will be no progress...)

As for your specific "preference to use an existing platform if one is found or to deploy one if not", well, we don't have a standard way to specify this. The "select" directive, as it described right now, seems to imply "use an existing platform only". But there's no reason why you can't invent your own directive, e.g. "select-or-deploy".

(By the way, this is a very standard approach. It's exactly how Terraform works, though Terraform does it at a nicely fine-grained level. Every Terraform "attribute" can either be provided, optional, or "computed". If not provided then it means that the orchestrator has to get it ... somehow. And this could mean deploying something new. So, different aspects of a resource can be "design time" while others are "runtime". The resource is just a box of attributes.)

I think we've got a bit entangled in the word "requirement". We often use it generally to mean "I have these requirements for my service". But then it also refers to a specific TOSCA keyword and feature that is about creating a relationship to a capability "socket" of a node. I'd like to suggest that we differentiate these terms. I'm going to, perhaps clumsily, call the general use "needs", as opposed to the "requirements" keyword.

The question, then, is how to express "needs" in TOSCA. There are actually many ways:

Node templates are the most obvious. You are expressly exactly what resources you need and what properties they should have.
We tend to assume that by default a node template expresses the need to deploy a new instance. So, we allow for directives that would modify that default behavior.
Metadata can also be added for orchestrator- or platform-specific needs.
We often need nodes to cooperate with each other, with custom relationship semantics (connections, links, dependencies, etc.). The "requirements" syntax is used to create these typed relationships. My understanding is that the relationship itself is the "need", and the syntax is just a way to alleviate the design complexity and allow for constraints on these relationships via a declarative approach.
Policies can be used to express needs for groups of nodes. Unfortunately, our policy grammar is very weak at the moment. We can apply values, but can't create rules for graph formation, which are probably the most interesting for a graph-centric language.

On Fri, Feb 4, 2022 at 12:27 PM Paul Jordan <paul.m.jordan@outlook.com> wrote:

I *think* that my use case might require something like 'dangling requirement' or 'broken graph' and so I offer it up in case consideration of a real requirement helps at all. My use case does not require 'occurrences' but the thread seems to have diverged from that.

My use case requires.

1) an abstract parent node type

2) two or more children of that parent node type which are also abstract.

3) Each of the nodes defined in 2) may have one or more child node types which represent concrete implementations.

4) Each of the node types defined in 3) has a dependency on some node type representing some compute platform. Such platforms are capable of running multiple node instances of the type defined in 3).

All the above would be defined in profiles or repositories.

Now as a template designer I want to specify a graph of node instances of the type as defined in 2). Choice of which concrete node type [defined in 3) ] could be influenced by any mix of:

Requirement to use a (new) single instance of a compute platform. This affinity requirement seems to rule out substitution.

Preference to use an existing platform if one is found or to deploy one if not. - This seems to be something like a 'dangling requirement' or 'broken graph' but I'm still not sure.

Filters on parameter values of nodes of type defined in 3). Which I believe is non-controversial.

Regards

Paul Jordan

On 04/02/2022 17:39, Tal Liron wrote:

Thanks Calin, that's quite clear.

I just want to clarify what I mean by "runtime: true".

It's true that requirements are functionally fulfilled and relationships are created at "deployment time". I did not intend to dispute that. But they should be able to be fully and determenistically validated at "design time" merely by providing inputs. The goal is to give the deployment a validated, known, well-designed graph. That's the core conceit of declarative approaches to orchestration.

I offered "runtime: true" as a compromise to satisfy Chris. Chris would like to allow some requirements to be fulfillable only at "deployment time", meaning that during "design time" (Day 0) you will not -- and cannot -- see a full graph. At design time you will see an edge going out but no vertex. So the only way to validate designs that use such a feature is to actually deploy them in a real environment (I call it the "runtime environment" as opposed to the "design environment"; that's the source of my "runtime" keyword). I don't like this feature at all, but OK, he insists it is important.

I'm also happy to find another name for it, but I don't think it should be "scope". The issue is not the source of the node representations. The issue is that there is no target node template for the requirement in the design. And there never will be. Even if the requirement is fulfilled, that fulfillment is not shown or named in the design. The node representation graph might be complete, but the TOSCA design graph is not. It's a node without a node template. Unfortunately, Chris is right in calling this feature "dangling". I don't like the word, but it does describe what is happening.

Some other potential names for this keyword:

external: true
(here we are explicitly saying that the requirement's target is external to the design; I like this one)

target: external
(this is even more explicit and maybe closest to Calin's proposed "scope"; the default value would be "internal", I guess; this might appeal most to you because you don't seem to like talking about the WHEN of the requirement, and here we are avoiding that by referring to the nature of the target node)

abstract: true
(this matches the old language in the TOSCA spec; I understand it to mean that the target node is abstract, i.e. not specified or named in the design; but some people might understand it as the requirement being abstract? not entirely clear to me but for some people it might make the most sense; possibly closest in meaning to Chris's "dangling")

fulfill: false
(we are saying: "avoid fulfilling this requirement during validation"; but could be misunderstood by reader to imply "never fulfill this")

optimistic: true
(as in "optimistic locking" for databases, whereby you assume it's safe to move on without actually locking; here we are assuming that a requirement is going to be fulfilled later and can, for the purposes of design validation, be considered valid even if it's not fulfilled)

lazy: true
(this is a why to talk about WHEN without being specific; obviously some negative connotations to this word so not so great; but is used a lot in software)

postpone: true
(again, seems like you don't like talking about the WHEN, but this one makes sense to me)

On Fri, Feb 4, 2022 at 8:46 AM Calin Curescu <calin.curescu@ericsson.com> wrote:

Hi,

Â

We spent a large portion of last year to clarify TOSCA fundamental concepts, such as profile, template, and representation graph.

The representation graph keeps the model and the state of the service on which any actions (incl. LCM) are performed.

It is formed from nodes (vertices) and relationships (edges).

The templates + inputs are just used to create and initialize the representation graph.

Requirements and capabilities are used to direct how the relationships in the representation graph are formed.

This always happens at deployment time (after all nodes in the representation graph have been created, they will be connected)

A target node could be identified (among all the nodes in the representation graph of this service or other services)

by the symbolic name specified in the template

in this case the target node will be constrained and thus planned at template design time

can be chosen only from the template nodes (of which we know the symbolic names)

by referring to a certain capability type, Âa certain node type, or by matching a certain node filter

in this case the target node will only be known at deployment time

this is a very important âlate bindingâ use-case where the model adapts to the template inputs and the environment

can be chosen from both the nodes created from this template (local scope) or nodes created by other templates (external scope)

this scope is not clearly specified in TOSCA yet, although a global scope was always expected as advertised by the TOSCA compositionality features

Â

So, for me there is no such thing as a âdangling requirementâ, I think of it Âas a âworking termâ used in our discussions to represent sometimes:

requirements that are not specifying a fixed, âsymbolic nameâ target

requirements that cannot be satisfied by âlocalâ nodes

Â

I also donât think there is a âdesign-timeâ or ârun-timeâ requirement, all of the relationships are established at deployment time.

Â

Different use-cases might need that requirements are satisfied:

locally (in the same template)

for relationships that must be internal to the service

for local completeness

note that this does not mean that a node-filter may not be used

externally (outside of the template)

for relationships that must be external to the service

for service compositionality

for service de/composition via substitution

either local or external, just that they are satisfied

Â

I also donât think there is such thing as a âbrokenâ representation graph. It is either connected or not connected to nodes of other services, and both are relevant as shown above.

Â

I donât think we can replace requirements with node_filters by using the âselectâ mechanism:

will not work if I want to use node_filters for requirements on âlocalâ nodes

will not work if I want the target to be a capability type and not a node type

will not work in substitutions

Â

BR/C

tosca message