RE: TOSCA v2 treatment of artefact, repository and nodes in general

[Chris Lauwers <lauwers@ubicity.com>]

Hi Paul,

 

Thanks for your well-thought-out set of ideas. We’d love to discuss those in more detail. Perhaps we can get started with a conference call sometime over the next week or so and then investigate if you might be interested in contributing more actively as a TOSCA TC member?

 

In the meantime, I have a number of questions and comments (in red). I know that Tal has responded as well. If our responses differ in certain areas, it is exactly because the questions you’re asking refer to aspects of the TOSCA specification that have not yet been very well defined. We’re trying to address those aspects in Version 2.0.

 

From: tosca-comment@lists.oasis-open.org <tosca-comment@lists.oasis-open.org> On Behalf Of paul.m.jordan@bt.com
Sent: Monday, October 5, 2020 11:01 PM
To: tosca-comment@lists.oasis-open.org
Subject: [tosca-comment] TOSCA v2 treatment of artefact, repository and nodes in general

 

I would like to make three inter-related comments:

 

1.       Allow get_input on various artefact keynames and on repository.url

As you know, nodes can have a property value defined either at design time or at the time they are instantiated. In the latter case this is by use of the get_input function. Nodes may also include artefacts which may be in repositories. The artefact type, file name, version, etc. and the url value of the repository can be declared at design time as a strings but in the TOSCA v2.0 cds02 it is not possible to define any of these values at instantiation time. I propose that this limitation be removed by allowing use of get_input for these keynames.

The use case for this requirement comes from a TMForum Catalyst project where a communication provider allows a corporate customer to deploy an application on a 5G MEC (Multi-Access Edge Computing) at short notice for a temporary period during a natural disaster. While the capability to deploy will be agreed in advance the precise image version and location of the state data will only be known at activation time.

Allowing these values to be input by an external party does create an additional attack vector and to mitigate this threat the input values should be validated. Therefore the syntax should allow inclusion of a constraint clause on each of the keywords which allow get_input.

 

A couple of question:

• Is information about the Catalyst project publicly available to non-TMF members?
• Allowing artifact file name, version (and any property values) to be specified dynamically at instantiation time could be useful, but allowing for instantiation-time definition of types could be problematic. It takes away a key TOSCA feature, which is the ability to fully validate service templates at design time (based on the type system). Could you provide example use cases where the artifacts provided at instantiation time could have different types?
• Is simple input validation sufficient to address the security concerns? As you probably know, ONAP imposes additional restrictions on CSAR files to make sure that any bundled artifacts have the correct checksum, and are otherwise guaranteed to be the exact versions of what was intended.

 

2.       Be explicit on whether all nodes must be capable of instantiation.

I believe that in TOSCA 1.3 there is an implicit assumption that each node in the template must be capable of being deployed by an orchestrator i.e. there is always at least one set of circumstances (due to interface operation, substitution, policy etc.) that would require the orchestrator to create an instance of the node; a node cannot exist in the template for any other reason.

 

I suggest that TOSCA 2.0 presents an opportunity to explicitly allow TOSCA nodes to be extended via relationship of type ‘extends/extendedBy’ to a node which cannot be instantiated itself, but which adds functionality or information. It may be convenient to give such extension nodes a different name or a directive to distinguish them from the classic node concept. If this proposal is rejected then I would like to see the standard make an explicit statement that all node types must be deployable.

My use case for this proposal is that I would like to extend node definitions to include metric definitions which make use of the TMForum Information Model for metrics. That model contains a powerful, but complex, set or related entities.

To date I have been achieving this by denormalizing the Metric Information Model and representing it as TOSCA data types which can be attached to a node. This has proved restrictive because:

a)The denormalization process means that the information model relationships can only be navigated in one direction once they are in TOSCA.

b) The information model contains relationships between abstract entities, but TOSCA does not allow relationships between abstract datatypes. My resolution to date has been to define relationships between each concrete TOSCA datatype but this is not scalable.

Thus I would prefer to define the metric entities as support nodes in TOSCA. I expect that there are other use cases including a more formal definition of licences and their conditions.

 

This is an area of great confusion (and much debate) in the TOSCA community, because the TOSCA specification offers little guidance here. My understanding (based on my own implementation of a TOSCA orchestrator) is as follows:

• TOSCA service templates include one topology template that includes a graph of node templates interconnected by relationships (in reality, these relationships are also templates, even though they are arguably not defined as such).
• At instantiation time, a TOSCA orchestrator first creates a “service instance model” from the topology template using a set of input values provided by the “user” (i.e. the system performing the instantiation). This “service instance model” is an in-memory or in-database representation of the actual service that is intended to be deployed in the “real world”. The service instance model again is a graph that consists of “node instances” (created from the corresponding node templates) interconnected by “relationship instances” (created from the corresponding relationship templates). Note that an orchestrator may also have to fulfil any “dangling” requirements (presumably from some sort of inventory) and decompose nodes using substitution mappings. The result is a fully-defined service instance model.
• Once a fully-defined service instance model has been created, the TOSCA orchestrator then “deploys” the service into the “real world” by
• running workflows (ideally automatically created based on dependencies in the instance model topology graph)
• that call interface operations defined on nodes and relationships
• that are implemented using artifacts

This approach leads to the following:

• When creating the service instance model, exactly one node instance must be created for each node template in a service topology template. Or, said a different way: based on current TOSCA syntax, node templates are never optional, and it is never valid to create more than one node instance in the service instance model for a given node template.
• There is of course more flexibility when running the workflows to “create” a running service in the “real world” from the service instance model. It is entirely possible for an implementation (as defined in an artifact) to create multiple external service components (“nodes”) from a single node instance in the service instance model. The number of actual service components to be created from a single node instance will likely be defined using a property value of that node instance.
• Using current grammar, it is perfectly possible to include node templates in a service template that are not intended to be orchestrated, but are only included to provide additional information to other node templates. Such node templates could be based on node types that do not define interfaces. Alternatively, such node templates could define interfaces but not define any implementations (i.e. artifacts) for any of the interface operations (which means that the workflows will effectively call “noops” for the information-only nodes).

Of course, some of this is kludgy, and some of it is too limiting, which is why we have started debating extensions to the TOSCA grammar to make all of this more elegant:

• There are many scenarios where it is desirable to create multiple node instances (in the service instance model) from a single node template, and then create one “external world” service component (“node”) from each of those node instances. The TOSCA v1.3 specification shows a VPN scenario that motivates this requirement. We’ve been debating language extensions to support this type of “node cardinality”. Such extensions could also be used to make nodes optional (similar to the way relationships can be optional).
• We’ve discussed expanding the use of the “directives” keyword in node templates to express more explicitly to the Orchestrator how node template are intended to be processed. As you know, we currently support the “substitute” directive (which specifies that a node must be decomposed using substitution mapping) and the “select” directive (which specifies that a node must be retrieve from an inventory). There are scenarios where multiple options should be considered. For example, one could consider a “create if not exists” scenario where the orchestrator first attempts to find a node from inventory, and if it is not found, it will create that node. That could be accomplished by using [ SELECT, CREATE ] directives. This would require introducing a new CREATE directive. If we’re going to introduce new directives, we could also consider a NOOP directive which would signal to the Orchestrator that no operations must be called for that node (of course NOOP nodes would still be reflected in the instance model).

 

3.       Generalise Artefact and Repository

If, from item 2, we accept that a deployable node can be extended by some other non-deployable support entity via a particular type of relationship which represents that support, then we can consider that:

a)       artefact is an extension of node

and that

b)      repository is an extension of  artefact.

I’m not sure I completely understand what you’re proposing here. A “node” in tosca is an entity whose lifecycle can/must be managed by an orchestrator. It seems to me that artifacts are more “static”. I’m not sure how this reconciles with the node concept.

 

Now both artefact and repository are very useful concepts for deploying software and, as you can see from above, I am making use of them, but the concept of V2.0 seems to be to move towards a ‘purer’ definition of nodes and relationships as vertices and edges from graph theory while shifting domain specific concepts to a separate, and non-mandatory, document. If true then there appears to be a case for including artefacts and repositories as second class entities. Their type would be derived from the support node type and in a template they would be linked to deployable nodes via a relationship specifically defined for that purpose (and probably derived from the ‘extends/extendedBy’ relationship type).

 

This feels right to me as doing so would solve the concerns raised in point 1.

 

 

Paul Jordan BT plc

 

Thanks again for your insights. Please let me know if/when you’d like to discuss further.

 

Chris Lauwers

Chair, TOSCA TC