RE: Differentiating TOSCA from HEAT

["Bruun, Peter Michael (CMS RnD Orchestration)" <peter-michael.bruun@hpe.com>]

Chris, and all,

 

We absolutely agree that the clone-and-modify option is bad, and that hackability is bad. I like the âglorified install scriptâ way of putting it. I cannot find any comments in red below, that I disagree with.

 

If everyone here also agrees, and then we can close this discussion.

 

But remember the demo using TOSCA for chemical plants? They were using TOSCA as a day 0 tool to design each plant separately â the same template never really got instantiated more than once. So features involving inputs would be of little interest or value.

 

So the opinions of Chris and myself are not the only tenable ones. Just because I think that clone-and-modify is a bad paradigm in my world of high-volume-telco-customer-facing-services, doesnât prove that it is bad in all worlds.

 

Being on the same page, however, is a necessary (but not sufficient) condition for having meaningful and convergent discussions instead of running in circles.

 

If we do agree, then we have a hard requirement for the language design:

 

All potential intended variability of each type of service in the catalog must be easily expressible as a function of the inputs. Note that this creates two levels of âintent basedâ, and I am not sure that was clear in the previous work on TOSCA for intent based modeling:

         The intent of the Day 0 designer to express in as few templates as possible the intended variability offered to the Day 1+ users

         The intent of the Day 1+ users, expressed by a) selecting a template from the catalog and b) providing input values

 

If the expressive power of inputs is too low (or too hard to use), then the users will begin to spawn variants of the templates for those cases that are not expressible. I hope we all agree that that is undesirable.

 

Alternatively, we face another bad scenario, that to express the intended variability, users invent their own home-grown formats to use as input to some TOSCA-generating scripts. That road, in my opinion defies the purpose of TOSCA as a standard.

 

Opinions?

 

Peter

 

 

From: Chris Lauwers [mailto:lauwers@ubicity.com]
Sent: 27. januar 2022 23:01
To: Bruun, Peter Michael (CMS RnD Orchestration) <peter-michael.bruun@hpe.com>; Tal Liron <tliron@redhat.com>
Cc: tosca@lists.oasis-open.org
Subject: RE: Differentiating TOSCA from HEAT

 

Thanks Peter. Just to make sure I understand your definitions correctly: both âcatalog-basedâ and âclone-and-modifyâ orchestration use TOSCA templates, but whereas the âcatalog-basedâ templates define inputs to express per-customer-instance variability, âclone-and-modifyâ templates do not use inputs and instead hard-code per-customer-instance values directly in the template?

 

More comments inline

 

 

From: Bruun, Peter Michael (CMS RnD Orchestration) <peter-michael.bruun@hpe.com>
Sent: Monday, January 17, 2022 11:25 PM
To: Chris Lauwers <lauwers@ubicity.com>; Tal Liron <tliron@redhat.com>
Cc: tosca@lists.oasis-open.org
Subject: RE: Differentiating TOSCA from HEAT

 

I think this is because we are on the same page, but the discussions and the uses I see in the field reveal that not everyone uses TOSCA that way.

 

I distinguish sharply between the templates in a catalog and the instantiations created from templates.

 

In catalog based orchestration, there is a sharp distinction between the actors who define the templates in the catalog and those who instantiated those templates to create a service. Particularly, to call it catalog based, the latter will not create or modify TOSCA templates.

Yes, this is the distinction between Day 0 (profile and service design) and Day 1 and 2 (service deployment and service management). Different tasks and as a result different skillsets.

 

It is possible and meaningful to use TOSCA as a tool for designing instances. A uses picks a template, adds a few nodes and relations and instantiates (aka. clone-and-modify). Maybe the altered template is even pushed back into a repository of templates.

I agree that this is possible. I just question if this is useful. This uses TOSCA as a glorified install script, not as an orchestration language.

 

It is also possible and meaningful to âhackâ a workflow/plan/runbook generated by an orchestrator, as Tal mentions in the mail from January 6, 2022:

 

There are various reasons why Heat isn't great. The relevant one for our group is: it creates its worfklows automatically for you but there is almost no visibility into them, and definitely no hackability.

Visibility is the task of tooling. Hackability defeats the whole purpose of automation. If something needs to be changed, change the template in the catalog and update/redeploy. Donât change the âinstanceâ or âworkflowâ generated from that template.

 

But in âcatalog based orchestrationâ, the actors who instantiate services:

         Do not write (or even necessarily know) TOSCA

         Do not hack any part of the orchestration process after deciding to deploy a template with some inputs

 

So like âcloud-native orchestrationâ, I see âcatalog based orchestrationâ as a restricted part of the general concept of âorchestrationâ, where the restriction comes at a cost and with some benefit. I believe I outlined the benefits below.

Yes, my assumption has been that this is exactly the focus of TOSCA. TOSCA is a language for automating lifecycle management. The whole point of automation is to achieve autonomy (i.e. to get human operators out of the loop).

 

The problem with catalog based orchestration is highlighted by Talâs second mail from January 6:

 

From the perspective of a from-the-trenches engineer: there's nothing more frustrating then a big automatic system that does a zillion things for you, but then can't do the simplest thing that you can do in a command terminal in one minute.

Welcome to Kubernetes 😊 One big automatic system that does a zillion things for you, but provides no visibility and no hackability.

 

There are two ways to go from there:

1.       Abandon the catalog based approach, and embrace that TOSCA is used for designing and customizing specific instances

2.       Ensure that the TOSCA language is expressive that the catalogued flexibility can be sufficient for users not to become frustrated

I donât really see the value in option 1. There are plenty of tools (Ansible comes to mind) that can act as âdeploy scriptsâ for that use case. Iâm not sure why people would use TOSCA as an alternative here. It doesnât leverage the real power of TOSCA, which is in the automated orchestration features.

 

We should focus instead on collecting feedback on where the TOSCA language needs to be extended to provide this âcatalogued flexibilityâ. Variable cardinality of node instances is one obvious issue, but as you know there has been some pushback against this feature.

 

Both ways are respectable, and I am perfectly happy with both, but they address different orchestration needs.

 

I am convinced that some of the discussions we have been having on input parameters and cardinalities come from not fully spelling out that there are those two different uses of TOSCA, and the requirements for the expressive power of the TOSCA language are not the same for the two approaches.

 

What I see in the field is that TOSCA users are failing with TOSCA because they are mixing the two approaches â in practice they use TOSCA as an instance design tool, but they also insist on all TOSCA templates being catalogued. The result is a catalog containing thousands of templates, each one expressing the topology needed for one specific instantiation.

I donât really see this as a âmixing and matchingâ. Instead, what this proves in my opinion is that use of TOSCA for Option 1 is doomed to fail. TOSCA will only be successful if people use it as intended. This means:

-          As a âcatalog-basedâ orchestration system that allows for per-customer-instance variability

-          With support for maintaining and managing instances generated from templates. I generally refer to this as âmodel-driven orchestrationâ: all orchestration actions are performed on these instance models (the ârepresentationsâ) first, and then propagated to the external platforms using interface operations that are invoked by the orchestrator.

-          Fully supporting deployment-time requirement fulfillment to âintegrateâ new services into existing infrastructure or into other services that may have been deployed previously. As you know, system integration is typically the most expensive part of new service deployment. TOSCA requirements help automate this type of integration.

-          Fully supporting substitution mapping to decompose abstract services recursively into device configuration on available resources.

-          Positioning âservice topologiesâ as the basic unit of management. What this means is that all service management operations (deploy, undeploy, modify/update, upgrade, etc.) are invoked on a service topology, not on individual service components (ânodesâ). It is the task of the orchestrator to translated management operations on service topologies into sets of operations on service components (e.g. by automatically creating workflows)

-          Positioning service topologies as the context within which external events (generated by the platforms) are interpreted and handled.

 

This is another reason, I think we  should spell out clearly that those two approaches exist and should not be mixed.

Or state clearly that TOSCA is intended for catalog-based orchestration 😊

 

Peter

 

Chris