RE: Differentiating TOSCA from HEAT

["Bruun, Peter Michael (CMS RnD Orchestration)" <peter-michael.bruun@hpe.com>]

I think this is because we are on the same page, but the discussions and the uses I see in the field reveal that not everyone uses TOSCA that way.

 

I distinguish sharply between the templates in a catalog and the instantiations created from templates.

 

In catalog based orchestration, there is a sharp distinction between the actors who define the templates in the catalog and those who instantiated those templates to create a service. Particularly, to call it catalog based, the latter will not create or modify TOSCA templates.

 

It is possible and meaningful to use TOSCA as a tool for designing instances. A uses picks a template, adds a few nodes and relations and instantiates (aka. clone-and-modify). Maybe the altered template is even pushed back into a repository of templates. It is also possible and meaningful to âhackâ a workflow/plan/runbook generated by an orchestrator, as Tal mentions in the mail from January 6, 2022:

 

There are various reasons why Heat isn't great. The relevant one for our group is: it creates its worfklows automatically for you but there is almost no visibility into them, and definitely no hackability.

 

But in âcatalog based orchestrationâ, the actors who instantiate services:

         Do not write (or even necessarily know) TOSCA

         Do not hack any part of the orchestration process after deciding to deploy a template with some inputs

 

So like âcloud-native orchestrationâ, I see âcatalog based orchestrationâ as a restricted part of the general concept of âorchestrationâ, where the restriction comes at a cost and with some benefit. I believe I outlined the benefits below.

 

The problem with catalog based orchestration is highlighted by Talâs second mail from January 6:

 

From the perspective of a from-the-trenches engineer: there's nothing more frustrating then a big automatic system that does a zillion things for you, but then can't do the simplest thing that you can do in a command terminal in one minute.

 

There are two ways to go from there:

1.       Abandon the catalog based approach, and embrace that TOSCA is used for designing and customizing specific instances

2.       Ensure that the TOSCA language is expressive that the catalogued flexibility can be sufficient for users not to become frustrated

 

Both ways are respectable, and I am perfectly happy with both, but they address different orchestration needs.

 

I am convinced that some of the discussions we have been having on input parameters and cardinalities come from not fully spelling out that there are those two different uses of TOSCA, and the requirements for the expressive power of the TOSCA language are not the same for the two approaches.

 

What I see in the field is that TOSCA users are failing with TOSCA because they are mixing the two approaches â in practice they use TOSCA as an instance design tool, but they also insist on all TOSCA templates being catalogued. The result is a catalog containing thousands of templates, each one expressing the topology needed for one specific instantiation.

 

This is another reason, I think we  should spell out clearly that those two approaches exist and should not be mixed.

 

Peter

 

From: Chris Lauwers [mailto:lauwers@ubicity.com]
Sent: 17. januar 2022 21:49
To: Bruun, Peter Michael (CMS RnD Orchestration) <peter-michael.bruun@hpe.com>; Tal Liron <tliron@redhat.com>
Cc: tosca@lists.oasis-open.org
Subject: RE: Differentiating TOSCA from HEAT

 

Doesnât TOSCA always assume a catalog-based orchestration paradigm? It starts with a template, and then customizes that template for the specific deployment by providing the appropriate input values. What is the alternative? Perhaps I donât understand your âclone and modifyâ concept?

 

Thanks,

 

Chris

 

 

From: Bruun, Peter Michael (CMS RnD Orchestration) <peter-michael.bruun@hpe.com>
Sent: Monday, January 17, 2022 5:09 AM
To: Chris Lauwers <lauwers@ubicity.com>; Tal Liron <tliron@redhat.com>
Cc: tosca@lists.oasis-open.org
Subject: RE: Differentiating TOSCA from HEAT

 

I donât have âtheâ final answers â this is all for discussion of course J My answers in-line below.

 

I am missing a concept of âCatalog based orchestrationâ.

 

In Catalog based orchestration, the clone-and-modify paradigm is forbidden. All requests start with something in the catalog plus some input, so all desired variability must be definable in the language that defines the contents of the catalog.

 

This is an argument from scale, and the classical example comes from the world of NFV.

 

• Say you define a template for creating core Telco infrastructure, packet-core, IMS, etc. deployed using NFV, then a large operator would maybe deploy one such infrastructure in each country that they operate in. There would always be local variability, and the deployment might need to be changed once every 2-5 years. So there is ample time to take a template, customize and tune it for the special local conditions, and then deploy.
• On the other hand, if you define hosted customer services, including datacenter, firewalls, access-networks, VPNs and on premise equipment (physical or virtual), and plan to sell such a service at the scale of 100.000 customers, then you want to have all services created from a common catalog, and any per-customer variability must be explicitly defined by the catalog. You could hand-craft such services per customer, and some operators do offer that. But the cost is just a lot higher when maintaining hand-crafted services 5 years later, when the person who did some special tweaks on a virtual router is no longer employed. So the price of such customized services must have a suitable offset relative to catalogued services.

 

TOSCA can be used in both modes â catalogued and custom. But for catalog based use, you would need to rely on two levels for defining a specific service:

1. Selection of a template from the catalog
2. Providing input parameter values to the template

 

The problem I see is, that if the input parameters mechanism is not sufficiently strong to express the desired variability of services, you can end up with proliferation of TOSCA template variants in the catalog. If a catalog contains 3,000 different variants of templates, it can be difficult to use. This is a real problem that some of our customers are facing when using TOSCA.

 

I am not directly in the loop there â just relaying, that it seems to be a real problem for users of TOSCA that they do not see clearly, how to express the variability they need at catalog level in terms of TOSCA input parameters.

 

A problem, I see in our work with the TOSCA language is, that the language requirements of Catalog based orchestration are different from the language requirements of clone-modify based orchestration.

 

Peter

 

 

From: Chris Lauwers [mailto:lauwers@ubicity.com]
Sent: 15. januar 2022 22:58
To: Bruun, Peter Michael (CMS RnD Orchestration) <peter-michael.bruun@hpe.com>; Tal Liron <tliron@redhat.com>
Cc: tosca@lists.oasis-open.org
Subject: RE: Differentiating TOSCA from HEAT

 

 

I definitely think that those 4 concepts are quite different.

 

I have a couple more questions for clarification if you donât mind:

 

-                      Declarative orchestration

This is an intrinsic feature of a language, stating that the language declares âwhatâ should exist (or not exist). The declarative language should not spell out not âhowâ it is made. It is still declarative if the language has a way to include the means to make it exist â artifacts, interfaces.

 

Does âdeclarative orchestrationâ then refer to orchestration systems (or orchestration methodologies) that use declarative languages?

I would say so, yes.

 

If a language provides âthe means to make it existâ, doesnât that language then also make assumptions about âhowâ those âmeansâ are expected to be used, in which case the language is moving into the âimperativeâ realm?

If there is not enough information to actually deploy the service, then we are talking about a language that is solely for modeling, design or specification. It would not be orchestratable.

 

So when we are talking âDeclarative orchestrationâ, it is safe to assume that the information to translate the declarative domain to the imperative domain is present.

 

Whether that information comes from the orchestrator, a profile or is defined within the language seems secondary to me. The criterion is that the orchestration is automatically derived, not formulated as imperative âworkflowsâ in the language.

 

-                      Cloud-native orchestration

This is, as I understood Talâs slides, a subset of orchestration problems, where the systems under orchestration satisfy a set of cloud-native constraints, that allows the orchestrator to have no concept of sequencing of actions. If just one of the systems does not satisfy those constraints, then cloud-native orchestration is not sufficient to solve the problem.

Yes, we need a better definition of âcloud native systemsâ or âcloud-native constraintsâ, but the one aspect you highlight here is that cloud-native orchestration has âno concept of the sequencing of actionsâ, which I understand to mean that there are no flows, which then seems to imply that cloud-native orchestration is by definition declarative. If thatâs the case, then I still have my original question: is there more to âcloud-native orchestrationâ than just being âdeclarative orchestrationâ?

 

Tal â please provide a better definition.

 

The assumption is that all systems in the world ought to comply with the cloud-native principles, and those that donât should either be fixed or discarded as outdated technology alongside the Steam Roller and the Edison Phonograph.

 

😊

 

-                      Intent-based orchestration

On this subject we have this: https://github.hpe.com/hpsd/hpsp/files/18603/Tamburri2019_Article_TOSCA_basedIntentModellingGoal.pdf

 

I agree 100% with this definition of intent:

 

Intent modelling Intent modelling entails modelling infrastructure blueprints by specifying a highest-level goal to be satisfied, regardless of how sub-level intents or goals are satisfied.

 

To me expressing intent is to say âI need Firewallingâ instead of âI need a Firewallâ. Or âI need Load-balancingâ not âI need a Load Balancerâ.

Is your expectation that one should be able to express âintentâ in an orchestration language?

 

Not âshouldâ, rather âcouldâ.

 

There are clearly cases where TOSCA can do this if the intent-decomposition is expressible as substitutions, but I also see intents and realizations of intent where TOSCA would force the designer to manually perform the decomposition of intent, and so the resulting TOSCA template would be derived from the original intent, but it would no longer express the original intent as an object in its own right. Notice that a program written in C is similarly derived from some original intent, but the intent itself is no longer explicitly present in-language. Clearly, C is not an intent-based language, and the fact that you can define and implement a function that expresses an intent doesnât make it so. It takes more to be Intent based than âbeing able to implement intentâ.

 

So unless we can prove that TOSCA with substitutions can explicitly express all possible intent, then I am not convinced that TOSCA is intent based.

 

In my opinion, âintentâ is a special case of a âdeclarative policyâ specified at the highest level of abstraction (i.e. the âbusiness viewâ using Strassnerâs policy continuum framework again). Unfortunately, TOSCA currently doesnât support declarative policies (it only supports imperative event/condition/action policies) but assuming we add declarative policies, then I absolutely believe TOSCA is intent-based.

 

Agreed. The fact that there are types of intent that cannot be directly represented in TOSCA doesnât mean that TOSCA is not intent based.

 

Of course a firewalling intent could be implemented as an actual firewall, but I remember when OpenFlow was in fashion, and you can express a simple firewalling or load-balancing intent as a set of OpenFlow rules. Those rules are not âa Nodeâ, they get deployed on a potentially variable number of switches. So where would I put this in my TOSCA template?

I implement firewall rules as TOSCA nodes, so I imagine one could model openflow rules the same way.

 

Ok. Could be a great example.

 

You could solve this by pushing intent to Policies â so add in your template a âFirewallingâ Policy, and Firewalling happens. But the implementation of that intent would have to be hard-coded in the orchestrator, and so the Policy is just a name for a hard-coded functionality, it is not a language for defining the firewalling intent.

I think this can be done without hardcoding using substitution mapping couple with mapping of declarative policies

 

Ok. Could be a great example.

 

-                      Desired state orchestration

This is a property of an orchestrator, where the way it knows what to do is expressed in terms of a desired state of the systems under orchestration. In the case of TOSCA service creation, the âdesired stateâ can be expressed as the pair of a TOSCA specification and a set of input values. For TOSCA service destruction, I am not sure how we express the desired state of ânow that service should no longer existâ, because I am still not sure how we identify a previously created service. Basically, without having defined identifiers for deployed services, the concept of âthat serviceâ with TOSCA eludes me.

 

Yes, the concept of âidentifiersâ in TOSCA needs more discussion.

 

In the Intent paper, referenced above, there is the distinction between Goal and Intent, and I think Goal is the same as âDesired Stateâ. A TOSCA template is not a Goal, but a Goal can be created by combining a template with a specific input and an âentry pointâ (deploy, undeploy, update/modify, upgrade, â). An Orchestrator that does that can be said to be doing âdesired state orchestrationâ.

 

In my opinion, a âgoalâ can be expressed using a declarative policy. Intent is a special case of a âgoalâ: it is a declarative policy expressed at the business view layer.

 

Ok.