RE: [tosca] Groups - TOSCA Operational Modalities uploaded

[Chris Lauwers <lauwers@ubicity.com>]

It would be very helpful if we clearly defined all the various pieces of terminology we use to describe different systems. The list of terms we have used in the past includes at least the following:

 

• Declarative orchestration
• Flow-based orchestration
• Imperative orchestration
• Cloud-native orchestration
• Intent-based orchestration
• Desired state orchestration

 

I suspect that once we get to the essence of what each of these mean, we will be left with only a small number of different paradigms (i.e. 2 😊)

 

Chris

 

From: Bruun, Peter Michael (CMS RnD Orchestration) <peter-michael.bruun@hpe.com>
Sent: Tuesday, December 7, 2021 8:44 AM
To: Tal Liron <tliron@redhat.com>
Cc: tosca@lists.oasis-open.org; Chris Lauwers <lauwers@ubicity.com>
Subject: RE: [tosca] Groups - TOSCA Operational Modalities uploaded

 

Hi Tal,

 

I think there is some confusion because you were not fully explaining the difference between a flow-based and fully declarative modality:

 

• A flow based orchestration translates the instantiation of a TOSCA service into a sequence of actions to be executed by the orchestrator. The sequencing may be explicit or implicit, event-driven.
• A fully declarative orchestration translates the instantiation of a TOSCA service into declarative âsnippetsâ in various declarative languages understood by the systems in the âcentrifugeâ, and those systems are then responsible for translating those into sequences of actions and events. So the orchestrator itself has no conception of time or sequencing, but events coming in may cause the orchestrator to push altered declarative snippets out. The orchestrator may also configure various monitoring and optimization systems (near the centre of the centrifuge), that would track the health and performance of the deployed systems, etc.

 

Example â deployment of VMs and applications+configurations on those VMs:

• Flow modality: It is the responsibility of the orchestrator to ensure that VMs are created and started before software is installed or configured on them.
• Declarative modality: The orchestrator would push HEAT declarations telling Openstack what VMs and networks should exist, and once these VMs happen to have started, they could come up having configured images that instruct them how and where to pull the configurations and applications they need. So all the time-based interactions happen between the systems themselves. They would, however, push events back into the orchestration level to provide the information for the single glass of pain.

 

I hope this more or less captures your intended difference between slides 1+3 and slides 2+4 respectively.

 

We perfectly agree that both modalities exist â in practice probably really just driven by the choice of underlying technologies â some of which may support the declarative approach, whereas others may belong to other paradigms that require explicit, flow-based orchestration.

 

So there is no question of one modality being better or worse than the other â but as you keep saying â we should make sure that we define TOSCA to support *both* modalities.

 

If you do happen to have a heterogeneous mix of systems where some fit in the declarative, cloud-native paradigm, and others donât, then there is an interesting third role of the orchestrator of bridging the two modalities within a single service instantiation. This would involve:

• Translating events from the declarative subsystems into wait-and-trigger state-transitions in the flow-model
• Holding back the outward push of declarations until other subsystems are ready. Declarative systems may also support declaration of wait-conditions that can be hooked up to progress events from the orchestratorâs execution of flow-transitions.

 

Does this make sense? Can we make a model that shows this clearly?

 

Peter

 

From: Tal Liron [mailto:tliron@redhat.com]
Sent: 6. december 2021 19:36
To: Bruun, Peter Michael (CMS RnD Orchestration) <peter-michael.bruun@hpe.com>
Cc: tosca@lists.oasis-open.org; Chris Lauwers <lauwers@ubicity.com>
Subject: Re: [tosca] Groups - TOSCA Operational Modalities uploaded

 

On Mon, Dec 6, 2021 at 12:11 PM Bruun, Peter Michael (CMS RnD Orchestration) <peter-michael.bruun@hpe.com> wrote:

There needs to be central coordination in order to ensure a convergent process, and that, in my experience, is exactly the role of the orchestrator. It may be, that this is what slide 2 is expressing â but that is not clear to me.

 

From your comments I think we are actually completely aligned! But because this new way of thinking about an orchestrator is so different, I simply prefer to call it a "coordinator" rather than an "orchestrator". The more classical type of orchestration (provisioning, installing, configuring, deploying, etc.) is still there, but it happens locally in the halo platforms. And that's where I think some of the problems you mention (convergence, event storms, event loops) are tamed. Out there it's really traditional orchestration in many respects, but handled very locally and very specifically (non-generically), so it's much easier to define, test, and debug. As for it not scaling -- well, the pyramid model is already failing us in terms of scale. Scaling our networks is an ongoing challenge and the only way forward is innovation.

 

Also, I was just showing an example in which the "coordinator" is broken up into several components. My intent was not to make them hardcoded. It does not have to be per my example as those roles could definitely all be centralized in one "coordinator" product or otherwise organized and distributed. The point is that there are various components that have to live in the center (rather than "the top", as in the pyramid), which is where they can get a global perspective and make global decisions (declaratively).

 

But yes, I agree with your bottom line that this scheme cannot work without regimented collaboration. It's definitely not just a happy circle -- there has to be some shared source of truth, and policies will become more and more important as we move towards this scheme -- policies that can adapt, clash, reconcile, and allow for necessary exceptions. If you insist on calling this "orchestration" you are welcome to, but it does represent a very different paradigm from the classical pyramid. And really that's what I'm trying to do here, to ensure that TOSCA 2.0 supports multiple orchestration paradigms, including cloud native ones that are still a work in progress.