RE: Differentiating TOSCA from HEAT

Hi Peter, more questions (that will show my ignorance) 😊

Great questions. There is a limit to what more, I can disclose. Looking up patents with my name can give some information.

- Could you please explain what you mean by âcompositional graph grammarâ vs. âjust a graphâ?

A Graph Grammar corresponds to a Topology, like a BNF grammar corresponds to a program written in the language defined by that grammar.

These donât sound to me like different types of the same âclassâ of things. For example, wouldnât you be able to create a BNF grammar for your âGraph Grammarâ. BNF just formalizes syntax, not the language and its associated semantics. If I understand correctly, wouldnât the differences be more in the types of data supported and in the operations supported on those data? Traditional languages operate on strings, numbers, objects, etc. whereas I assume a graph language operates on graphs?

A BNF grammar for a programming language defines a class of texts that are syntactically valid for that language. You add a static semantics (type-checking, etc. to further constrain valid programs). A BNF grammar is not fully compositional, because you cannot add new production rules without modifying existing rules, but the BNF language itself could easily be modified to become fully compositional.

Apologies for being a little dense again, but what do you mean by âa BNF grammar is not fully compositionalâ. What is it that canât be composed? And what do you mean by âproduction rulesâ? What is it that is being produced (for which rules are needed)?

Furthermore, you usually want the language to have a denotational semantics â adding a production to the graph should not alter the semantics of existing constructs. The DSD language is compositional â you can incrementally add new production rules to the service grammar, that allow you to express more complex services (more topologies).

I assume we agree that TOSCA is denotational as well?

Example: Rule/trigger-based programming is compositional, because you can keep adding more rules that would âfireâ under certain conditions. But it is not denotational, because adding one rule could completely alter the meaning of existing rules. Basically, such a non-denotational rule-base gives you something akin to the proverbial Aprils fool âconditional comefromâ construct https://en.wikipedia.org/wiki/COMEFROM

Doesnât this example mix and match the semantics of the language vs. the semantics of the program created using the language? One can always add code to a program that changes the behavior of previous code that was already written, but hopefully that code doesnât change the semantics of what is expressed using the language?

- What is it about TOSCA in your opinion that makes it not Turing-complete (as opposed to a compositional graph grammar?)

TOSCA is not trying to be a programming language (aka Turing-complete). Speculatively, I guess you might be able to make substitutions recursive, and so claim Turing completeness, but that is not a recommended use of TOSCA. Graph Grammars by themselves are not a programming language either â that would be like claiming that BNF is a programming language. So of course there is more to thatâ

Perhaps weâre not entirely on the same page about the objectives for TOSCA. I absolutely try to look at TOSCA as a programming language. My orchestrator doesnât know how to do anything other than what it is programmed to do by TOSCA service templates, i.e. by applying TOSCA language semantics to those templates. The TOSCA language semantics define how graph instances are created from templates (using provided input values), how requirements are fulfilled, how substitution mapping is used to decompose services, how to create workflows that ârealizeâ the resulting service topology graphs on the external platforms, how to generate events (ânotificationsâ) by monitoring external platforms, how to âtriggerâ ECA policies in response to events, etc.

Also, substitution mapping is definitely intended to be recursive. I use substitution mapping primarily to deal with abstraction: I map abstract service representations into lower-level topologies that apply configurations to external platforms. The most practical way to accomplish this is to do this recursively (by mapping âbusiness viewâ to âsystem viewâ to âadministrator viewâ to âdevice viewâ to âinstance viewâ using John Strassnerâs policy continuum framework).

- What do you mean by âcompletely declarativeâ? In my opinion, any declarative language requires a tool that turns âdeclarationsâ into the âimperativeâ sequence of steps that must be taken to ârealizeâ those declarations. If that tool has built-in knowledge about the set of entities that can be used in those declarative descriptions (e.g., Kubernetes), then nothing further is required. However, now that we have removed the Simple Profile types from the TOSCA language, TOSCA no longer has any built-in knowledge about anything, and every type is a âcustomâ type. This means that âimperativeâ support must be provided to âimplementâ those custom types. Fortunately, the TOSCA language includes imperative features that support this. The fact that TOSCA is both declarative and imperative is one of the very important differentiators of TOSCA, in my opinion:

o Service designers use TOSCA as a declarative language, using types defined in the various TOSCA profiles

o Profile designers use TOSCA as an imperative language by defining interfaces, operations, notifications, and artifacts that âimplementâ the types in their profiles

TOSCA, HOT and DSD are the same family, equally declarative, as Tal pointed out.

The languages are declarative in the sense that an _expression_ in that language does not say: âFirst do this, then thatâ. You can add artifacts etc, that allows an orchestrator to create a plan/playbook/process/â to deploy the service, but the orchestrator could also derive how to undeploy the same service, or modify it. So the language itself does not imply the direction of actions, only policies for creating actions after being given a direction. Compare to imperative languages like C++/Java/Workflows. There is no way to âreverseâ the actions of such languages. Functional languages are not reversible, but logic languages â Prolog â are (unless you use cuts â!â), so they can be used declaratively. The fact that there are policies/artifacts/interfaces/operations/etc that guide the translation from declarative to sets of actions does not make these languages less declarative in their essence.

I donât look at âundeployâ as a way to âundoâ, but rather as one possible âentry pointâ into the TOSCA service template (just like there may be multiple entry points in a Java or Python package). In fact, I maintain that there really are only 5 âcanonicalâ entry points into a TOSCA service template to which all other actions can be reduced: deploy, undeploy, update/modify, upgrade, and ârun workflowâ (there are of course also the southbound callbacks that represent notifications).

By the way, I am aware of the âreversible programmingâ research topic, but that is something different.

I would also like to come back to some of my earlier comments:

- We continue to talk about âcloud-native orchestrationâ without actually ever defining that term. In order to have productive discussions, we need clear definitions about terminology.

Here is an initial shot at a definition:

The orchestration can be cloud-native if and only if:

1. The orchestrator itself is cloud-native

While this makes a lot of sense, this feels strictly like an implementation detail, and perhaps not part of the definition for âcloud native orchestrationâ

2. Every part of a service can be created/deleted/configured/reconfigured on a cloud-native sub-system by sending a declarative configuration to that sub-system

Does this imply then that being âdeclarativeâ is a prerequisite for being cloud-native?

3. Any order of pushing declarations to sub-systems has the same effect (temporal symmetry)

Didnât you argue earlier that this may not always be possible?

4. Any required coordination between the sub-systems is handled by communication directly between the sub-systems, and does not require mediation by the orchestrator

Iâm not sure I agree with this. In fact, this implies distributed coordination that may lead to exactly the types of event storms you were concerned about earlier. I believe each subsystem should only be concerned with its âlocalâ domain (which I believe is what Tal suggested). If inter-domain coordination is required, sub-domains should escalate to their âparentâ coordinator to handle inter-domain issues (since those inter-domain issues are in fact âlocalâ to the parent coordinate).

5. Optionally (?): The orchestrator is informed by each sub-system about its state, allowing the orchestrator to represent snapshots of the end-to-end state of the service

I donât believe this should be optional 😊

6. Optionally (?): If the orchestrator is informed about deviations from the desired end-to-end service configuration in a sub-system, it may react by pushing additional configurations to the sub-systems

Same comment here. I donât believe this is optional.

Tal, please improve.

What I may be getting wrong here is that there is still a hierarchy of âthe orchestratorâ and the âsub-systemsâ, but could there be an architecture of peer-orchestrators, where each orchestrator would appear as a sub-system to the other orchestrators? But in such an architecture, could the peer-orchestrators have conflicting âobjectivesâ so they might start âfightingâ by perceiving the configurations pushed by other orchestrators as conflicting with their own objectives?

Yes, which is why I believe coordination should be top-down, not peer-to-peer.

- The same holds for the âmoving from the inside outâ (the centrifuge concept) vs. âmoving from the top downâ. I believe Tal wanted to position these as two very different orchestration concepts, but I must admit Iâm not entirely clear on the core distinction between these two. Is the âcentrifugeâ concept an implementation of the Operator pattern (where declarative descriptions are converted into more fine grain declarative descriptions) or am I missing the point? If so, I have previously suggested âdecompose and delegateâ as a way to describe the same concept. If this is what the centrifuge concept represents, then I agree that this is a âmust-haveâ feature. But I believe this feature should be used for âdecomposingâ abstract representations into concrete descriptions that can actually be orchestrated, not as the orchestration functionality itself. Either way, I believe it would be helpful to better describe these concepts.

This is for Tal.

/Peter

Thanks. Extremely helpful and productive!

tosca message