Re: Proposal for a new way to think of declarative policies and groups in TOSCA 2.0

[Tal Liron <tliron@redhat.com>]

Just to clarify a point, in response to Priya's concern:

I am not suggesting any change to grammar here, only a cleanup of the syntax. The exact same grammatical "relationships" between nodes and policies that we have in TOSCA 1.3 are present in this proposal, and vice versa.

I think policies as they stand are a good and clear part of TOSCA grammar and see no reason to change it. To explain it as simply as I can, this is how I understand policies in both TOSCA 1.3 and how I want them to continue into TOSCA 2.0:

• A policy relates to zero or more node templates.
• Otherwise, the TOSCA syntax has no opinion on the runtime semantics of the policy. We are aware of the following variations, at least:
• the policy can continuously "apply" to the runtime nodes,
• or it can "occur" at one moment (e.g. during a "lifecycles, such as a provisioning phase),
• or it can "enforce" certain rules in response to various events,
• or it can "limit" certain behaviors at certain intervals,
• or it can "execute" a complex set of actions in response to changing runtime conditions (which can be configurable "workflows"), which in turn can lead to other reactions,
• etc.
• All of these runtime semantics can be modeled using TOSCA data types.
• TOSCA's intrinsic functions (get_property etc.) provide a way to map data from across the topology into the policy (similarly to how "inputs" are used in operations).
  

That said, I'm uncomfortable with policy triggers, and I think we all agree that they are not very usable in their current state. I think it was a mistake to add that feature to the TOSCA grammar, because it is at once an extremely opinionated and insufficient grammar. Instead, the feature should have been expressed by modeling data types (policy properties). I'll explain:

• Do you need an event-condition-action? Great. Model your exact algorithm using TOSCA data types.
• A popular algorithm, or family of algorithms, is the Rete Algorithm, a.k.a. the "combing" algorithm. (There are others, too, e.g. based on pattern matching.)
  
• (Drools is an example of a modern implementation of Rete.)
  
• Rete is the algorithm that TOSCA's policy trigger is aspiring to, i.e. using cascading boolean conditionals.
• But, note, that Rete is more than just conditions: Rete also defines the flow of state as the algorithm progresses.
  
• Real-world Rete also assumes a "knowledge base", with its own universe of functions for data retrieval, orthogonal to TOSCA's intrinsic functions.
  

So, how do I imagine doing this without specialized TOSCA syntax? There are two ways.

First, the community can create a "Rete" profile for TOSCA, which would include a Rete base policy type and all the data types required. This would be a lowest-common denominator, with the assumption that any Rete implementation should be able to handle it. E.g. Drools, CLIPS, Jess, etc. TOSCA implementations can specify as to whether they support this profile or not. For example, ETSI might require "Rete" profile support as a way to express these rules without specifying the policy engine. Implementations would then be able to translate the TOSCA Rete profile to the rule language of a particular engine. (I think this would never be trivial, but if it's a requirement then it's a requirement.) The translation implementation could be attached as an artifact (for generic TOSCA orchestrators, like Ubicity), or be built into the system.

The second option (and my preference) would be to avoid weak lowest-common denominators and create profiles for specific policy engines and their unique features. For example, there would be a "Drools" profile. You would then attach a ".drl" file as an artifact (here's an example) and use the policy type's properties to map from the TOSCA topology to the drl parameters. The Drools rule language is of course much richer than anything we would define as a lowest-common denominator in TOSCA and we would avoid the problems of translation.

On Tue, Jun 9, 2020 at 11:56 AM Tal Liron <tliron@redhat.com> wrote:

First, remove the "policies" section in the topology template and remove the "targets" keyword from policies. Instead, the reference will happen in the other direction, from node templates via the addition of an optional "policies" keyword, which is a list of policy assignments to be applied in order. This works a bit like the "artifacts" keyword for node templates -- something that is attached to the node template but does not have to be declared at the type. (Also, like artifacts, they could also be assigned at the node type, in which case they act as defaults to be added to all node templates of that type.)Â I think this is simpler and more understandable than what we have now, which is multiple ways to group nodes together and requiring filters for policy triggers. This way we attach policies to where they need to be.

So what are groups for? They are purely organizational and time-saving, a way to avoid replication. The idea is that groups will have some of the same keywords as node templates, with the understanding that they will apply to all the node templates in the group.

With this new understanding, we can remove the group type. Groups in turn will have no "type", "properties", nor "attributes" keywords. All they have is: "metadata", "description", "node_templates" (instead of "members") with no restrictions on what nodes can be added. And, relevant to this discussion, there is a "policies" keyword to be applied to all nodes in the group. If a node also has its own policies, those will be applied before those of the group (they are higher priority). We can also add an "artifacts" keyword as a way to attach the same artifact to all nodes. (And there can also be a "directives" keyword, again applied to all member node templates. Although I really want to see this keyword disappear!)

Groups should not have attributes, for the same reason they don't have interfaces. They are not real-world resources. Still, is it possible to add properties to a group as a logical entity, as in TOSCA 1.3? Well, yes, by attaching a policy to it. So we are not losing that ability, but rather clarifying what is meant by adding properties to groups.

As for how to handle group metadata, it's as always up the implementation. If the implementation wants to apply that metadata to all node templates in the group or store it independently and treat the group as a logical entity, that should not be our business.

Note the renaming of "members" to "node_templates". This is to avoid namespace ambiguities. There would also be a "groups" keyword to allow a group to include other groups. There is no problem with that in this new understanding, as in the end a group is just a bunch of node templates and not a real-world resource.

Here's an example to illustrate how it all works together:

topology_template:

 node_templates:

ÂÂÂ app:

ÂÂÂÂÂ type: MyApplication

ÂÂÂÂÂ policies:

ÂÂÂÂÂ - fail-fast:

 Â Â type: Tolerance

 Â Â properties:

 Â Â downtimeAllowance: 5 s

 Â Â downtimeWarning: 3 s

ÂÂÂ server:

ÂÂÂÂÂ type: VirtualMachine

ÂÂÂ db:

ÂÂÂÂÂ type: AbstractDatabase

ÂÂÂÂÂ policies:

ÂÂÂÂÂ - substitution:

ÂÂÂÂÂÂÂÂÂ type: Substitution

ÂÂÂÂÂÂÂÂÂ properties:

ÂÂÂÂÂÂÂÂÂÂÂ options: [ MariaDB, MySQL ]

ÂÂÂÂÂÂÂÂÂÂÂ cluster: false

 groups:

ÂÂÂ hosts:

ÂÂÂÂÂ metadata:

ÂÂÂÂÂÂÂ my-orchestrator.instantiate: 'true'

ÂÂÂÂÂ node_templates: [ server, db ]

ÂÂÂÂÂ policies:

ÂÂÂÂÂ - high-availability:

 Â Â type: Redundancy

 Â Â properties:

 Â Â redundantNodes: 2

 Â Â loadBalancer: RoundRobin

 Â - geo-redundancy:

 Â Â type: MultiSite

ÂÂÂ Â Â Â properties:

ÂÂÂÂÂÂÂÂÂÂÂ sites: [ Boston-123, Shanghai-456 ]

The question remains as to what to do with policy triggers. In my view, there is no benefit in having grammar for this, as it is entirely domain specific, moreover it is often dependent on the runtime environment rather than a design consideration. E.g. in the example above "fail-fast" will be applied when the downtime is higher than 5 seconds, while the other policies do not have specific "triggers". Creating a grammar for this will never satisfy all real world requirements, while on the other hand would need to be very complex in order to try to satisfy just some of those requirements. (This is the same criticism I have for workflows, which I would also like to see removed from TOSCA.)

That said, if other folk insist on having policy triggers, they could be added to the policy assignment as the keyword "when". From the example above, using some pseduo-grammar:

ÂÂÂÂÂ - fail-fast:

 Â Â type: Tolerance

ÂÂÂÂÂÂÂÂÂ when:

ÂÂÂÂÂÂÂÂÂ - "downtime" attribute is greater than 5 s

There is pseudo-grammar here because I have no idea how we can create grammar that would properly express all conditions, and I am not sure it is smart to have to model all runtime attributes, which might actually not be known by the designer (orchestration-implementation-spoecific).