RE: [tosca] Beyond TOSCA workflows

Hi Tal,

Iâm not sure that what youâre suggesting/describing is incompatible with the âworkflowâ approach in TOSCA. I look at TOSCA as an end-to-end service description that is not tied to any specific orchestration technology, and as a result can be used for services that combine components in multiple orchestration domains. Using this philosophy, nothing ever gets really orchestrated by TOSCA directly. Instead, TOSCA is responsible for âdecomposingâ a service into sub-components and âdelegatingâ the orchestration of those subcomponents to domain specific orchestrators. Those domain-specific orchestrators could be as simple as bash shells (that run scripts) or as complicated as Kubernetes. Either way, using this approach, something needs to coordinate the âdelegationâ to the different sub-orchestrators. That is the task of a TOSCA orchestrator, and this is where TOSCA workflows come into play. This is also the area where your âdeep integrationâ approach would be very helpful, since the TOSCA spec talks about âartifact processorsâ that handle orchestration of subcomponents by processing artifacts that describe those subcomponents, but the spec is very vague about how the interaction between a TOSCA orchestrator and an artifact processor is supposed to be handled. It would be much more convenient if domain-specific orchestrators could handle TOSCA artifacts. So, there is lots of value in integrating TOSCA more deeply with existing orchestrators, but I think there is also value in keeping orchestration functionality in TOSCA to allow it to coordinate across multiple orchestrators.

Chris

From: tosca@lists.oasis-open.org [mailto:tosca@lists.oasis-open.org] On Behalf Of Tal Liron
Sent: Friday, October 11, 2019 10:12 AM
To: tosca@lists.oasis-open.org
Subject: [tosca] Beyond TOSCA workflows

I would very much hope the TOSCA user community could re-examine the inclusion of a workflow grammar in the TOSCA standard. The grammar was limited from the start, but I think recent discussions have emphasized how challenging it would be to overcome these limitations. More importantly, I think they've underscored the diversity of requirements from this feature. Different orchestrators work ... differently. And I would think that by definition we would never be able to please everyone.

So what can we do? Does this mean giving up on handling this important integration feature?

I think it's worth going back to the original problem we're trying to solve. Cloud workloads are encapsulations: descriptive topologies + functionality. "Legacy" functionality has been delivered in artifacts like Bash scripts, Ansible playbooks, Mistral worfklows, YANG models, and other custom solutions. In the newer cloud-native world the artifacts can be container images or executables that become embedded into the cloud platform, such as Kubernetes operators and Juju charms. (And sometimes there are functional dependencies on management platforms: these are normally not encapsulated with the workload, but are rather seen as a requirement for the cloud. The management platform must be "pre-installed".)

If we're going to use TOSCA for the descriptor, then we start with a rather superficial encapsulation. We can package our scripts and playbooks and recipes into the CSAR package, and refer to them in TOSCA by attaching them as "artifacts" to nodes. But there's no real interoperability. For example, the Ansible playbook paradigm relies on "hosts", which can be explicit or generated from code, while TOSCA has node templates and relationships. And I think that's the problem we're trying to address: How to we make that encapsulation deep? How do we integrate the description and the functionality?

The engineering solution is quite obvious to me in this particular case. I would want to create custom "roles" for Ansible that can parse the TOSCA and feed the "hosts" to rest of the system. Actually, this is exactly how Ansible handles many cloud features right now. This allows me to use Ansible to its full potential while also using TOSCA to its full potential. Am I using Mistral instead? Then again we'd want to add TOSCA-awareness to Mistral. The same with Kubernetes operators, Juju charms, etc. E.g. with Kubernetes I am adding TOSCA topology information as metadata to Kubernetes resources, so that an operator (service mesh operator?) would be able to make use of architectural design. All good orchestrators are extensible, exactly in order to allow them to work with different operational systems.

I keep repeating this point again and again: we already have very mature orchestration solutions. And in some cases (proprietary vendor products) we have no choice but to use them. If TOSCA is to be useful in the orchestration space it needs to be able to add something in this space, not take away existing features.

We have taken the opposite approach by adding a workflow grammar to TOSCA. We are now dictating a paradigm and expecting orchestrators to deal with it. Some orchestrators might be somewhat aligned with it, others will not be. One engineering solution might be to add another orchestration layer to orchestrate the orchestrator, which is not only complex but also hides away the feature set of the "real" orchestrator. In Puccini I've been experimenting with a half-way approach: I can export TOSCA workflows to Ansible roles and simple playbooks, but I can also incorporate these roles into a much fuller playbook. Or I can export a BPMN2 process, and then include it as a sub-process within a bigger process. But in both cases these integration points represent the lowest-common denominator set of features from the TOSCA workflow grammar. Rather than strengthening my final orchestration solution, they are a weak point that I have to work around, and so I don't think this method will ever be good. The good orchestration path is the opposite: in other words, rather than TOSCA exporting to Ansible, Ansible should consume TOSCA. (My next project will be to do something like that for Ansible.)

So, what can we do in TOSCA and OASIS to make orchestration better rather than worse? Well, we've always had the chicken-and-egg problem in TOSCA, whereby we don't have a mature ecosystem of orchestration solutions for it. If we did, then this encapsulation problem would already have been solved. Actually, we could have multiple, competing solutions, and that competition could only do good things for orchestration and of course for TOSCA. TOSCA could be the standard grammar for describing cloud topologies across a wide range of products. The types/models/profiles might be diverse, but the tools would be the same. One dream is to have a full-featured, graphical TOSCA IDE where we can design and debug profiles and service templates, an IDE that could be used with whatever orchestrator we are using. The development process to follow is less about standards and more about open source collaboration, such that it would in the interest of everyone to participate and make such an IDE better by contributing code. A comparison can be made to the world of web development: there are many competing web GUI frameworks, but all of them rely on the de facto standard web language, _javascript_ (or WASM) and CSS. So even though we work with different APIs and paradigms for each _javascript_ framework, we're still all talking _javascript_/CSS and all benefit from a strong _javascript_/CSS toolchain. TOSCA can be the WASM of cloud orchestration.

But we can't force the creation of an ecosystem, and we definitely can't force it by creating yet another orchestration grammar when the field is already so dense with competing solutions and approaches. In the same way we haven't been able to force the use of the Simple Profile as a way to model cloud topologies. Lowest-common denominator approaches are doomed to do a poor job. The industry thrives on differentiation, not sameness. Standards bodies are at their best when they enable interoperability and create opportunities for integration, not when they restrict differentiation.

I think we need to admit defeat and remove the workflow grammar, just as we should remove the Simple Profile. The solution to this problem is not within the TOSCA TC per se, but rather in the open source community that is growing around TOSCA. Encourage more and more developers to integrate TOSCA into their orchestration solutions by making it as easy as possible to for them consume TOSCA. Create extensions to Ansible, Terraform, Juju, Cloudify, JBoss, etc., that make them "TOSCA aware". Use TOSCA's powerful type system to model the resources that make sense within each of these orchestration paradigms. Show how these orchestrators become better with TOSCA added. I just can't see how TOSCA can be relevant if we can't show that. And this work should be done not as part of the TOSCA spec, but as part of the open source community efforts around the spec.

I'm trying to do my part in this via various PoCs I build around Puccini. But my volunteer resources are very limited! If you want to join my efforts, I'd be very happy to help you get you up to speed. If you want to go at it in your own way, I'd be happy to take a look and give feedback.

tosca message