Hi Tal,
After reviewing this again, Iâm not sure I completely understand your terminology:
-
In the spec, we talk about an âArtifact Processorâ. This is intended to be a piece of code that is not necessarily built-in to the orchestrator, but that is provided by artifact type designers to correctly âprocessâ the artifact within the context of the orchestrator.
-
Adam in is presentation earlier this week talked about a Configurator (I hope Iâm remembering this correctly). From the discussion, it sounded like the Configuration was identical to (or at least similar to) the concept of the Artifact Processor used in the
TOSCA spec.
-
In your email below, you talk about an Executor. At first glance, I interpreted the Executor to be similar to an Artifact Processor or to a Configurator, but after reading this again Iâm not so sure.
Would you mind explaining your concept of an Executor?
Many thanks,
Chris
From: Tal Liron <tliron@redhat.com>
Sent: Monday, August 23, 2021 7:28 AM
To: Chris Lauwers <lauwers@ubicity.com>
Cc: tosca@lists.oasis-open.org
Subject: Re: [tosca] Operation implementations
I just want to point out that beyond the example uses cases we listed, the real mess is when we start discussing the details of how the mechanisms would work.
For non-artifact operations:
-
How do you discover the executor? (DNS? Consul? Node attribute? Kubernetes CR status?)
-
How do you authenticate the executor? (TLS? Where do we store the certs? Not in TOSCA, for sure...)
-
How do you authorize? (A user? Does the user depend on the TOSCA service template, or whoever is deploying at this moment?)
The answer to some or all of these may be purposefully out of the scope of TOSCA. This would especially be true if you have a rich client of some sort that has its own configuration and operational paradigm. In this case, TOSCA would just
be using lightweight IDs of some sort (e.g. URLs) for the implementation (the "what") and not have anything to do with the "how".
However ... the mechanism itself might be involve more than just "send a packet to this endpoint". For example, there could be a handshake, the establishment of a session, or even a transaction (e.g. NETCONF). So, in TOSCA we would want
a way to declare the contours of this transaction. You could think of it as a "mini-workflow". Grammatically, you can think of it as a mini-DSL. The point is that one size does not fit all here, so we want to make sure TOSCA is as flexible as is necessary.
For all of the above it would be quite verbose if we would have to repeat everything for every operation, on every interface, on every type. An operation type (looks like a data type) can come with not just the structure, but also default
values, as well as a way to configure them across the entire service template.
-
What kind of and which compute element will be executing? (A VM? A jump server? A container in a Kubernetes pod? One or all? Or serverless? Or an orchestrator like Ansible Tower? Or a pipeline like Hadoop?)
-
How and when do we deploy the artifact on the executor? (What is the "event", and what does TOSCA need to know about the event?)
-
We need to think "installation" rather than "copying over". Having a file just be able to run is surely nice to have but it's a trivial example.
-
"Installation" can be complex. Artifact properties can describe how to do it. (Make sure Python is installed? Which version? With which packages?)
-
Again, we have the challenges of authentication and authorization in addition to all the above aspects of the "how".
Until now, our answer to all of the above requirements has been, mostly, to use artifact properties and operation inputs. But beyond the awkwardness of it (which inputs are for the "what" and which for the "how"?) there is a challenge of
scalability, which TOSCA's OOP cannot solve if you need to configure aspects across type hierarchies. For example, some nodes might need to execute on one jump server, while others might need to execute on others. If all have the same node type (and interface
type) then there is no simple way to do this with just artifacts and operations. We could use policies and/or groups, of course, but we would need the TOSCA orchestrator to know how to get what it needs from there. I do think operation types can fill in this
important gap, the idea being that the type can decide which executor to use, and also define the validated schema as well as other "how" configurations of what TOSCA will be sending to the executor.
If we want to get a point where it is possible to create portable service definitions in TOSCA, we need to make sure that we clearly define the âmembraneâ between the TOSCA orchestrator
and the external world. This membrane currently consists of âoperation implementationsâ and the information needed by those operation implementations.
During this weekâs TOSCA Language Ad-Hoc meeting, there were at least three different viewpoints expressed about how operation implementations can be defined in TOSCA:
- Operation implementations are artifacts. Each artifact type is âexecutedâ by an artifact processor that knows how to handle artifacts of that type. This is the mechanism
that was prescribed in v1.3
- Operation implementations can be arbitrary strings. Some implementations do not need to be provided by a file. Instead, a string may be sufficient (e.g., a URL).
- Operation implementations are arbitrary YAML. In order to allow for validation, Tal has suggested introducing an Operation Type against which this YAML can be validated.
We should explore all three of these viewpoints. One way to do this is to work through specific examples where the best way (or only way) to provide the operation implementation
is through one of these three options.
During the meeting, the following use cases were mentioned:
- âDeployingâ a file into a specific directory (presumably on some remote âhostâ)
- Running a shell script
o
Within the Orchestrator execution environment
o
One a remote âhostâ
o
A *.py file
o
A python egg
o
How to deal with virtual environments?
o
Best way to deal with dependencies?
o
A .java file
o
A jar file
o
Using Ansible
o
Using Ansible Tower
- Running Cloudify workflows (where the workflow is identified by name)
- Making a REST call
- Making a NETCONF call
- Starting a VM using a VM Image
- Running a container using a Container image
o
Using plain Docker (or some other container mgmt. system)
o
Using Kubernetes (or some other orchestration system)
For each of these use cases, it would be helpful to:
-
Provide an example snippet of TOSCA code
-
Explain how an orchestrator is expected to process this code.
Please contribute when you have a chance. If we get enough of these, Iâll try to create a spot for them in our public github repository.
Thanks,
Chris
|