Re: TOSCA inputs and outputs for shell scripts

[adam souzis <adam@onecommons.org>]

On Tue, Oct 6, 2020 at 7:56 PM Chris Lauwers <lauwers@ubicity.com> wrote:

ÂÂ ÂClearly using environment variable for outputs doesnât work. As alternatives, we discussed passing a file descriptor, or an environment variable that specifies a location to which to write outputs. Have you given any more thought to this?

Yes I considered implementing something like that where the shell scripts could echo outputs to a file descriptor that was set in an environment variable and the orchestrator would read from that file after the script terminated. But I didn't find implementing this was necessary since my orchestrator has TOSCA extensions that lets theÂuser configure the environment variables passed to the shell script or process and also a template which has access to the script's stdout, stderr and returncode. So essentially the user is free to implement any private protocol between the orchestrator and shell script as needed. This is particularly useful if you need to work with pre-existingÂscripts.

Iâm not sure how having access to the scriptâs stdout and stderr allows the orchestrator to retrieve the appropriate operation output values. If a script has to return output values, doesnât it need to be designed to do so?

When my orchestrator invokes the artifact processor for bash scripts, it will âcloneâ the stdout file descriptor to a new file descriptor (typically nr. 3), and then redirect stdout and stderr to various log files. The script will then âechoâ any required output values to the âclonedâ file descriptor.

Â

Yes, I just mean that the user defined templates can implement a protocol like yours or something different as needed. For example, to support Terraform it can read outputs from its Terraform state file (which is encoded as json).Â

Â

Regarding file descriptors, their main benefits compared to a file path are that file descriptors are more secure: easier to avoid potential race conditions and information leaks, and they can point to a private temp file not visible on the file system. They are also more flexible (e.g. they don't require access to a file system, they can point to pipe so they can more easily support streaming, etc.)

Â

ÂÂÂÂÂÂÂÂÂ Even for input values, environment variable are not always convenient. What is the standard way for encoding maps, lists, or complex data types so they can be carried in an environment variable? Iâm curious how your implementation handles this.

Â

My implementation handles this by trying to parse the variable's value as JSON, and if parsing fails,Âtreat it as a string. This approach is easy to implement and handles numbers, lists, and maps pretty much as the user would expect.

Thatâs a reasonable and elegant approach, but why wouldnât we always encode all input values as JSON?

Â

Yes that makes sense... in my case also needs to be able to handle external environment variables that weren't set by the orchestrator.

Â

Thanks,

Â

Chris

Â