Re: [tosca] tosca.datatypes.json and tosca.datatypes.xml

[Tal Liron <tliron@redhat.com>]

Hm, my current thinking is that this is a bit of a rabbit hole. Different formats have their own ways of using schemas, and there are in fact multiple standards, as well as non-standard schema libraries. My thinking is that this may be beyond the scope of TOSCA. Custom schema validations could happen later in the deployment toolchain. TOSCA metadata can be used to provide hints for such validators, so that they don't have to hardcode specific properties, and instead can traverse the representation data and look for such metadata. We can provide an example of such a use case in the spec, as a tutorial.

That said, the minimal validation of these formats -- without schemas -- is pretty straightforward "low-hanging fruit". E.g. Kubernetes specs have a lot of embedded JSON (terrible design, but so it goes). We might not be able to validate this JSON per the target schema at the TOSCA level, but we could at least make sure that users don't embed invalid JSON.

So, perhaps it's best to remove the "schema" constraint and have a "format" constraint instead, with values such as "yaml", "json", and "xml".

YANG is an interesting and important use case, but again I think it's out-of-scope for TOSCA. A YANG validator can be appended to the toolchain. That might actually be a good example to include in the spec!

Puccini does let you create your own custom constraints (and custom functions) using _javascript_ scriptlets, but this feature is obviously non-portable. I'm thinking that a metadata-based solution would be best.

On Thu, Jan 9, 2020 at 2:27 PM Chris Lauwers <lauwers@ubicity.com> wrote:

Hi Tal,

Â

Yes, if we want to avoid special-purpose code in the TOSCA parser, then we should investigate if we can accomplish JSON and or XML validation using the schema constraint. While weâre at it, we should discuss changes to the schema constraint itself:

Â

• I have done a number of project where it was necessary to define the schema of configuration data using YANG. We did this by specifying a YANG module in the schema constraint associated with the string property that contained the configuration data.
• However, this is technically not a correct use of the schema constraint, since the schema constraint expects the schema itself to be provided as (inline) string data.
• Expecting schemas to be provided inline is unrealistic for all but the most trivial schemas.
• We should consider using âschema artifactsâ instead.
• Unfortunately, artifact definitions can only be associated with node types and/or node templates, which makes the use of artifacts in data type definitions much more complex.

Â

Iâll add this as a topic to our next TOSCA Language Ad-Hoc meeting

Â

Thanks,

Â

Chris

Â

From: tosca@lists.oasis-open.org <tosca@lists.oasis-open.org> On Behalf Of Tal Liron
Sent: Wednesday, January 8, 2020 12:17 PM
To: tosca@lists.oasis-open.org
Subject: [tosca] tosca.datatypes.json and tosca.datatypes.xml

Â

These special types were introduced in TOSCA Simple Profile 1.2. Not much is said about them, but I am assuming that it is expected that parsers;

1. Validate that the content is indeed proper JSON or XML
2. Otherwise not change the data -- it should be left as a string (plain text)

This makes these types quite special indeed, because this kind of validation would have to be hardcoded in some way in the parser. There is no TOSCA constraint that could do that: unfortunately no simple regexp that could do the job. ;)

Â

Because we are going to separate the Simple Profile from the grammar, I think we have to choose from the following options:

1. Remove these types and implement this feature via a constraint. The "schema" constraint (section 3.6.3.2) seems like a good candidate, however from its description is seems like "schema" is intended to enhance these types, i.e. to add an XSD schema to "xml". Perhaps we can clarify and enhance this constraint: first argument would be "xml", "json", "yaml", etc., and the optional second argument would be the schema for that format.
2. Move these types to be part of the grammar, just like "string", "map", "list", etc. (and of course remove the "tosca.datatypes." prefix from their names). I would also suggest that we might want to add a "yaml" type, for plain-text embedded YAML.
3. Remove these types entirely: they can't be in the Simple Profile (not without special hardcoded parser support), and keeping them in the grammar will add unnecessary clutter.

I'm leaning on #1, if only because the "schema" constraint is totally unclear to me right now, and I do think this kind of validation mechanism could be useful.