We've reached a general agreement that "profiles" should no longer be implicitly imported in TOSCA and instead would require users to explicitly import them. This will effectively decouple the Simple Profile from the language grammar, making it optional.
But we're still not sure how to achieve this explicit import exactly, nor what exactly would be entailed. Here are some ideas and topics we can discuss tomorrow.
First, the easy part:
Specifying Grammar
tosca_definitions_version: tosca-1.3
This could be the convention instead of "tosca_simple_yaml_1_3", "tosca_simple_profile_for_nfv_1_0", etc. We don't want the profile name here. And we definitely don't need that "yaml" insertion. Note how this immediately removes the confusion from the NFV profile, where the "1_0" version doesn't tell you which version of the TOSCA language it is.
Why keep the word "tosca" there? Well, to clarify for alternative "TOSCA-like" grammars to plug in. For example, Cloudify uses the value "cloudify_dsl_1_3" (and Puccini supports it, too). Why dash instead of underscore? Just my personal preference for values vs. keywords. :) Underscores look too much like keywords to me.
Ideally, I would like to change the keyword itself, too, because we are no longer referring to "definitions". "tosca_grammar_version" would make more sense to me now. However, I think that for backward compatibility it might be best to keep this keyword as is, because it is the starting point of most parsers. I guess we're stuck with it!
So, how do we explicitly import "simple_profile" (instead of "simple_yaml") and "simple_profile_for_nfv"? How do we specify the version we want? Here are some ideas for the "imports" section:
Bare Filenames
If we assume that "imports" are relative URLs (as previously discussed) then we can also assume that parsers would place built-in profiles relative to the current location. So something like "file: simple-profile-1.6.yaml" could work.
There is one advantage to this: it doesn't change the existing "imports" grammar. This would allow for cleaner backwards compatibility. However, it is bad in every other aspect. It creates a potentially ambiguous overlap with actual files in the CSAR and on remote URLs. And it's not immediately clear to readers that this a "built-in" profile. Errors messages would also be not very helpful: "file/URL not found" doesn't tell the user much about available profiles.
Keywords
Variation on the above. Instead of filenames, it would just be special reserved worlds, e.g. "simple-profile-1.6". Without the ".yaml" suffix this doesn't look like a filename anymore, and thus users would not confuse it with a file. Another variation would some sort of character prefix, e.g. "@simple-profile-1.6". But this adds other annoyances, such as having parsers explicitly disallowing users from having actual filenames with this prefix.
"profile:" Keyword
Another option is to change the grammar itself. "imports:" right now has "file:" as required, but we can make it so that it can have either "file:" or "profile:" (new keyword) as required. So:
imports:
- file: mytypes.yaml
- profile: simple-1.6
The obvious advantage is complete clarity for the user, as well as for parsers (it would trigger a separate codepath). It also opens up the possibility that importing a "profile" could be technically different from importing a "file". For example it could include grammatical extensions as well as just imported types. This is not something we have to decide on now, but it opens up the possibility for evolving the "profile" concept in the future.
One immediate issue I can see is that we would need to decide what to do with the short notation. Right now, importing a string is equivalent to importing a "file:". It could make sense to keep it that way for reasons of familiarity, so that if someone does want to import a profile they would have to explicitly specify so, e.g.:
imports:
- mytypes.yaml
- profile: simple-1.6
This doesn't seem too bad to me. But perhaps I'm missing some possible annoyance or disadvantage.
"profile:" URL Scheme
A variation on the above. If we assume that the "file:" keyword in "imports" refers to a URL (absolute or relative), then we can define a custom URL scheme for profiles. E.g.:
imports:
- mytypes.yaml # this is a relative URL, relative to the current document
(Note the lack of a space between "profile:" and "simple-1.6" -- this is a string, not a YAML dict.) The end result is essentially identical to that of using a keyword, however it could make things more difficult for implementers. For example, if you're using a library to handle URL parsing, you might have to circumvent that library with a special check for the "profile:" scheme, and not all URL libraries would allow for this. Another potential awkwardness is that URLs have some quirky rules for character sets and encodings, and also reserve special characters such as "/" and ":". E.g. a space might be escaped in a roundtrip as "%20". This is not such a big deal, but we can avoid the whole topic easily by just making it a keyword, allowing the string to be anything.
Required Grammar Versions for Profiles
Decoupling the profile from the grammar introduces an issue. Until now, profiles were intrinsically tied to the grammar, but if they are separate we introduce potential incompatibilities. For example, what would happen if a lower version of a grammar imports a profile designed around a higher version grammar?
The simplest solution would be to keep the coupling, semantically. So, if the profile has "tosca_definitions_version: tosca-1.4", it would mean that the importing file would have to have the exact same grammar version. Simple, and also safest. But there as an obvious disadvantage: a profile could potentially be designed to support various versions of the grammar. For example, if it doesn't use any grammar 1.4 features, why not let it work on both grammar 1.3 and 1.4? Or maybe it could have slightly different variations for grammar 1.3 and 1.4? This could end up being very important as TOSCA becomes more popular and we have more and more TOSCA products out there that might not progress as quickly as others in terms of embracing new grammar versions. A profile author might want to support those products, too.
This feature could easily become a rabbit hole, as profiles could start developing "#ifdef"-like support for varying according to grammar versions. You find horribly unreadable messes like this in quite a few programming languages and their library use, so I hope we can find a solid system for this from the start.
In my view, the best solution is for profile authors to provide different versions of the profile for each grammar version they want to support. How they do this is up to them, but it shouldn't be hard to do, either by manual copy-pasting or by using whatever templating system is desired (Jinja2, etc.). The parser would then know to import the correct variation of the profile per grammar versions. These means that whatever name is used for the profile (e.g. "simple-1.6") it would actually refer to a table matching grammar version to profile content. A simple lookup, and we're good to go.