Dear group,
I apologize for missing the meeting today. As you may have heard some things happened at Red HatÂ+ IBM and we were a bit busy. :)
But, I have taken some time the past weekend to think about the problem and try to create an example. I even started adding some TOSCA 1.3 features to Puccini in order to help me play around with templates.
Ultimately, I could not come up with what seemed to me like a reasonable example. Here are my thoughts. I very much hope that they will be read in a constructive tone.
1) I don't think TOSCA should define workflows at all, and I objected to the inclusion of this feature early on. TOSCA workflows enforce a very specific orchestration model based on step-by-step operations in serial or parallel. There are few orchestrators that use such a model natively (notably Cloudify and BPMN middleware, to an extent), and other orchestrators could possibly be retrofitted to support it with some kind of adapter layer. But, such adaptations are an offensive overhead. They hide away the orchestrator's true strengths and introduce considerable complexity and significant points of potential failure.
TOSCA interfaces and operations are more reasonable. Their calling scope is a node or a relationship or a group, the atomic units that any TOSCA orchestrator would have to implement in one way or another, whether its a logical record or a physically provisioned resource. They provide entry points for the orchestrator without enforcing an orchestration model. (More on this in 3, below.)
2) The example we are seeing from ETSI convinces me of this even more. If an ETSI implementation had "true" TOSCA orchestration, that would mean that it could fully run TOSCA workflows. So ... why would it need an external Mistral workflow? Shouldn't TOSCA be used instead? And if TOSCA can't even do a better job than Mistral in this case (and I believe it could never catch up with native orchestration features), then the question as to why we need a workflow grammar at all is even more glaring.
3) With that said, I do think TOSCA can and should go beyond interfaces to increase the operational contact with orchestrators. Rather than talk about "workflows", I think it's better to use control language, specifically "loops". In open loops, control is delegated between subsystems. That, to me, seems to be the general problem we are trying to solve here.
Open loops need full-duplex communication. Right now in TOSCA we have mostly have a one-way street via interfaces: from orchestrator to our template (well, there are return values via get_operation_output, but that's still rather poorly defined). We do have an implementation of the other direction: policy triggers. They are useful, but it seems to me like we can do more.
My current thinking is that it may be useful to define "events" in the TOSCA template. Events would be defined in relation to various TOSCA resources, not just policies, and they could optionally include a "trigger" condition but also stay open ended, left for the orchestrator to implement as is appropriate to its technology. So, anywhere you can put an interface (node, relationship, group) you would also be able to put "events". Example:
node_types:
 MyDBMS:
  derived_from: DBMS
  interfaces:
   Backup:
    type: Backup
  events:
   OutOfResources:
    type: OutOfResources
event_types:
 OutOfResources:
  properties:
   timestamp:
    type: timestamp
The orchestrator would know that nodes of this type could potentially send such an event, in which case it would take control. What it actually does -- run a "workflow"? turn on a subsystem? -- is implementation-specific. We can provide a richer grammer of hints as to what should happen next, for example to call an operation, trigger another event upon success or failure (closed loop), etc. So, many of things that we define right now in workflow step activities could be defined in events, instead. As with interfaces, the scope is limited to this particular node and we are not enforcing any behavioral model on how the orchestrator should work.
I'll be on leave for next week's meeting, but am happy to discuss this on the mailing list if there is interest.