Re: [sca-assembly] [NEW ISSUE] Assembly specification unclear on Contribution vs Deployment - when can errors in artifacts be reported?

[Jim Marino <jim.marino@gmail.com>]

hi,

Sorry for the delay in responding ot this but I have been traveling  
and without a properly functioning internet connections. I do have  
comments inline but I have the feeling that we are lacking definitions  
of the various contribution states. I'll define two states to clarify  
my comments below:

1. When a contribution is in the installed state, its exports are  
visible in the domain and deployables are ready for deployment
2. When a contribution is in the deployed state, its components are  
available to service a request on one or more runtimes in the domain

Jim

On Jun 5, 2009, at 4:51 AM, Simon Nash wrote:

> See comments inline below.
>
>  Simon
>
> Jim Marino wrote:
>> On Jun 2, 2009, at 5:28 AM, Simon Nash wrote:
>>> Jim Marino wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> > >> in sca-contribution.xml files within a Contribution. If an  
>>>>>> artifact   > >> is deployed which has dependencies
>>>>>> > >> on other artifacts, then those dependent artifacts are  
>>>>>> also deployed.
>>>>>> > >> When the SCA runtime has one or more deployable artifacts,  
>>>>>> the   > >> runtime attempts to put those artifacts
>>>>>> > >> and any artifacts they depend on into the Running state.   
>>>>>> This can   > >> fail due to errors in one or more of the  
>>>>>> artifacts
>>>>>> > >> or the process can be delayed until all dependencies are  
>>>>>> available.
>>>>>> > >> 11.3.1
>>>>>> > >> Checking for errors in artifacts MUST NOT be done for  
>>>>>> artifacts in   > >> the Installed state (ie where the artifacts  
>>>>>> are
>>>>>> > >> simply part of installed contributions] {ASM120xx]
>>>>>> > > This seems over-restrictive.  For example, what about  
>>>>>> malformed   > > artifacts
>>>>>> > > such as .composite files with XML syntax errors?  It might  
>>>>>> be useful   > > to
>>>>>> > > inform the user about such problems when the artifacts  
>>>>>> are   > > installed, but
>>>>>> > > the proposed rule would apparently prohibit an SCA runtime  
>>>>>> from even
>>>>>> > > discovering these problems at installation time.  I think a  
>>>>>> better   > > rule
>>>>>> > > would be to allow such errors to be detected as long as  
>>>>>> this does not
>>>>>> > > prevent any artifacts from being deployed.  For example:  
>>>>>> Any errors in
>>>>>> > > artifacts in the Installed state (i.e., where the artifacts  
>>>>>> are part   > > of
>>>>>> > > installed contributions and have not been deployed) MUST  
>>>>>> NOT prevent   > > the
>>>>>> > > SCA runtime from deploying artifacts.
>>>>>> > >
>>>>>> >
>>>>>> > I agree with the general statement that runtimes should be  
>>>>>> allowed to   > detect syntactic errors upfront since that is  
>>>>>> what users would expect   > and want (i.e. contributions should  
>>>>>> not be installed if they contain   > invalid artifacts).  
>>>>>> However, I don't follow the part about not   > preventing  
>>>>>> artifacts from being deployed. Is it the case that an error   >  
>>>>>> in some random contribution artifact A, does not prevent a  
>>>>>> composite C   > in the same contribution from being deployed?  
>>>>>> Many users would want to   > prevent that scenario. For  
>>>>>> example, I would expect most users would   > only want "clean"  
>>>>>> contributions in a production environment.
>>>>>> >
>>>>>> I think (I hope) what Simon means is that it's ok for a runtime  
>>>>>> to
>>>>>> detect errors at install time (if possible and reasonable) as  
>>>>>> long as
>>>>>> it's also possible to go ahead and deploy those artifacts even  
>>>>>> when
>>>>>> the deployment engine knows there are errors in the artifacts.
>>>>> In Jim's scenario above, if A has a known error but C does not  
>>>>> have
>>>>> a known error, I think compliant runtimes should be allowed to  
>>>>> deploy
>>>>> C if requested.  I am leaning towards going further and saying  
>>>>> that
>>>>> compliant runtimes should be required to deploy C if requested.
>>>>>
>>>> I think requiring runtimes to deploy artifacts from contributions  
>>>> that contain known errors would be very a bad thing to do. Here's  
>>>> why:
>>>> 1. Most users have stringent regulation over production  
>>>> environments and do not permit application artifacts containing  
>>>> detectable errors to be deployed. In most cases I have seen, this  
>>>> also goes for development environments. For example, good  
>>>> development environments have mechanisms in place to verify  
>>>> artifacts such as integration tests that fail a build if an error  
>>>> is detected. This avoids accumulating broken artifacts and cruft  
>>>> in source repositories.  Requiring a compliant runtime to make a  
>>>> contribution containing detectable errors available in a domain  
>>>> would negate being able to lock down a production or development  
>>>> environment in this way.  2. The random artifact A could be  
>>>> exported by the contribution. It is possible for a composite  
>>>> contained in another contribution to be deployed at some later  
>>>> date that references artifact A. The error in A will surface at  
>>>> that later point. I think most people would prefer the error to  
>>>> be raised at the time the contribution containing A was  
>>>> "installed" in the domain. 3. It's very difficult to calculate  
>>>> with absolute certainty that contribution C does not indirectly  
>>>> reference A, particularly if A can be located dynamically in  
>>>> code. Providing fail-fast behavior avoids runtime errors from  
>>>> surfacing where they can cause more damage. I do agree compliant  
>>>> runtimes should be allowed to deploy contributions containing  
>>>> invalid artifacts. However, they should not be required to do so.
>>>>> A more interesting question is if another composite B has a known
>>>>> error, should compliant runtimes be required or allowed to  
>>>>> deploy B
>>>>> if requested?  I think this depends on the kind of error.  Some
>>>>> errors might be severe enough to prevent the deployment from
>>>>> succeeding.  Some errors might be less severe so that it would be
>>>>> possible to allow deployment, even though some of the resulting
>>>>> components would not be executable.  The SCA specs should define
>>>>> which errors fall into which of these categories.
>>>>>
>>>> How can a runtime be certain an error in B will not cause a  
>>>> serious problem if B is deployed? Can you provide examples of  
>>>> such errors? Jim
>>> An example would be a misplaced Java annotation.  In this case, the
>>> JavaCAA spec says that the SCA runtime MUST NOT run the component
>>> which uses the invalid implementation code.  This implies to me that
>>> other components in the same contribution can be deployed and run.
>>>
>>> Simon
>>>
>>>
>> It may be useful to break this down into two parts. First, are you  
>> saying that it should be possible for a runtime to selectively  
>> deploy a composite where some components are in error, i.e. only  
>> deploy the components contained in a composite that do not contain  
>> errors? I can't imagine a case, even in development, where this  
>> would be useful. I certainly can't imagine why someone would want  
>> this behavior in a production environment.
> In this case, all the components would be deployed, but some of them
> could not be run.  So if component B has a wire to component C, both
> B and C are deployed and the wire is valid.  If B then makes an
> invocation on C, this causes a runtime error.  If B never makes an
> invocation on C, B is allowed to run with errors.
>
I'm not sure I would define C as "deployed" in this scenario. I would  
describe deployment as the process of making components in a composite  
available to service requests on one or more runtimes. However, I  
don't think we need to argue that point at the moment. Regardless of  
how one defines deployment, in this scenario, B can't be deployed  
(using either definition of the term) if B's reference to C is  
required. Since a functioning wire cannot be established, it is not  
safe for the runtime to allow B to service requests. As a developer  
of  a service, I would want to know upfront if a wire is not valid.  
Otherwise, there is no way to determine this without making a service  
invocation.

> I see the above behaviour as one that runtimes should be allowed but
> not required to support.  IMO it would also be valid for a runtime
> to fail the deployment of the composite containing B and C.
>
> Here's my proposal for what should be allowed and required.
>
> 1. Runtimes MAY allow contributions to be installed even though
>   they contain known errors.  In this case, the runtime MAY detect
>   and report the errors, and MUST perform the installation.
>
I don't think the runtime should be required to proceed with the  
installation as it leads to the same problems I outlined previously.  
This seems to be in contradiction to statement 2 below, but I am  
probably reading it incorrectly. I think it is very important that a  
runtime be allowed to fail an installation if the contribution  
contains known errors. By contribution installation, I mean the  
process by which exported contribution artifacts become available for  
import by other contributions.

> 2. Runtimes MAY prevent installation of contributions containing
>   known errors.
>
> 3. Some types of errors (defined by the spec) MUST be detected at
>   deployment time.  In this case the composite containing the
>   error MUST NOT be deployed, and the installed contribution
>   containing the composite MUST NOT be uninstalled.
>
Sorry, I'm not sure I understand this. Is this saying that the runtime  
should not attempt uninstallation without operator intervention?  
Consider the following sequence:

---> install contribution ---> deploy composite ---> X error

There needs to be a way to recover and for a contribution containing  
an erroneous composite to be uninstalled:

---> uninstall ---> install fixed contribution ---> deploy composite  
---> success

The uninstall operation should be a valid operation that can be  
performed by an administrator/deployer person.

> 4. Some types of errors (defined by the spec) MAY be detected at
>   deployment time and otherwise MUST be detected at runtime.
>   If the error is detected at deployment time, the composite
>   containing the error MUST NOT be deployed, and the installed
>   contribution containing the composite MUST NOT be uninstalled.
>   If the error is detected at runtime, the deployed composite
>   containing the error MUST NOT be undeployed.
>

I think this is too vague. Some runtimes (like Fabric3) have the  
notion of transactional deployment. It should be permissible for  
"runtime" errors encountered during a deployment to result in a  
rollback of the domain state to what it was prior to the deployment  
operation.

> 5. Some types of errors (defined by the spec) cannot be detected
>   at deployment time and MUST be detected at runtime.  In this
>   case the deployed composite containing the error MUST NOT be
>   undeployed.
>
>> For the second part, we can assume the error is in an artifact not  
>> used by the composite being deployed. I'll start with a  
>> definition:  when a contribution is in the installed state, its  
>> exports are visible in the domain and deployables are ready for  
>> deployment.
>> I would expect the type of error identified above to be reported  
>> upfront when an attempt is made to install the contribution. How  
>> can a runtime be sure it is safe to install the contribution  
>> otherwise - i.e. latent errors are ruled out? The only way it is  
>> safe is if the Java class can never be used as a component  
>> implementation. This is very difficult to determine at best. In  
>> order for this to work, the runtime would need to use an algorithm  
>> similar to the following:
>> 1. Determine if the Java class is directly referenceable from  
>> outside the contribution via an export/import. All applicable  
>> export/import types must be examined
>> 2. Introspect all composites and their hierarchies (includes and  
>> implementation uses) in the contribution to calculate where the  
>> Java artifact is reachable from
>> 3. Determine if any of the composites are referenceable from  
>> outside the contribution via an export/import. All applicable  
>> export/import types must be examined
>> If any of these steps result in true, the runtime cannot guarantee  
>> safety. This algorithm becomes much more complex for other  
>> artifacts - for example, an artifact referenced by qname several  
>> layers deep in a hierarchy.  It's also worth noting performing this  
>> calculation is likely to severely impact install time.
>> Of course, a runtime could choose not to perform this check and  
>> install the contribution, throwing an error later if an attempt is  
>> made to use the artifact.
> >
> This would be my preference.  I agree that trying to guarantee safety
> wouldn't be practical.
>
> >               However, I personally wouldn't want to use a runtime  
> that
>> does that. Consider the following scenario:
>> 1.  Contribution A is installed with composite C and Java  
>> implementation J. A exports J's package. C does not reference J. J  
>> contains an error as described above.
>> 2.  C is deployed
>> 3. At a later point in time, contribution B is installed containing  
>> composite C2. B imports J's package and the runtime resolves the  
>> import to contribution A's export. C2 uses J as a component  
>> implementation.
>> 4. An attempt is made to deploy C2, which surfaces the error in J.
>> At this point there is a serious problem as the error in J cannot  
>> be corrected without undeploying C. This could potentially be fixed  
>> by deploying a new contribution with an updated version of J but if  
>> B's import were fixed to a specific version of J's package, there  
>> is no way to guarantee the import will resolve to the contribution  
>> export containing the fixed version of J. In other words, we have  
>> now introduced a serious issue in a production environment that is  
>> entirely avoidable.
> IMO this is a corner case that is outweighed by the benefit of  
> allowing
> contributions with errors to be installed.
>

I think this happens more frequently than we are lead to believe. In  
any event, even if it is a corner case, it is important to allow  
runtimes the opportunity to handle it properly given the serious  
problems that may arise if it happens.

>> One way to avoid this is to put the onus on users to properly  
>> structure contributions and have good test coverage. In the above  
>> scenario, A has mixed concerns and should probably be separated  
>> into distinct contributions.
> >
> This would be my preferred solution for the corner case.
>
>>               A better way to avoid this is by allowing a runtime  
>> to report the error upfront and not install contribution A. This  
>> won't catch all cases (some errors depend on the state of the  
>> domain and must be detected at deployment), but it will catch a lot  
>> of them.
> I don't think this is "better", but I also don't think SCA should
> disallow it.
>
I think we just are going to need to agree to disagree here. The  
important thing is the specification should accommodate both  
approaches and allow implementors the freedom to choose the strategy  
that best suits their users.

>> I think we need to take a step back and ask: why would we *require*  
>> a runtime to deploy a contribution in the above scenario? I'm happy  
>> to allow runtimes to do this but it should not be the required or  
>> mandated default behavior. In fact, most people I know would never  
>> want this to be possible in a production or staging environment.  
>> Even in development, most people I know want fail-fast behavior  
>> because they practice iterative development and have lightweight  
>> containers that boot quickly.
> My preference for development would be different, but I think SCA
> should allow runtimes to support either approach.
>
>  Simon
>
>> Jim
>> ---------------------------------------------------------------------
>> To unsubscribe from this mail list, you must leave the OASIS TC that
>> generates this mail.  Follow this link to all your TCs in OASIS at:
>> https://www.oasis-open.org/apps/org/workgroup/portal/ 
>> my_workgroups.php
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this mail list, you must leave the OASIS TC that
> generates this mail.  Follow this link to all your TCs in OASIS at:
> https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php