Re: [sca-assembly] [NEW ISSUE] Assembly specification unclear onContribution vs Deployment - when can errors in artifacts be reported?

[Simon Nash <oasis@cjnash.com>]

See comments inline below.

   Simon

Jim Marino wrote:
> 
> On Jun 2, 2009, at 5:28 AM, Simon Nash wrote:
> 
>> Jim Marino wrote:
>>>>
>>>>
>>>>
>>>>
>>>>> > >> in sca-contribution.xml files within a Contribution. If an 
>>>>> artifact   > >> is deployed which has dependencies
>>>>> > >> on other artifacts, then those dependent artifacts are also 
>>>>> deployed.
>>>>> > >> When the SCA runtime has one or more deployable artifacts, 
>>>>> the   > >> runtime attempts to put those artifacts
>>>>> > >> and any artifacts they depend on into the Running state.  This 
>>>>> can   > >> fail due to errors in one or more of the artifacts
>>>>> > >> or the process can be delayed until all dependencies are 
>>>>> available.
>>>>> > >> 11.3.1
>>>>> > >> Checking for errors in artifacts MUST NOT be done for 
>>>>> artifacts in   > >> the Installed state (ie where the artifacts are
>>>>> > >> simply part of installed contributions] {ASM120xx]
>>>>> > > This seems over-restrictive.  For example, what about 
>>>>> malformed   > > artifacts
>>>>> > > such as .composite files with XML syntax errors?  It might be 
>>>>> useful   > > to
>>>>> > > inform the user about such problems when the artifacts are   > 
>>>>> > installed, but
>>>>> > > the proposed rule would apparently prohibit an SCA runtime from 
>>>>> even
>>>>> > > discovering these problems at installation time.  I think a 
>>>>> better   > > rule
>>>>> > > would be to allow such errors to be detected as long as this 
>>>>> does not
>>>>> > > prevent any artifacts from being deployed.  For example: Any 
>>>>> errors in
>>>>> > > artifacts in the Installed state (i.e., where the artifacts are 
>>>>> part   > > of
>>>>> > > installed contributions and have not been deployed) MUST NOT 
>>>>> prevent   > > the
>>>>> > > SCA runtime from deploying artifacts.
>>>>> > >
>>>>> >
>>>>> > I agree with the general statement that runtimes should be 
>>>>> allowed to   > detect syntactic errors upfront since that is what 
>>>>> users would expect   > and want (i.e. contributions should not be 
>>>>> installed if they contain   > invalid artifacts). However, I don't 
>>>>> follow the part about not   > preventing artifacts from being 
>>>>> deployed. Is it the case that an error   > in some random 
>>>>> contribution artifact A, does not prevent a composite C   > in the 
>>>>> same contribution from being deployed? Many users would want to   > 
>>>>> prevent that scenario. For example, I would expect most users 
>>>>> would   > only want "clean" contributions in a production environment.
>>>>> >
>>>>> I think (I hope) what Simon means is that it's ok for a runtime to
>>>>> detect errors at install time (if possible and reasonable) as long as
>>>>> it's also possible to go ahead and deploy those artifacts even when
>>>>> the deployment engine knows there are errors in the artifacts.
>>>> In Jim's scenario above, if A has a known error but C does not have
>>>> a known error, I think compliant runtimes should be allowed to deploy
>>>> C if requested.  I am leaning towards going further and saying that
>>>> compliant runtimes should be required to deploy C if requested.
>>>>
>>> I think requiring runtimes to deploy artifacts from contributions 
>>> that contain known errors would be very a bad thing to do. Here's why:
>>> 1. Most users have stringent regulation over production environments 
>>> and do not permit application artifacts containing detectable errors 
>>> to be deployed. In most cases I have seen, this also goes for 
>>> development environments. For example, good development environments 
>>> have mechanisms in place to verify artifacts such as integration 
>>> tests that fail a build if an error is detected. This avoids 
>>> accumulating broken artifacts and cruft in source repositories.  
>>> Requiring a compliant runtime to make a contribution containing 
>>> detectable errors available in a domain would negate being able to 
>>> lock down a production or development environment in this way.  2. 
>>> The random artifact A could be exported by the contribution. It is 
>>> possible for a composite contained in another contribution to be 
>>> deployed at some later date that references artifact A. The error in 
>>> A will surface at that later point. I think most people would prefer 
>>> the error to be raised at the time the contribution containing A was 
>>> "installed" in the domain. 3. It's very difficult to calculate with 
>>> absolute certainty that contribution C does not indirectly reference 
>>> A, particularly if A can be located dynamically in code. Providing 
>>> fail-fast behavior avoids runtime errors from surfacing where they 
>>> can cause more damage. I do agree compliant runtimes should be 
>>> allowed to deploy contributions containing invalid artifacts. 
>>> However, they should not be required to do so.
>>>> A more interesting question is if another composite B has a known
>>>> error, should compliant runtimes be required or allowed to deploy B
>>>> if requested?  I think this depends on the kind of error.  Some
>>>> errors might be severe enough to prevent the deployment from
>>>> succeeding.  Some errors might be less severe so that it would be
>>>> possible to allow deployment, even though some of the resulting
>>>> components would not be executable.  The SCA specs should define
>>>> which errors fall into which of these categories.
>>>>
>>> How can a runtime be certain an error in B will not cause a serious 
>>> problem if B is deployed? Can you provide examples of such errors? Jim
>> An example would be a misplaced Java annotation.  In this case, the
>> JavaCAA spec says that the SCA runtime MUST NOT run the component
>> which uses the invalid implementation code.  This implies to me that
>> other components in the same contribution can be deployed and run.
>>
>>  Simon
>>
>>
> 
> It may be useful to break this down into two parts. First, are you 
> saying that it should be possible for a runtime to selectively deploy a 
> composite where some components are in error, i.e. only deploy the 
> components contained in a composite that do not contain errors? I can't 
> imagine a case, even in development, where this would be useful. I 
> certainly can't imagine why someone would want this behavior in a 
> production environment.
> 
In this case, all the components would be deployed, but some of them
could not be run.  So if component B has a wire to component C, both
B and C are deployed and the wire is valid.  If B then makes an
invocation on C, this causes a runtime error.  If B never makes an
invocation on C, B is allowed to run with errors.

I see the above behaviour as one that runtimes should be allowed but
not required to support.  IMO it would also be valid for a runtime
to fail the deployment of the composite containing B and C.

Here's my proposal for what should be allowed and required.

1. Runtimes MAY allow contributions to be installed even though
    they contain known errors.  In this case, the runtime MAY detect
    and report the errors, and MUST perform the installation.

2. Runtimes MAY prevent installation of contributions containing
    known errors.

3. Some types of errors (defined by the spec) MUST be detected at
    deployment time.  In this case the composite containing the
    error MUST NOT be deployed, and the installed contribution
    containing the composite MUST NOT be uninstalled.

4. Some types of errors (defined by the spec) MAY be detected at
    deployment time and otherwise MUST be detected at runtime.
    If the error is detected at deployment time, the composite
    containing the error MUST NOT be deployed, and the installed
    contribution containing the composite MUST NOT be uninstalled.
    If the error is detected at runtime, the deployed composite
    containing the error MUST NOT be undeployed.

5. Some types of errors (defined by the spec) cannot be detected
    at deployment time and MUST be detected at runtime.  In this
    case the deployed composite containing the error MUST NOT be
    undeployed.

> For the second part, we can assume the error is in an artifact not used 
> by the composite being deployed. I'll start with a definition:  when a 
> contribution is in the installed state, its exports are visible in the 
> domain and deployables are ready for deployment.
> 
> I would expect the type of error identified above to be reported upfront 
> when an attempt is made to install the contribution. How can a runtime 
> be sure it is safe to install the contribution otherwise - i.e. latent 
> errors are ruled out? The only way it is safe is if the Java class can 
> never be used as a component implementation. This is very difficult to 
> determine at best. In order for this to work, the runtime would need to 
> use an algorithm similar to the following:
> 
> 1. Determine if the Java class is directly referenceable from outside 
> the contribution via an export/import. All applicable export/import 
> types must be examined
> 2. Introspect all composites and their hierarchies (includes and 
> implementation uses) in the contribution to calculate where the Java 
> artifact is reachable from
> 3. Determine if any of the composites are referenceable from outside the 
> contribution via an export/import. All applicable export/import types 
> must be examined
> 
> If any of these steps result in true, the runtime cannot guarantee 
> safety. This algorithm becomes much more complex for other artifacts - 
> for example, an artifact referenced by qname several layers deep in a 
> hierarchy.  It's also worth noting performing this calculation is likely 
> to severely impact install time.
> 
> Of course, a runtime could choose not to perform this check and install 
> the contribution, throwing an error later if an attempt is made to use 
> the artifact.
 >
This would be my preference.  I agree that trying to guarantee safety
wouldn't be practical.

 >               However, I personally wouldn't want to use a runtime that
> does that. Consider the following scenario:
> 
> 1.  Contribution A is installed with composite C and Java implementation 
> J. A exports J's package. C does not reference J. J contains an error as 
> described above.
> 2.  C is deployed
> 3. At a later point in time, contribution B is installed containing 
> composite C2. B imports J's package and the runtime resolves the import 
> to contribution A's export. C2 uses J as a component implementation.
> 4. An attempt is made to deploy C2, which surfaces the error in J.
> 
> At this point there is a serious problem as the error in J cannot be 
> corrected without undeploying C. This could potentially be fixed by 
> deploying a new contribution with an updated version of J but if B's 
> import were fixed to a specific version of J's package, there is no way 
> to guarantee the import will resolve to the contribution export 
> containing the fixed version of J. In other words, we have now 
> introduced a serious issue in a production environment that is entirely 
> avoidable.
> 
IMO this is a corner case that is outweighed by the benefit of allowing
contributions with errors to be installed.

> One way to avoid this is to put the onus on users to properly structure 
> contributions and have good test coverage. In the above scenario, A has 
> mixed concerns and should probably be separated into distinct 
> contributions.
 >
This would be my preferred solution for the corner case.

>                A better way to avoid this is by allowing a runtime to 
> report the error upfront and not install contribution A. This won't 
> catch all cases (some errors depend on the state of the domain and must 
> be detected at deployment), but it will catch a lot of them.
> 
I don't think this is "better", but I also don't think SCA should
disallow it.

> I think we need to take a step back and ask: why would we *require* a 
> runtime to deploy a contribution in the above scenario? I'm happy to 
> allow runtimes to do this but it should not be the required or mandated 
> default behavior. In fact, most people I know would never want this to 
> be possible in a production or staging environment. Even in development, 
> most people I know want fail-fast behavior because they practice 
> iterative development and have lightweight containers that boot quickly.
> 
My preference for development would be different, but I think SCA
should allow runtimes to support either approach.

   Simon

> Jim
> 
> ---------------------------------------------------------------------
> To unsubscribe from this mail list, you must leave the OASIS TC that
> generates this mail.  Follow this link to all your TCs in OASIS at:
> https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php
> 
>