ISSUE 122: ChangeSummary and Projection

["Barack, Ron" <ron.barack@sap.com>]

Title: ISSUE 122: ChangeSummary and Projection

Hi Everyone,

As a second component regarding ChangeSummary and non-containment graphs, I'd like to address how I think projection can be used in this regard.  Note that this is not a proposal, it's just some thoughts about what direction to go…there are some details to be worked out.   Although as always I consider backwards compatibility to be essential, there's actually not much of a problem here.  Since the "project()" operation did not exist in 2.1, no existing code is using it, so whatever "effect" we define for projection has on change summary, we will not break any existing code.  So backwards compatibility in this case boils down to saying that within a single HelperContext, including XML marshalling and unmarshalling, there must be no changes to change summary behavior.

First, some background regarding projection.  Containmainment is a central concept in SDO, corresponding to UML aggregation.  However, in practice, containment is often used to control the behavior of SDO functionality rather than the inherent structure of the model.  Two instances where containment is (mis-)used in this way are (a) containmaint is used to define the structure of the XML document to which the SDO should be serialized and (b) containment is used to define the "scope" of a change summary.  Projection allows an alternate containment structure to be imposed on an existing model, with the so that the model seen and manipulated by users is not necessarily determined by such factors.

In particular, I believe the most important use-case for ChangeSummary is that ChangeSummaries are consumed by a DAS backend that uses the list of changed objects (and potentially the old values associated with them) to optimize database access. If we imagine a DAS that is based on JPA, then without the CS, the DAS would have to pull the entire graph into the persistence context, effectively calling EntityManager.merge on every object or maybe calling EntityManager.persist when the object is not already in the store.  With the help of a ChangeSummary, an update operation becomes:

        public void  update(POShoppingCart cart) {
                DataObject cartObj = (DataObject)cart;
                ChangeSummary cs = cartObj.getChangeSummary();
                if (cs!=null) {
                        List<DataObject> changes = cs.getChangedDataObjects();
                        for (DataObject o: changes) {
                                if (cs.isModified(o)) {
                                        em.merge(o);
                                } else if (cs.isDeleted(o)) {
                                        em.remove(o);
                                } else if (cs.isCreated(o)) {
                                        em.persist(o);
                                }
                        }
                }
        }

The reason why projection comes into the picture is that a DAS that sits over an RDB will probably want its model to reflect the RDB, and for models based on RDBs (or on JPA persistent objects) will not have much containment.  So the types seen by the DAS should not necessarly have containment properties.  But without containment properties, getChangedDataObjects is a useless operation.  The goal of this proposal is to provide some meaningful and useful behavior for getChangedDataObjects() in such use-cases.

As in my consideration of dealing with OrphanHolders, I will define the ChangeSummary behavior in terms of diffs on data graphs, ie, comparisons of their before and after states.  This does not imply a particular implementation strategy: implementations may also work by some kind of change logging or by marking the "in-scope" DataObjects.

It is important to note that the containment structure has two effects on the change summary.  Besides defining the scope (the set of DataObjects whose before and after states are considered, the containment structure also determines whether data objects are considered to be created (new in the containment structure) or deleted (removed from the containment structure).  The same set of operations can produce creates and deletes in one projections, and only modify's in another projection.

SDO 2.1 has the restriction that an object may be "in-scope" of at most one change summary.  I would like to loosen this a little in regard to projection.  I would say that an object may be in scope of at most one change summary per projection, and that at most one of the projections may have logging enabled.  To say it another way, not matter how many projections of an object exist, the SDO implementation mantains at most one "before-image" of any object.

As long as we are in the projection from which beginLogging was called, everything remains the same as in 2.1.  The question is, what happens when the change summary is queried from another helper context.  The direction I would suggest going is that the change summary (in particular, the list returned by getChangedObjects) be calculated diffing the current and the old-state in the projection in which beginLogging was called. Of course, the DataObjects themselves should be projected into the querying HelperContext.  In a way, this makes the behavior of CS consistent with the behavior normally associated with projection.  The CS is itself a type of DataObject, it has properties (changedObjects in this case can be viewed as a multivalued property) and those properties have values that do not change, regardless of the projection from which they are accessed.  Using the projection in which "beginLogging" was called as the basis for calculating the delta assures that the "before-image" and the "after-image" can be meaningfully compared. 

The situation becomes significantly more complex if there is an XML serialization in the picture.  Since we want to maintain backwards compatibility I do not think we can change the change summary serialization/ deserialization algorithm.  This means that the if the result of an XMLHelper.load is a DataObject with logging enabled, then the "before image" is based on the XML document, that is, the HelperContext used to calculate the change summary is the HelperContext that owns the XMLHelper.   Symettrically, I believe we also need a restriction that the XMLHelper.save be called from the same context from which "beginLogging" was called.  The bottom line here is that the structure used for XML serialization must be consistent with the structure used to calculate the change summary scope.  I would say calling XMLHelper.save in a different HelperContext is a user-error that implementations MAY handle, must this would be non-standard behavior.

To me, this is a pretty unfortunate limitation, but one I can live with at least for the SDO 3.0 timeframe.  If the group decides this is too restrictive, then it may be possible to expand the change summary serialization in some way, as a topic for further research.

Best Regards,
Ron