Hi Blaise,
Thanks a lot for your comments. My attempts at
answers inline...
Ron
Hi Ron,
Are we introducing a requirement that
the change summary scope of the data object in the original helper context match
the change summary scope in the helper context the data object is projected
into?
no, the scope of the original (beginLogging)
projection does not have to match the scope of the projection in which
getChangeSummary() was called. The whole idea is
that beginLogging can be called in a context where a containment structure
has been defined (eg, the metadata comes from an XSD) but the change
summary is inspected in a context where there are very few
containment relationships (as will be typical if the metadata comes from
JPA). The scope as calculated from the JPA model will probably
contain only the root node. The scope as calculated from the XSD
model will be more reasonable. So the scopes are not
identical.
I think that there is a restriction when
serializing to XML. You can only serialize the CS to XML if the
scope as defined by the context where beginLogging() was called
matches the projection where XMLHelper.save is called. This is easiest to see if you imagine the simple
case where beginLogging is called, and then the XML shipped over the wire to a
client. The client will see the beginLogging flag is on, and will
naturally assume that beginLogging was called in the current projection (ie, in
the context that calls XMLHelper.load()). We'd have to adjust our
XML format if we want to specify which projection should be used. This is
if course possible, but I want to get this simple case defined
first.
Things get a little messy if
there is projection over keys involved. The context where
getChangeSummary().getChangedObjects() is called should be at least as detailed
as the context where beginLogging() was called. Imagine 2 contexts, both
of which have an "AddressType", but in one context AddressType has a
relatationship to the type "CountryType" while in
the other context AddressType has a string property holding the key value of the
referenced CountryType. If I call beginLogging in the context with
"CountryType", modify a property of a Country, and then call
getChangeSummary.getChangedObjects in the context where "CountryType" is not
defined, then we won't be able to project the element in this list into the
appropriate context. To me, this is an untroubling user-error, a
corner-case. The use-cases I'm imagining are that the ChangeSummary is
being consumed by the DAS, and the DAS is the one with the complexer type
system. Things are trimmed (through introducing keys) when transporting to
clients.
Some
comments on your proposal:
4.14.4 ChangeSummary and Projection
The
scope of a ChangeSummary is defined by the containment structures and
orphanHolders of the projection in which ChangeSummary.beginLogging() was
called.
What does this sentence really mean? Does
it mean that regardless where the data graph is projected into it still obeys
the change summary rules of the original helper context?
It means the scope is defined by the original context. If
you imagine an implementation that marks objects as being "in-scope" by setting
a flag, it means that beginLogging will run along marking
objects according to the containment structures and orphan holders of
the context in which it was called. And it means that the change summary
must somehow remember the context in which beginLogging was
called, so that this same context can be used when endLogging/
getChangedObjects is called.
That is, the scope is determined
by the HelperContext of the DataObject from which the ChangeSummary was
retrieved (either through getChangeSummary() or as the value of a property
having ChangeSummaryType). At most one projection at a time may be actively
logging: calling ChangeSummary.beginLogging in a second projection results in
undefined behavior. After change logging has been terminated with a call
to ChangeSummary.endLogging(), a further call to ChangeSummary.beginLogging() in
any projection MUST reset the change state of all objects in the scope as
defined by either projection, the list of changed objects MUST be cleared and
the old values of the first logging session MUST no longer be
visible.
Assuming the "second projection" is the
active projection, why wouldn't the beginLogging call just reset the change
summary?
In
2.1.1, calling beginLogging() twice (without calling endlogging) is a
no-op, so I would say that this is the behavior we want when beginLogging is
called twice in the same context. Maybe we should
just extend this, and say calling beginLogging twice (regardless of context) is
a no-op. On the other hand, this seems like a user error to me,
intuitively, I want to throw an exception here. So now we have 3 potential
behaviors, and I can see reasons for each of them. Being undecisive, I
took the cowards way out and left it
undefined.
Changes MUST tracked for all changes in scope of
an active change summary, regardless of the projection in which the changes are
made.
The ChangeSummary, and its properties, follow the
normal rules associated with the projection of DataObjects and property values.
The ChangeSummary’s contents MUST be visible in all projections, not only in
the projection from which beginLogging was called. The list returned by
ChangeSummary.getChangedObjects() MUST contain the same underlying business
objects, regardless of whether the ChangeSummary is being inspected in the
projection in which beginLogging was called, or another projection. The elements
in the list MUST be consistent (i.e., projected into) the context of the
DataObject from which the ChangeSummary was retrieved.
The following example, based on the metadata from
the previous section’s example, illustrates this
behavior.
// Create an envelope DataObject containing a change summary.
DataObject dataGraph =
_helperContext.getDataFactory().create(SDO_URI,"DataGraphType");
// Create the SDO graph root node
DataObject cal =
_helperContext.getDataFactory().create(School.class);
// Add it to the graph
dataGraph.add("school",cal);
// Fill the data graph
cal.set("name","Berkeley");
DataObject billy =
cal.createDataObject("students");
billy.set("name", "Billy
Bear");
DataObject bob = cal.createDataObject("students");
bob.set("name",
"Bobbie Bear");
DataObject basketWeaving =
cal.createDataObject("courses");
basketWeaving.set("name", "Basket
Weaving");
DataObject algol =
cal.createDataObject("courses");
algol.set("name", "Algol");
DataObject
revolution = cal.createDataObject("courses");
revolution.set("name",
"Revolution");
// hook things
up
billy.getList("courses").add(basketWeaving);
billy.getList("courses").add(algol);
bob.getList("courses").add(basketWeaving);
bob.getList("courses").add(revolution);
// Create a second context defined by an XSD
HelperContext hc2 =
SDO.getHelperContextFactory().createHelperContext();
hc2.getXSDHelper().define(getClass().getClassLoader().getResourceAsStream("com/sap/sdo/testcase/internal/pojo/ex/projection.xsd"),
null);
// Project from the java context to the XSD context
DataObject projection
= hc2.getDataFactory().project(cal);
// Turn on change logging in the projected DataObject
projection.getChangeSummary().beginLogging();
// Make a few changes in the logging context
projection.set("name","Stanford");
DataObject billy2 = (DataObject)projection.getList("students").get(0);
billy2.set("name","Chauncy Cardinal");
// Make the
original context active by projecting back to it
cal =
_helperContext.getDataFactory().project(projection);
// Inspect
the change summary
ChangeSummary
changeSummary = cal.getChangeSummary();
assert.equals(2,
changeSummary.getChangedObjects().size());
assert.true(changeSummary.getChangedObjects().contains(cal));
assert.true(changeSummary.getChangedObjects().contains(billy));
-Blaise
Barack, Ron wrote:
7C3EF93EEBC6EB4A8B4470853DE86566E7C8E8@dewdfe18.wdf.sap.corp
type="cite">
Hi Everyone,
Here is a draft resolution for this issue. It is a
new sub-section of the section describing projection. It goes right
after the section "Projection and Keys". For an explanation of why
I chose this approach, please see my origninal email,
below.
I don't really mean to be overwhelming everyone with
proposals. This is probably the last major change we think really needs
to be in SDO 3. I would like to resolve all the issues with
proposals and at that point we could produce a CD that would describe the all
major features of SDO 3. There is still a lot to discuss, especially
closing loopholes that are standing in the way of interoperability and
portability...but we would at least have a document to which we could point
people when they ask what SDO 3 is. And we would have a document that
people could start implementing, testing, etc.
Best Regards,
Ron
4.14.4 ChangeSummary and Projection
The scope of
a ChangeSummary is defined by the containment structures and orphanHolders of
the projection in which ChangeSummary.beginLogging() was called. That is, the
scope is determined by the HelperContext of the DataObject from which the
ChangeSummary was retrieved (either through getChangeSummary() or as the value
of a property having ChangeSummaryType). At most one projection at a time may
be actively logging: calling ChangeSummary.beginLogging in a second projection
results in undefined behavior. After change logging has been terminated with a
call to ChangeSummary.endLogging(), a further call to
ChangeSummary.beginLogging() in any projection MUST reset the change state of
all objects in the scope as defined by either projection, the list of changed
objects MUST be cleared and the old values of the first logging session MUST
no longer be visible.
Changes MUST tracked for all changes in scope
of an active change summary, regardless of the projection in which the changes
are made.
The ChangeSummary, and its properties, follow
the normal rules associated with the projection of DataObjects and property
values. The ChangeSummary’s contents
MUST be visible in all projections, not only in the projection from which
beginLogging was called. The list returned by
ChangeSummary.getChangedObjects() MUST contain the same underlying business
objects, regardless of whether the ChangeSummary is being inspected in the
projection in which beginLogging was called, or another projection. The
elements in the list MUST be consistent (i.e., projected into) the context of
the DataObject from which the ChangeSummary was
retrieved.
The following example, based on the metadata
from the previous section’s example,
illustrates this behavior.
// Create an envelope DataObject containing a change summary.
DataObject dataGraph =
_helperContext.getDataFactory().create(SDO_URI,"DataGraphType");
// Create the SDO graph root node
DataObject cal =
_helperContext.getDataFactory().create(School.class);
// Add it to the graph
dataGraph.add("school",cal);
// Fill the data graph
cal.set("name","Berkeley");
DataObject billy =
cal.createDataObject("students");
billy.set("name", "Billy
Bear");
DataObject bob =
cal.createDataObject("students");
bob.set("name", "Bobbie
Bear");
DataObject basketWeaving =
cal.createDataObject("courses");
basketWeaving.set("name", "Basket
Weaving");
DataObject algol =
cal.createDataObject("courses");
algol.set("name", "Algol");
DataObject
revolution = cal.createDataObject("courses");
revolution.set("name",
"Revolution");
// hook things
up
billy.getList("courses").add(basketWeaving);
billy.getList("courses").add(algol);
bob.getList("courses").add(basketWeaving);
bob.getList("courses").add(revolution);
// Create a second context defined by an XSD
HelperContext hc2 =
SDO.getHelperContextFactory().createHelperContext();
hc2.getXSDHelper().define(getClass().getClassLoader().getResourceAsStream("com/sap/sdo/testcase/internal/pojo/ex/projection.xsd"),
null);
// Project from the java context to the XSD context
DataObject
projection = hc2.getDataFactory().project(cal);
// Turn on change logging in the projected DataObject
projection.getChangeSummary().beginLogging();
// Make a few changes in the logging context
projection.set("name","Stanford");
DataObject billy2 = (DataObject)projection.getList("students").get(0);
billy2.set("name","Chauncy Cardinal");
// Make the
original context active by projecting back to it
cal =
_helperContext.getDataFactory().project(projection);
// Inspect
the change summary
ChangeSummary changeSummary = cal.getChangeSummary();
assert.equals(2,
changeSummary.getChangedObjects().size());
assert.true(changeSummary.getChangedObjects().contains(cal));
assert.true(changeSummary.getChangedObjects().contains(billy));
Hi Everyone,
As a second component regarding ChangeSummary and
non-containment graphs, I'd like to address how I think projection can be used
in this regard. Note that this is not a proposal, it's just some
thoughts about what direction to go…there are some details to be worked
out. Although as always I consider backwards compatibility to be
essential, there's actually not much of a problem here. Since the
"project()" operation did not exist in 2.1, no existing code is using it, so
whatever "effect" we define for projection has on change summary, we will not
break any existing code. So backwards compatibility in this case boils
down to saying that within a single HelperContext, including XML marshalling
and unmarshalling, there must be no changes to change summary
behavior.
First, some background regarding projection.
Containmainment is a central concept in SDO, corresponding to UML
aggregation. However, in practice, containment is often used to control
the behavior of SDO functionality rather than the inherent structure of the
model. Two instances where containment is (mis-)used in this way are (a)
containmaint is used to define the structure of the XML document to which the
SDO should be serialized and (b) containment is used to define the "scope" of
a change summary. Projection allows an alternate containment structure
to be imposed on an existing model, with the so that the model seen and
manipulated by users is not necessarily determined by such
factors.
In particular, I believe the most important use-case for
ChangeSummary is that ChangeSummaries are consumed by a DAS backend that uses
the list of changed objects (and potentially the old values associated with
them) to optimize database access. If we imagine a DAS that is based on JPA,
then without the CS, the DAS would have to pull the entire graph into the
persistence context, effectively calling EntityManager.merge on every object
or maybe calling EntityManager.persist when the object is not already in the
store. With the help of a ChangeSummary, an update operation
becomes:
public void update(POShoppingCart cart) {
DataObject cartObj = (DataObject)cart;
ChangeSummary cs = cartObj.getChangeSummary();
if (cs!=null) {
List<DataObject> changes =
cs.getChangedDataObjects();
for (DataObject o: changes) {
if (cs.isModified(o)) {
em.merge(o);
} else if
(cs.isDeleted(o)) {
em.remove(o);
} else if
(cs.isCreated(o)) {
em.persist(o);
}
}
}
}
The reason why projection comes into the picture is that a
DAS that sits over an RDB will probably want its model to reflect the RDB, and
for models based on RDBs (or on JPA persistent objects) will not have much
containment. So the types seen by the DAS should not necessarly have
containment properties. But without containment properties,
getChangedDataObjects is a useless operation. The goal of this proposal
is to provide some meaningful and useful behavior for getChangedDataObjects()
in such use-cases.
As in my consideration of dealing with OrphanHolders, I will
define the ChangeSummary behavior in terms of diffs on data graphs, ie,
comparisons of their before and after states. This does not imply a
particular implementation strategy: implementations may also work by some kind
of change logging or by marking the "in-scope" DataObjects.
It is important to note that the containment structure has
two effects on the change summary. Besides defining the scope (the set
of DataObjects whose before and after states are considered, the containment
structure also determines whether data objects are considered to be created
(new in the containment structure) or deleted (removed from the containment
structure). The same set of operations can produce creates and deletes
in one projections, and only modify's in another
projection.
SDO 2.1 has the restriction that an object may be "in-scope"
of at most one change summary. I would like to loosen this a little in
regard to projection. I would say that an object may be in scope of at
most one change summary per projection, and that at most one of the
projections may have logging enabled. To say it another way, not matter
how many projections of an object exist, the SDO implementation mantains at
most one "before-image" of any object.
As long as we are in the projection from which beginLogging
was called, everything remains the same as in 2.1. The question is, what
happens when the change summary is queried from another helper context.
The direction I would suggest going is that the change summary (in particular,
the list returned by getChangedObjects) be calculated diffing the current and
the old-state in the projection in which
beginLogging was called. Of course, the
DataObjects themselves should be projected into the querying
HelperContext. In a way, this makes the behavior of CS consistent with
the behavior normally associated with projection. The CS is itself a
type of DataObject, it has properties (changedObjects in this case can be
viewed as a multivalued property) and those properties have values that do not
change, regardless of the projection from which they are accessed. Using
the projection in which "beginLogging" was called as the basis for calculating
the delta assures that the "before-image" and the "after-image" can be
meaningfully compared.
The situation becomes significantly more complex if there is
an XML serialization in the picture. Since we want to maintain backwards
compatibility I do not think we can change the change summary serialization/
deserialization algorithm. This means that the if the result of an
XMLHelper.load is a DataObject with logging enabled, then the "before image"
is based on the XML document, that is, the HelperContext used to calculate the
change summary is the HelperContext that owns the XMLHelper.
Symettrically, I believe we also need a restriction that the XMLHelper.save be
called from the same context from which "beginLogging" was called. The
bottom line here is that the structure used for XML serialization must be
consistent with the structure used to calculate the change summary
scope. I would say calling XMLHelper.save in a different HelperContext
is a user-error that implementations MAY handle, must this would be
non-standard behavior.
To me, this is a pretty unfortunate limitation, but one I
can live with at least for the SDO 3.0 timeframe. If the group decides
this is too restrictive, then it may be possible to expand the change summary
serialization in some way, as a topic for further
research.
Best Regards,
Ron