|
Hi Everyone,
In this weeks call, we discussed 2 possible
compromises:
1. Projecting a ChangeSummary has the effect of
calling endLogging(). This means, changes are only track if they are made
in the context where beginLogging was called. The ChangeSummary, and in
particular getChangedObjects, is however available in the target
context.
2. We define behavior only in the case where the
two scopes "match". Match here means that the set of "in-scope" objects is
the same in both contexts. The scopes may be differently organised, that
is, in one context the scope could be defined using containment, in the other,
the scope could be defined using orphanHolder properties. The behaviour
when the scopes do not match will be left undefined, but the wording would
suggest that it is at least legal for an implementation to have a stragegy like
our original proposal.
The hope is that (1) would be less problematic to
implement. The mechanism through which changes are tracked would remain
unchanged from 2.1.1, that is, only the active projection must be
considered. The projected ChangeSummary is a "read-only" object, which is
perhaps not so problematic. On the other hand, it does limit some
use-cases. For instance, in the original proposal we could use a
projection to define a change summary scope
Option (2) is simply the "weasel solution": I
don't really like intentially leaving holes in the spec.
Please consider which approach is more acceptable, and
I will update the proposal accordingly.
Best Regards,
Ron
Hi Blaise,
During last week's call we had the idea that the proposal
would be much more acceptable if we had some requirement that the scopes of the
two projections must somehow match. The idea was that we could perhaps
make some clever use of OrphanHolder properties to achieve
this.
The idea works well for what I guess would be the most
common case, where the entire graph is in scope, eg, when a DataGraphType is
used as an envelope. In the more general case, where only a sub-graph is
supposed to be in scope, I simply can't make it work. Bottom line,
there is no way to assure that the scopes match. Containment is a
relationship between object instances. OrphanHolder properties make their
decisions based on metadata alone. I can have a tree with only one type of
node (a typical tree with parent-child relationships). On one branch of
the tree I have a change summary. Only this half of the tree is
in-scope. There's no way to represent this using OrphanHolders. It
also seems to imply either the use of orphanHolder properties in contexts where
they would not otherwise be required, or allowing "Compatible Types" to differ
in that one has orphanHolders while the other doesn't.
It seems to me that there are a few
alternatives:
1. Allow the original and projected scopes to
differ. The scope of the change summary in the context where beginLogging
was called is the one that counts. This is the original
proposal.
2. Require the scope's to match, knowing that this
imposes limitations on when project can be called. In cases where a
DataGraph is being projected, this approach yields the same result as the
first.
3. Allow the scopes to differ, but only changes that
are in both scopes are projected.
Best Regards,
Ron
Hi Ron,
In general I have some reservations about the
complexity of implementing what I believe is being proposed. To determine
changes in a change summary we will need to be aware of the helper context the
begin logging operation was called in and (if projections is involved) use a
different but "compatible" set of types to determine the scope. Below are
some additional comments to see if I correctly understand what is being
proposed.
beginLogging()
In SDO 2.1.1 beginLogging() resets the
change summary, I believe there is even a TCK test case for this.
Use
Case #1 - Project from no Containment to Full Containment
In this
scenario data is brought in from a database (no containment) and projected into
a helper context with metadata defined from an XML schema (full
containment). If beginLogging() is called after the data object has been
projected into the full containment helper context then:
- The behaviour of the change summary (in the full containment helper
context) remains the same.
- The behaviour of the change summary (in the no containment helper context)
does not track changes according to its own metadata, but instead the metadata
from the full containment helper context.
- Since getChangedObjects() returns the same list in both HelperContexts, we
are now giving a change summary knowledge beyond its scope (at least the
change summary in the no containment helper context), by using reset it can
even affect objects beyond its scope.
- Presumably the
getOldContainer()/getOldContainmentProperty()/getOldSequence() calls remain
specific to the helper context (i.e. the change summary from the no
containment HelperContext would always return null for getOldContainer since
there was no defined containment relationships)?
Use Case #2 -
Project from Full Containment to no Containment
In this scenario data is
read from an XML document in a helper context with metadata defined from an XML
schema (full containment), and projected into a helper context with no
containment (metadata derived from a database). If beginLogging is called
after the data object has been projected into the no containment helper
context.
- The behaviour of the change summary (in the no containment helper context)
remains the same
- The behaviour of the change summary (in the full containment helper
context) does not track changes according to its own metadata, but instead the
metadata from the no containment helper context.
Other
Consequences on Spec
Sections talking about ChangeSummary will need to be
updated to reflect that the rules are based on the HelperContext on which
beginLogging was called.
-Blaise
Barack, Ron wrote:
7C3EF93EEBC6EB4A8B4470853DE86566EC5360@dewdfe18.wdf.sap.corp
type="cite">
Hi Blaise,
Thanks a lot for your comments. My attempts at
answers inline...
Ron
Hi Ron,
Are we introducing a requirement that
the change summary scope of the data object in the original helper context
match the change summary scope in the helper context the data object is
projected into?
no, the scope of the original
(beginLogging) projection does not have to match the scope of
the projection in which getChangeSummary() was called. The
whole idea is that beginLogging can be called in a context where a
containment structure has been defined (eg, the metadata comes from an
XSD) but the change summary is inspected in a context where
there are very few containment relationships (as will be
typical if the metadata comes from JPA). The scope as
calculated from the JPA model will probably contain only the root
node. The scope as calculated from the XSD model will be more
reasonable. So the scopes are not
identical.
I think that there is a restriction when
serializing to XML. You can only serialize the CS to XML if the
scope as defined by the context where beginLogging() was called
matches the projection where XMLHelper.save is called.
This is easiest to see if
you imagine the simple case where beginLogging is called, and then the XML
shipped over the wire to a client. The client will see the beginLogging
flag is on, and will naturally assume that beginLogging was called in the
current projection (ie, in the context that calls
XMLHelper.load()). We'd have to adjust our XML format if we want
to specify which projection should be used. This is if course possible,
but I want to get this simple case defined first.
Things get a little messy if
there is projection over keys involved. The context where
getChangeSummary().getChangedObjects() is called should be at least as
detailed as the context where beginLogging() was called. Imagine 2
contexts, both of which have an "AddressType", but in one context
AddressType has a relatationship to the type
"CountryType" while in the other context AddressType has a string property
holding the key value of the referenced CountryType. If I call
beginLogging in the context with "CountryType", modify a property of a
Country, and then call getChangeSummary.getChangedObjects in the context where
"CountryType" is not defined, then we won't be able to project the element in
this list into the appropriate context. To me, this is an untroubling
user-error, a corner-case. The use-cases I'm imagining are that the
ChangeSummary is being consumed by the DAS, and the DAS is the one with the
complexer type system. Things are trimmed (through introducing keys)
when transporting to clients.
Some
comments on your proposal:
4.14.4 ChangeSummary and Projection
The scope of a ChangeSummary is defined by the containment structures
and orphanHolders of the projection in which ChangeSummary.beginLogging() was
called.
What does this sentence really mean?
Does it mean that regardless where the data graph is projected into it still
obeys the change summary rules of the original helper context?
It means the scope is defined by the original context. If
you imagine an implementation that marks objects as being "in-scope" by
setting a flag, it means that beginLogging will run along marking
objects according to the containment structures and orphan holders
of the context in which it was called. And it means that the change
summary must somehow remember the context in which beginLogging
was called, so that this same context can be used when endLogging/
getChangedObjects is called.
That is, the scope is determined
by the HelperContext of the DataObject from which the ChangeSummary was
retrieved (either through getChangeSummary() or as the value of a property
having ChangeSummaryType). At most one projection at a time may be actively
logging: calling ChangeSummary.beginLogging in a second projection results in
undefined behavior. After change logging has been terminated with a call
to ChangeSummary.endLogging(), a further call to ChangeSummary.beginLogging()
in any projection MUST reset the change state of all objects in the scope as
defined by either projection, the list of changed objects MUST be cleared and
the old values of the first logging session MUST no longer be
visible.
Assuming the "second projection" is
the active projection, why wouldn't the beginLogging call just reset the
change summary?
In
2.1.1, calling beginLogging() twice (without calling endlogging) is
a no-op, so I would say that this is the behavior we want when beginLogging is
called twice in the same context. Maybe we should
just extend this, and say calling beginLogging twice (regardless of context)
is a no-op. On the other hand, this seems like a user error to me,
intuitively, I want to throw an exception here. So now we have 3
potential behaviors, and I can see reasons for each of them. Being
undecisive, I took the cowards way out and left it
undefined.
Changes MUST tracked for all changes in scope
of an active change summary, regardless of the projection in which the changes
are made.
The ChangeSummary, and its properties, follow
the normal rules associated with the projection of DataObjects and property
values. The ChangeSummary’s contents
MUST be visible in all projections, not only in the projection from which
beginLogging was called. The list returned by
ChangeSummary.getChangedObjects() MUST contain the same underlying business
objects, regardless of whether the ChangeSummary is being inspected in the
projection in which beginLogging was called, or another projection. The
elements in the list MUST be consistent (i.e., projected into) the context of
the DataObject from which the ChangeSummary was
retrieved.
The following example, based on the metadata
from the previous section’s example,
illustrates this behavior.
// Create an envelope DataObject containing a change summary.
DataObject dataGraph =
_helperContext.getDataFactory().create(SDO_URI,"DataGraphType");
// Create the SDO graph root node
DataObject cal =
_helperContext.getDataFactory().create(School.class);
// Add it to the graph
dataGraph.add("school",cal);
// Fill the data graph
cal.set("name","Berkeley");
DataObject billy =
cal.createDataObject("students");
billy.set("name", "Billy
Bear");
DataObject bob =
cal.createDataObject("students");
bob.set("name", "Bobbie
Bear");
DataObject basketWeaving =
cal.createDataObject("courses");
basketWeaving.set("name", "Basket
Weaving");
DataObject algol =
cal.createDataObject("courses");
algol.set("name", "Algol");
DataObject
revolution = cal.createDataObject("courses");
revolution.set("name",
"Revolution");
// hook things
up
billy.getList("courses").add(basketWeaving);
billy.getList("courses").add(algol);
bob.getList("courses").add(basketWeaving);
bob.getList("courses").add(revolution);
// Create a second context defined by an XSD
HelperContext hc2 =
SDO.getHelperContextFactory().createHelperContext();
hc2.getXSDHelper().define(getClass().getClassLoader().getResourceAsStream("com/sap/sdo/testcase/internal/pojo/ex/projection.xsd"),
null);
// Project from the java context to the XSD context
DataObject
projection = hc2.getDataFactory().project(cal);
// Turn on change logging in the projected DataObject
projection.getChangeSummary().beginLogging();
// Make a few changes in the logging context
projection.set("name","Stanford");
DataObject billy2 = (DataObject)projection.getList("students").get(0);
billy2.set("name","Chauncy Cardinal");
// Make the
original context active by projecting back to it
cal =
_helperContext.getDataFactory().project(projection);
// Inspect
the change summary
ChangeSummary changeSummary = cal.getChangeSummary();
assert.equals(2,
changeSummary.getChangedObjects().size());
assert.true(changeSummary.getChangedObjects().contains(cal));
assert.true(changeSummary.getChangedObjects().contains(billy)); -Blaise
Barack, Ron wrote:
7C3EF93EEBC6EB4A8B4470853DE86566E7C8E8@dewdfe18.wdf.sap.corp
type="cite">
Hi Everyone,
Here is a draft resolution for this issue. It is
a new sub-section of the section describing projection. It goes right
after the section "Projection and Keys". For an explanation of
why I chose this approach, please see my origninal email,
below.
I don't really mean to be overwhelming everyone with
proposals. This is probably the last major change we think really
needs to be in SDO 3. I would like to resolve all the
issues with proposals and at that point we could produce a CD that would
describe the all major features of SDO 3. There is still a lot to
discuss, especially closing loopholes that are standing in the way of
interoperability and portability...but we would at least have a document to
which we could point people when they ask what SDO 3 is. And we would
have a document that people could start implementing, testing,
etc.
Best Regards,
Ron
4.14.4 ChangeSummary and
Projection
The scope of a ChangeSummary is defined by the
containment structures and orphanHolders of the projection in which
ChangeSummary.beginLogging() was called. That is, the scope is determined by
the HelperContext of the DataObject from which the ChangeSummary was
retrieved (either through getChangeSummary() or as the value of a property
having ChangeSummaryType). At most one projection at a time may be actively
logging: calling ChangeSummary.beginLogging in a second projection results
in undefined behavior. After change logging has been terminated with a call
to ChangeSummary.endLogging(), a further call to
ChangeSummary.beginLogging() in any projection MUST reset the change state
of all objects in the scope as defined by either projection, the list of
changed objects MUST be cleared and the old values of the first logging
session MUST no longer be visible.
Changes MUST tracked for all changes in scope
of an active change summary, regardless of the projection in which the
changes are made.
The ChangeSummary, and its properties, follow
the normal rules associated with the projection of DataObjects and property
values. The ChangeSummary’s contents
MUST be visible in all projections, not only in the projection from which
beginLogging was called. The list returned by
ChangeSummary.getChangedObjects() MUST contain the same underlying business
objects, regardless of whether the ChangeSummary is being inspected in the
projection in which beginLogging was called, or another projection. The
elements in the list MUST be consistent (i.e., projected into) the context
of the DataObject from which the ChangeSummary was
retrieved.
The following example, based on the metadata
from the previous section’s example,
illustrates this behavior.
// Create an envelope DataObject containing a change summary.
DataObject dataGraph =
_helperContext.getDataFactory().create(SDO_URI,"DataGraphType");
// Create the SDO graph root node
DataObject cal =
_helperContext.getDataFactory().create(School.class);
// Add it to the graph
dataGraph.add("school",cal);
// Fill the data graph
cal.set("name","Berkeley");
DataObject billy
= cal.createDataObject("students");
billy.set("name", "Billy
Bear");
DataObject bob =
cal.createDataObject("students");
bob.set("name", "Bobbie
Bear");
DataObject basketWeaving =
cal.createDataObject("courses");
basketWeaving.set("name", "Basket
Weaving");
DataObject algol =
cal.createDataObject("courses");
algol.set("name",
"Algol");
DataObject revolution =
cal.createDataObject("courses");
revolution.set("name",
"Revolution");
// hook things
up
billy.getList("courses").add(basketWeaving);
billy.getList("courses").add(algol);
bob.getList("courses").add(basketWeaving);
bob.getList("courses").add(revolution);
// Create a second context defined by an XSD
HelperContext hc2 =
SDO.getHelperContextFactory().createHelperContext();
hc2.getXSDHelper().define(getClass().getClassLoader().getResourceAsStream("com/sap/sdo/testcase/internal/pojo/ex/projection.xsd"),
null);
// Project from the java context to the XSD context
DataObject
projection = hc2.getDataFactory().project(cal);
// Turn on change logging in the projected DataObject
projection.getChangeSummary().beginLogging();
// Make a few changes in the logging context
projection.set("name","Stanford");
DataObject billy2 = (DataObject)projection.getList("students").get(0);
billy2.set("name","Chauncy Cardinal");
// Make
the original context active by projecting back to it
cal =
_helperContext.getDataFactory().project(projection);
//
Inspect the change summary
ChangeSummary changeSummary =
cal.getChangeSummary();
assert.equals(2,
changeSummary.getChangedObjects().size());
assert.true(changeSummary.getChangedObjects().contains(cal));
assert.true(changeSummary.getChangedObjects().contains(billy));
Hi Everyone,
As a second component regarding ChangeSummary and
non-containment graphs, I'd like to address how I think projection can be
used in this regard. Note that this is not a proposal, it's just some
thoughts about what direction to go…there are some details to be worked
out. Although as always I consider backwards compatibility to be
essential, there's actually not much of a problem here. Since the
"project()" operation did not exist in 2.1, no existing code is using it, so
whatever "effect" we define for projection has on change summary, we will
not break any existing code. So backwards compatibility in this case
boils down to saying that within a single HelperContext, including XML
marshalling and unmarshalling, there must be no changes to change summary
behavior.
First, some background regarding projection.
Containmainment is a central concept in SDO, corresponding to UML
aggregation. However, in practice, containment is often used to
control the behavior of SDO functionality rather than the inherent structure
of the model. Two instances where containment is (mis-)used in this
way are (a) containmaint is used to define the structure of the XML document
to which the SDO should be serialized and (b) containment is used to define
the "scope" of a change summary. Projection allows an alternate
containment structure to be imposed on an existing model, with the so that
the model seen and manipulated by users is not necessarily determined by
such factors.
In particular, I believe the most important use-case for
ChangeSummary is that ChangeSummaries are consumed by a DAS backend that
uses the list of changed objects (and potentially the old values associated
with them) to optimize database access. If we imagine a DAS that is based on
JPA, then without the CS, the DAS would have to pull the entire graph into
the persistence context, effectively calling EntityManager.merge on every
object or maybe calling EntityManager.persist when the object is not already
in the store. With the help of a ChangeSummary, an update operation
becomes:
public void update(POShoppingCart cart)
{
DataObject cartObj = (DataObject)cart;
ChangeSummary cs = cartObj.getChangeSummary();
if (cs!=null)
{
List<DataObject> changes =
cs.getChangedDataObjects();
for (DataObject o: changes) {
if (cs.isModified(o)) {
em.merge(o);
} else if
(cs.isDeleted(o)) {
em.remove(o);
} else if
(cs.isCreated(o)) {
em.persist(o);
}
}
}
}
The reason why projection comes into the picture is that a
DAS that sits over an RDB will probably want its model to reflect the RDB,
and for models based on RDBs (or on JPA persistent objects) will not have
much containment. So the types seen by the DAS should not necessarly
have containment properties. But without containment properties,
getChangedDataObjects is a useless operation. The goal of this
proposal is to provide some meaningful and useful behavior for
getChangedDataObjects() in such use-cases.
As in my consideration of dealing with OrphanHolders, I
will define the ChangeSummary behavior in terms of diffs on data graphs, ie,
comparisons of their before and after states. This does not imply a
particular implementation strategy: implementations may also work by some
kind of change logging or by marking the "in-scope" DataObjects.
It is important to note that the containment structure has
two effects on the change summary. Besides defining the scope (the set
of DataObjects whose before and after states are considered, the containment
structure also determines whether data objects are considered to be created
(new in the containment structure) or deleted (removed from the containment
structure). The same set of operations can produce creates and deletes
in one projections, and only modify's in another
projection.
SDO 2.1 has the restriction that an object may be
"in-scope" of at most one change summary. I would like to loosen this
a little in regard to projection. I would say that an object may be in
scope of at most one change summary per projection, and that at most one of
the projections may have logging enabled. To say it another way, not
matter how many projections of an object exist, the SDO implementation
mantains at most one "before-image" of any object.
As long as we are in the projection from which
beginLogging was called, everything remains the same as in 2.1. The
question is, what happens when the change summary is queried from another
helper context. The direction I would suggest going is that the change
summary (in particular, the list returned by getChangedObjects) be
calculated diffing the current and the old-state in the projection in which beginLogging was called. Of course, the DataObjects themselves should be
projected into the querying HelperContext. In a way, this makes the
behavior of CS consistent with the behavior normally associated with
projection. The CS is itself a type of DataObject, it has properties
(changedObjects in this case can be viewed as a multivalued property) and
those properties have values that do not change, regardless of the
projection from which they are accessed. Using the projection in which
"beginLogging" was called as the basis for calculating the delta assures
that the "before-image" and the "after-image" can be meaningfully
compared.
The situation becomes significantly more complex if there
is an XML serialization in the picture. Since we want to maintain
backwards compatibility I do not think we can change the change summary
serialization/ deserialization algorithm. This means that the if the
result of an XMLHelper.load is a DataObject with logging enabled, then the
"before image" is based on the XML document, that is, the HelperContext used
to calculate the change summary is the HelperContext that owns the
XMLHelper. Symettrically, I believe we also need a restriction
that the XMLHelper.save be called from the same context from which
"beginLogging" was called. The bottom line here is that the structure
used for XML serialization must be consistent with the structure used to
calculate the change summary scope. I would say calling XMLHelper.save
in a different HelperContext is a user-error that implementations MAY
handle, must this would be non-standard behavior.
To me, this is a pretty unfortunate limitation, but one I
can live with at least for the SDO 3.0 timeframe. If the group decides
this is too restrictive, then it may be possible to expand the change
summary serialization in some way, as a topic for further
research.
Best Regards,
Ron
|