On Jul 9, 2012, at 2:32 PM, Drummond Reed wrote:
Mike,
I am a huge supporter of the XDI protocol following Eintein's dictum of "As simple as possible but no simpler".
So the main question I have, when reading your summary from Yuriy below -- which matches exactly the discussion I've seen Mark Horstmeier at Kynetx having with Markus and Animesh Chowdhury at Neustar -- is that:
- if an XDI client is allowed to send multiple operations to a server
- AND those operations are unordered
- AND the server commits all operations until it hits one that produces and error and then stops and reports the error
- THEN it is completely unpredictable what operations will be committed or not committed in the message.
Now, I'm not a protocol expert, or a transactional integrity expert, but it seems to me that there is a fundamentally flaw in this approach and we as a TC would be negligent in putting out a protocol governed by those rules.
It seems there are three basic ways to fix this flaw and still "keep it simple":
- Allow only one operation per message (as Bill has suggested several times in the past). This eliminates any need for any transactional integrity at all.
** Bill: Yes. Multiple operations per message implies a transaction, so not doing transactional integrity then for me violates the principle of least surprise. As always, +1 for one op per message on my part. I also would like to point out that you
can send each message over the same connection without reopening a connection, whether that connection is HTTP, Web Sockets, or XMPP.
This is what I assume we'd have with the current proposal for delivering a single message/response over HTTP.
- Allow multiple unordered operations in a message but specify that either all the operations are committed or none are.
This sounds like transaction grouping and group atomicity as are available in RDBMSs, except for being unordered. It seems to me that different orders would often produce different results, as you say above, so unordering should not be allowed.
** Bill: Agreed on lack of ordering being an issue with this option. It also conveys transactions by grouping multiple options together and using transactional integrity language such as 'all operations are committed or none are'.
- Allow multiple operations in a message but require the operations to be ordered and specify that all successful operations up to the one that causes an error are committed and none of the operations from from the one that causes error forward are committed.
This is what I was assuming when proposing we think about XDI over a persistent connection. It sounds to me much easier to implement than transactions that group multiple operations into a unit.
**Bill: This is doable, but your error handling is going to be complicated, for both XDI endpoints and XDI messengers (I prefer endpoints and messengers over server and client terminology). Also the message looks like a transaction, which may be confusing
to implementers.
Even with single message per HTTP request, what happens if the requests arrive in a different order than the order sent? This would seem to introduce uncertainty. A persistent connection actually does not have this problem since earlier requests are guaranteed
to be earlier in the stream. Perhaps we should *require* opening a persistent connection whenever there's danger of reordering giving unpredictable results.
**Bill: Good question. For a $add and $get I do not believe there is a danger of unpredictable results, unless we are requiring that the implicit contexts containing subjects and objects in the statements added. For $delete, the
danger is present.
As a result I propose the following $ words to allow a single mutex per XDI context for state changing operations:
$lockholder - A predicate indicating that the subject, which is a context, is locked by the object, which is an XDI entity (individual or organization) with the given i-name.
$lock - A $op that adds two statements to the graph. For the first statement the subject is the target context, the predicate is $lockholder, and the object is the i-name of the lock holder. For the second statement the subject
is the first statement in an XDI cross-reference, the predicate is $expires, and the object is a date-time literal.
$unlock - A $op that removes the lock statements.
Business rules:
1. An XDI compliant endpoint MUST execute a $delete operation to remove both of these statements within 3 minutes (arbitrary, can change) of the expiration value of a lock.
2. An XDI compliant endpoint MUST NOT remove a lock unless it receives a $unlock from the lock holder, or an agent thereof.
3. An XDI compliant endpoint MUST NOT allow any context-altering (other than versioning information) operation to be executed if a lock is in place, unless the message is from the lock holder. 4. An XDI compliant operation MAY queue operations that
are not executed, which are received when a context is locked, and then execute them when the lock is released or expires.
This solves some of the problem, but maybe not all of it. A single user can still send messages that cause an unpredictable behavior I think - I'd need to go over it in more depth to have more confidence either way.
Thoughts?
=Drummond
On Mon, Jul 9, 2012 at 9:11 AM, Michael Schwartz
<mike@gluu.org> wrote:
Drummond,
I don't think we should implement transactions in XDI 1.0. LDAP has lived without transactions for 20 years and not been too much the worse for it. Transactional persistence is the domain on SQL databases right now. Also, nothing would stop an XDI server vendor
from implementing transactions, and once the data exists that define the best practices, we define standards at a later time. I just don't think its on the critical must-have features.
Note: I do think that XDI servers should be transactional with regard to single operations!
I thought Yuriy Zabrovarnyy might have some thoughts on this... here they are:
Untill now we agreed on the rules that are described here:
-
https://wiki.oasis-open.org/xdi/XdiMessagePatterns
-
https://wiki.oasis-open.org/xdi/MessagingErrorCode
The rules are:
a) all operations are independent even if they come in one message.
b) if one operation is failed then further operation execution is
stopped and error is send with actual problem and
statement where problem occurs. More info on the OX wiki
http://ox.gluu.org/doku.php?id=docs:graphoperations
Example: Lets say we have three operations. Two of them are successfully executed and the last one fails. The error is sent to the client but the two successfully executed operation are NOT ROLLED BACK. c) operations are executed in order they appears in the
message request (accept $add operation where sorting is made on server to provide correct execution + variable support)
3. CURRENT oxServer IMPLEMENTATION
Some time ago i've already investigated question about transaction support with LDAP. And problem is that OpenDJ does not provide such feature. There are only one more or less good implementation provided by UnboundID, see
http://www.dirmgr.com/blog/2009/4/6/transactions-in-the-unboundid-directory-server.html.
- Mike
-------------------------------------
Michael Schwartz
Gluu
Founder / CEO
office: +1 646-810-8761
mike@gluu.org
On Mon, 9 Jul 2012, Drummond Reed wrote:
Joseph, my intuition is that your intuition is right ;-)
Seriously, it keeps coming up that we need enough transaction integrity
semantics in the XDI protocol so that clients can unambiguously ask a
server to do what the client wants, and servers can unambiguously know how
to tell a client what messages/operations worked and what didn't. In other
words, accurate, unambiguous communication of state.
Since in XDI every operation request in every message is a state change
(even a $get is a state change if it is logged), then maybe we just need to
bite the bullet and apply the multiplicity patterns to XDI messaging. The
basic rules would be:
1. A single message with a single operation would not need
any transaction integrity semantics - the message either succeeds or fails,
and the message ID already identifies the message. However the server may
need to return the message ID in the response.
2. A single message with a collection of operations would need to order
the operations using ordinal statements. The server performs the operations
in order. If an operation fails, the whole message fails, and the server
sends back the full XDI statement of the operation that failed (because
that XDI statement is already unique).
3. A collection of messages (whether with single operations or
collections of operations) would need to:
1. Order the messages so the server can perform them in order.
2. Include basic commit/rollback semantics for message collections.
If you agree, then we should put together a formal proposal for this so we
can start discussing it.
=Drummond
On Sat, Jul 7, 2012 at 4:50 PM, Joseph Boyle <
planetwork@josephboyle.net>
wrote:
On Jul 6, 2012, at 6:53 AM, Barnhill, William [USA] wrote:
I propose that rather than deciding which methods to explicitly support,
we instead create a generic authentication mechanism, i.e. something like
SASL. If XDI messaging were a connection-oriented protocol then I would
suggest an XDI binding of the SASL abstraction layer. However, to my
knowledge XDI messaging has always been, and is intended to remain, a
connectionless protocol (or more precisely a message-oriented protocol).
I strongly feel that it will turn out to be advantageous to maintain
connections, and that we should anticipate this. I realize we want to get
the simplest case (no connection, minimal HTTP methods/codes, minimal
authentication) out the door quickly, but can we give this a little thought
now? It may be as simple as numbering messages and responses so that the
client knows which response was to which message, or only sending responses
in order received. Is it clear where a message ends - do we have a clear
end of message marker?
---------------------------------------------------------------------
To unsubscribe, e-mail:
xdi-unsubscribe@lists.oasis-open.org
For additional commands, e-mail:
xdi-help@lists.oasis-open.org