[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [xdi] Transactional integrity & the XDI protocol (was Re: [xdi] [External] Connection)
Whoever proposes this needs to give use cases, because I'm having a hard time seeing why this is a must have, versus a nice to have. As I said, we we are having no such problems with the XDI programming we've undertaken.
- Mike ------------------------------------- Michael Schwartz Gluu Founder / CEO office: +1 646-810-8761 mike@gluu.org On Mon, 9 Jul 2012, Drummond Reed wrote:
I think Bill was suggesting that sending multiple $get operations may not be a problem. So doing multiple searches in one message makes sense. It's other operations that explicitly change the state of the graph that are problematic. Since I can see reams of examples of where an XDI messenger would need to send an XDI endpoint multiple $add operations (given that every XDI statement is a different $add operation), it doesn't seem feasible to tell XDI developers that they can only send one $add, $mod, or $del operation per message. The protocol would just end being woefully inefficient compared to simply ordering operations. So some basic transactional integrity semantics along the lines of Bill's suggestion seems to be needed. =Drummond On Mon, Jul 9, 2012 at 4:31 PM, Michael Schwartz <mike@gluu.org> wrote:TC: Simply sending multiple operations implies transactions? If I am submitting multiple opearations, I am going to do some research on the server to see how its handled... I am not going to assume that they will be handled as an transaction, especially if I'm not even using the transaction reserved words you suggested earlier. And frankly, you are thinking of the wrong use case here. You should be thinking about getting all your search requests in one query, which is vastly more efficient than sending a bunch of individual messages. Per my previous comment, transactions are at the single operation level, I don't see any ambiguity for the developer. - Mike ------------------------------**------- Michael Schwartz Gluu Founder / CEO office: +1 646-810-8761 mike@gluu.org On Mon, 9 Jul 2012, Barnhill, William [USA] wrote: Drummond,Thank you for a very well-thought out and complete summary. Comments inline -Bill ______________________________**__ From: xdi@lists.oasis-open.org [xdi@lists.oasis-open.org] on behalf of Joseph Boyle [planetwork@josephboyle.net] Sent: Monday, July 09, 2012 5:54 PM To: Drummond Reed Cc: Michael Schwartz; OASIS - XDI TC; Mark Horstmeier; Chowdhury, Animesh; Yuriy Zabrovarnyy Subject: Re: [xdi] Transactional integrity & the XDI protocol (was Re: [xdi] [External] Connection) On Jul 9, 2012, at 2:32 PM, Drummond Reed wrote: Mike, I am a huge supporter of the XDI protocol following Eintein's dictum of "As simple as possible but no simpler". So the main question I have, when reading your summary from Yuriy below -- which matches exactly the discussion I've seen Mark Horstmeier at Kynetx having with Markus and Animesh Chowdhury at Neustar -- is that: * if an XDI client is allowed to send multiple operations to a server * AND those operations are unordered * AND the server commits all operations until it hits one that produces and error and then stops and reports the error * THEN it is completely unpredictable what operations will be committed or not committed in the message. Now, I'm not a protocol expert, or a transactional integrity expert, but it seems to me that there is a fundamentally flaw in this approach and we as a TC would be negligent in putting out a protocol governed by those rules. It seems there are three basic ways to fix this flaw and still "keep it simple": 1. Allow only one operation per message (as Bill has suggested several times in the past). This eliminates any need for any transactional integrity at all. ** Bill: Yes. Multiple operations per message implies a transaction, so not doing transactional integrity then for me violates the principle of least surprise. As always, +1 for one op per message on my part. I also would like to point out that you can send each message over the same connection without reopening a connection, whether that connection is HTTP, Web Sockets, or XMPP. This is what I assume we'd have with the current proposal for delivering a single message/response over HTTP. 1. Allow multiple unordered operations in a message but specify that either all the operations are committed or none are. This sounds like transaction grouping and group atomicity as are available in RDBMSs, except for being unordered. It seems to me that different orders would often produce different results, as you say above, so unordering should not be allowed. ** Bill: Agreed on lack of ordering being an issue with this option. It also conveys transactions by grouping multiple options together and using transactional integrity language such as 'all operations are committed or none are'. 1. Allow multiple operations in a message but require the operations to be ordered and specify that all successful operations up to the one that causes an error are committed and none of the operations from from the one that causes error forward are committed. This is what I was assuming when proposing we think about XDI over a persistent connection. It sounds to me much easier to implement than transactions that group multiple operations into a unit. **Bill: This is doable, but your error handling is going to be complicated, for both XDI endpoints and XDI messengers (I prefer endpoints and messengers over server and client terminology). Also the message looks like a transaction, which may be confusing to implementers. Even with single message per HTTP request, what happens if the requests arrive in a different order than the order sent? This would seem to introduce uncertainty. A persistent connection actually does not have this problem since earlier requests are guaranteed to be earlier in the stream. Perhaps we should *require* opening a persistent connection whenever there's danger of reordering giving unpredictable results. **Bill: Good question. For a $add and $get I do not believe there is a danger of unpredictable results, unless we are requiring that the implicit contexts containing subjects and objects in the statements added. For $delete, the danger is present. As a result I propose the following $ words to allow a single mutex per XDI context for state changing operations: $lockholder - A predicate indicating that the subject, which is a context, is locked by the object, which is an XDI entity (individual or organization) with the given i-name. $lock - A $op that adds two statements to the graph. For the first statement the subject is the target context, the predicate is $lockholder, and the object is the i-name of the lock holder. For the second statement the subject is the first statement in an XDI cross-reference, the predicate is $expires, and the object is a date-time literal. $unlock - A $op that removes the lock statements. Business rules: 1. An XDI compliant endpoint MUST execute a $delete operation to remove both of these statements within 3 minutes (arbitrary, can change) of the expiration value of a lock. 2. An XDI compliant endpoint MUST NOT remove a lock unless it receives a $unlock from the lock holder, or an agent thereof. 3. An XDI compliant endpoint MUST NOT allow any context-altering (other than versioning information) operation to be executed if a lock is in place, unless the message is from the lock holder. 4. An XDI compliant operation MAY queue operations that are not executed, which are received when a context is locked, and then execute them when the lock is released or expires. This solves some of the problem, but maybe not all of it. A single user can still send messages that cause an unpredictable behavior I think - I'd need to go over it in more depth to have more confidence either way. Thoughts? =Drummond On Mon, Jul 9, 2012 at 9:11 AM, Michael Schwartz <mike@gluu.org<mailto: mike@**gluu.org <mike@gluu.org>>> wrote: Drummond, I don't think we should implement transactions in XDI 1.0. LDAP has lived without transactions for 20 years and not been too much the worse for it. Transactional persistence is the domain on SQL databases right now. Also, nothing would stop an XDI server vendor from implementing transactions, and once the data exists that define the best practices, we define standards at a later time. I just don't think its on the critical must-have features. Note: I do think that XDI servers should be transactional with regard to single operations! I thought Yuriy Zabrovarnyy might have some thoughts on this... here they are: Untill now we agreed on the rules that are described here: - https://wiki.oasis-open.org/**xdi/XdiMessagePatterns<https://wiki.oasis-open.org/xdi/XdiMessagePatterns> - https://wiki.oasis-open.org/**xdi/MessagingErrorCode<https://wiki.oasis-open.org/xdi/MessagingErrorCode> The rules are: a) all operations are independent even if they come in one message. b) if one operation is failed then further operation execution is stopped and error is send with actual problem and statement where problem occurs. More info on the OX wiki http://ox.gluu.org/doku.php?**id=docs:graphoperations<http://ox.gluu.org/doku.php?id=docs:graphoperations> Example: Lets say we have three operations. Two of them are successfully executed and the last one fails. The error is sent to the client but the two successfully executed operation are NOT ROLLED BACK. c) operations are executed in order they appears in the message request (accept $add operation where sorting is made on server to provide correct execution + variable support) 3. CURRENT oxServer IMPLEMENTATION Some time ago i've already investigated question about transaction support with LDAP. And problem is that OpenDJ does not provide such feature. There are only one more or less good implementation provided by UnboundID, see http://www.dirmgr.com/blog/**2009/4/6/transactions-in-the- **unboundid-directory-server.**html<http://www.dirmgr.com/blog/2009/4/6/transactions-in-the-unboundid-directory-server.html> . - Mike ------------------------------**------- Michael Schwartz Gluu Founder / CEO office: +1 646-810-8761<tel:%2B1%20646-810-8761> mike@gluu.org<mailto:mike@**gluu.org <mike@gluu.org>> On Mon, 9 Jul 2012, Drummond Reed wrote: Joseph, my intuition is that your intuition is right ;-) Seriously, it keeps coming up that we need enough transaction integrity semantics in the XDI protocol so that clients can unambiguously ask a server to do what the client wants, and servers can unambiguously know how to tell a client what messages/operations worked and what didn't. In other words, accurate, unambiguous communication of state. Since in XDI every operation request in every message is a state change (even a $get is a state change if it is logged), then maybe we just need to bite the bullet and apply the multiplicity patterns to XDI messaging. The basic rules would be: 1. A single message with a single operation would not need any transaction integrity semantics - the message either succeeds or fails, and the message ID already identifies the message. However the server may need to return the message ID in the response. 2. A single message with a collection of operations would need to order the operations using ordinal statements. The server performs the operations in order. If an operation fails, the whole message fails, and the server sends back the full XDI statement of the operation that failed (because that XDI statement is already unique). 3. A collection of messages (whether with single operations or collections of operations) would need to: 1. Order the messages so the server can perform them in order. 2. Include basic commit/rollback semantics for message collections. If you agree, then we should put together a formal proposal for this so we can start discussing it. =Drummond On Sat, Jul 7, 2012 at 4:50 PM, Joseph Boyle <planetwork@josephboyle.net< **mailto:planetwork@josephboyle.**net <planetwork@josephboyle.net>wrote:On Jul 6, 2012, at 6:53 AM, Barnhill, William [USA] wrote: I propose that rather than deciding which methods to explicitly support, we instead create a generic authentication mechanism, i.e. something like SASL. If XDI messaging were a connection-oriented protocol then I would suggest an XDI binding of the SASL abstraction layer. However, to my knowledge XDI messaging has always been, and is intended to remain, a connectionless protocol (or more precisely a message-oriented protocol). I strongly feel that it will turn out to be advantageous to maintain connections, and that we should anticipate this. I realize we want to get the simplest case (no connection, minimal HTTP methods/codes, minimal authentication) out the door quickly, but can we give this a little thought now? It may be as simple as numbering messages and responses so that the client knows which response was to which message, or only sending responses in order received. Is it clear where a message ends - do we have a clear end of message marker? ------------------------------**------------------------------**--------- To unsubscribe, e-mail: xdi-unsubscribe@lists.oasis-**open.org<xdi-unsubscribe@lists.oasis-open.org> <mailto:xdi-**unsubscribe@lists.oasis-open.**org<xdi-unsubscribe@lists.oasis-open.org>For additional commands, e-mail: xdi-help@lists.oasis-open.org<**mailto: xdi-help@lists.oasis-**open.org <xdi-help@lists.oasis-open.org>>------------------------------**------------------------------**--------- To unsubscribe, e-mail: xdi-unsubscribe@lists.oasis-**open.org<xdi-unsubscribe@lists.oasis-open.org> For additional commands, e-mail: xdi-help@lists.oasis-open.org
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]