Synchronization Use Cases

Some rough notes from us about the synchronization use cases we’d like to add to the use case discussions.

From: Daniel A. Perry
Sent: Friday, October 08, 2010 5:39 PM
To: Richard Sand
Subject: Synchronization Use Cases

Please let me know your comments:

-- Synchronization Use Cases --

General Scenario: A hosting provider and customer need to exchange updates to identity information bi-directionally. The user base being dealt is assumed to be quite large.

Synchronization Case 1: Customer needs to send a number of updates to the provider. This is trivial to do with the current SPML standard.

Synchronization Case 2: Customer needs to pull changes made to the identities in the hosting provider back into their own systems. When dealing with a large user base, it’s very inefficient to have to search for changes by comparison, and searching for users off modification can be unreliable. The service provider should maintain a revision number for the user repository, and should support this revision number in a manner to allow only the changed users to be queried.

Synchronization Case 3: Case 3 is a combination of cases 1 and 2. When doing bi-directional synchronization both cases 1 and 2 will be performed. Use case is an optimization where the synchronization is started with the customer sending a batch of updates to the provider, and also including the revision that it last successfully synchronized with the provider. The provider responds with a list of changes that since the revision number, and also with the current revision number as per use case2.

Synchronization Case 4: Case 4 is further refinement of use case 3. In case 3 only the provider was keeping trying of a revision number. Use case 4 adds a revision number to both sides. This allows for a simple three message exchange which either party (customer or provider) can initiate to achieve a full synchronization. The message exchange is as follows:

1) Initiating party requests synchronization, and specifies the current revision of the other’s database that they have.

2) Other party responds with all changes since that revision. That same message asks the initiator for all changes since the last applied revision.

3) Initiator responses with all changes since the last revision.

Then both parties can asynchronously process those changes, and only update the current-applied revision number which they store for the other party when those changes are successful. (This model is similar to the retransmit mechanism in TCP). Finally, the last-revision sent to retrieve all changes since then will need to be an expression. This allows for cases where a few changes fail to apply during synchronization and need to be retried. Next are a few sub-cases delaying with retries:

Case 4a) Repeated retries. In this case, the last-applied revision is a simple expression such as “>X”, indicating that the initiating party has applied all changes up to revision X, and wish to retrieve all changes after X. The issue with approach is that if a mixture of success synchronizations and failures occur that those events will be retried again. For some targets this may cause problems, for other it may not and may save the extra effort of keeping track of exactly what failed.

Case4b) Specified Retries. In this case, the initiating party needs to keep track of exactly what changes applied successfully and which need to be retried. The initiator will request a set of changes like “A-C,E,>G” (requesting that A through C and E be retried, and any new events after G be sent as well); skipping the changes that have been applied successfully. This allows the client to avoid receiving duplicate events from a previous run, but puts the burden of keeping details about exactly what failed on the client.

The responder needs to support either of syntaxes for querying changes, leaving the decision for “simple with duplicates” vs. “complex with no duplicates” up to the initiating client.

provision message