Hi,
I was afraid the proposal was going in this direction. This note is probably going to sound overly negative but I'll go ahead anyway...
One of the design criteria we had with the original programming model was simplicity for end-users, focusing on common cases, and minimal use of APIs. I'm concerned we are beginning to loose sight of those goals. I'm also concerned at the extent of changes being proposed and impact they will have on the other SCA specifications.
I wonder if part of the problem is everyone has a specific set of use-cases in mind and we are jumping into API design before fully understanding the scenarios each of us wants to enable. Would it be beneficial to step back and enumerate from an end-user perspective what scenarios we want to enable first? Once we have a common understanding of these, we can look at alternative proposals by comparing how application code would need to be written. A preference would be to simplify what is currently in the specification through subtraction and accommodate common use cases through existing mechanisms as opposed to introducing a new programming model, which is being done with the proposal. For uncommon cases, I think the bar should be "possible but potentially difficult". This approach would hopefully bring forth issues in design as well as usability while at the same time avoiding the risk of introducing wholesale changes this late in the specification process. Is this a reasonable approach?
That said, I wanted to comment more specifically on the proposal. I'll start by summarizing my concerns and then respond specifically inline.
Jim
My general concerns are centered on:
1. The complexity introduced in this approach
The proposed API is considerably more complex then what we currently have, which I believe can be made to work with (hopefully) no or minor changes. It is also much more complex than alternative conversational programming models such as Seam and the alternative simplification proposed by Mike and myself. I'll list specific areas below where I think it is too complex.
2. The association of service contract signatures with infrastructure-specific types
Service contracts should not have SCA- or other infrastructure-specific types in their operation signatures. Requiring SCA types in service contract signatures:
a. Ties clients and service provider implementations forever to SCA (annotations are OK since they are not "required" to run - e.g. in Java they need not be on the classpath for a class to be loaded).
b. Makes interop much more difficult to achive. For example, I'm not sure how CallableReference would be represented to a .NET, JMS, or JAX-WS client.
c. Is inconsistent with both service-based design (operations take *application/business* data as parameters) and the direction most Java-based programming models have been evolving to. That is, supporting basic Java classes that in most scenarios have little to no code-level dependencies on infrastructure, which promotes simplicity, a level of portability, and makes migration to future specification versions much easier.
3. Bleeding of protocol-specific requirements into the programming model
The proposed API unnecessarily reflects a scheme where conversation ids are generated through a forward synchronous invocation. This is not always a requirement, and when it is the infrastructure can handle such cases transparently to application code.
4. The apparent loss of import functionality such as what is possible today with @EndsConversation and @Callback today (see comments inline)
5. Increased difficulty in testing
Requiring the use of CallableReference makes out-of-container testing much more difficult than using mocks.
6. The amount of change introduced by the proposal, particularly as it will significantly impact assembly and bindings
I believe the current proposal will require wholesale changes in the way conversations are defined at the assembly level as well as specific binding manifestations. For example, the current JMS binding section on conversations would need to be completely re-written. --------------------------------------------
More specific comments inline...
On Jul 24, 2008, at 3:58 AM, Simon Nash wrote:
Jim,
Thanks for these excellent questions. My responses are in <scn>...</scn> below.
Simon
Simon C. Nash, IBM Distinguished Engineer
Member of the IBM Academy of Technology
Tel. +44-1962-815156 Fax +44-1962-818999
Hi,
I'm sorry I was unable to attend the second half of Wednesday's meeting so want to caveat my response by saying I may be missing some of the discussion context. I'll break down my response into several parts.
1. Question on CallableReference
I think it would be helpful if all of the code examples were first laid out, particularly since I don't think the examples can be made to compile (I'm probably missing something here). One of the places I'm having trouble is with the genericized version of CallableReference, which I assume takes the forward service interface as a parameter type. So, if OrderService is defined as:
public interface OrderService {
public void orderApples(int quantity, CallableReference<OrderService>);
public void orderPlums(int quantity, CallableReference<OrderService>);
}
I'm having trouble seeing how this can work:
public void orderApples(int quantity, CallableReference<OrderService> myClient) {
boolean success = placeOrder("apples", quantity);
myClient.getCallback().reportResult(success);
}
without adding a parameter type to CallableReference taking the callback interface, as in CallableReference<FORWARD, CALLBACK>. If this were added, I think the programming model would be very cumbersome and would require people to declare a forward-only CallableReference as CallableReference<SomeInterface, Void>. I may be missing something really basic here.
<scn> I didnt't try compiling this, and when I wrote it I did wonder whether it would work as written. You may be correct that it would need a cast of the result of getCallback() in order to compile. The code would then look like:
public void orderApples(int quantity, CallableReference<OrderService> myClient) {
boolean success = placeOrder("apples", quantity);
((OrderCallback)myClient.getCallback()).reportResult(success);
}
I agree that it would be bad to add a second generic parameter to CallableReference.</scn>
Forcing application code to cast like this is equally problematic. One of the original proposals for programming model APIs required this and was rejected for the following reasons:
1. It's not a discoverable API. In other words, there is nothing obvious in the API that indicates a cast is possible and safe. This forces code to make potentially dangerous assumptions about references. It also requires people to "read the manual" as opposed to looking at an API or getting "drop-down" help in an IDE.
2. From an infrastructure provider perspective it is also problematic: it forces references to always be proxied or have a mixin applied statically or during class load time. This prohibits optimizations where references are directly injected.
We discussed the casting approach at length and instead chose an explicit cast operation in the API as a cleaner alternative.
I'm also really confused by the following:
CallableReference<OrderService> myConversation = myService.startNewOrder(); // returns an ID for the entire fruit order
CallableReference<OrderService> myAppleOrder = myConversation.createCallbackReference(); // myAppleOrder is linked to myConversation
myAppleOrder.orderApples(12); // the infrastructure sends IDs for both myConversation and myAppleOrder
CallableReference<OrderService> myPlumOrder = myConversation.createCallbackReference(); // myPlumOrder is linked to myConversation
myPlumOrder.orderPlums(6); // the infrastructure sends IDs for both myConversation and myPlumOrder
Specifically:
- I don't think "myAppleOrder.orderApples(12)" will compile
<scn> Sorry, you are correct. This should be myAppleOrder.getService().orderApples(12);</scn>
- What is the purpose of myAppleOrder and myPlumOrder? Where would these be used?
<scn> These are "proxy" CallableReference objects that contain an identity for the callback. They are both linked to the CallableReference object myConversation, which contains an identity for the conversation. Note that they are created using the createCallbackReference() API, not the createCallableReference() API. This proxy-linking arrangement allows both CallableReference objects to be immutable, while providing the logical equivalent of both a CallbackID and a ConversationID on the forward call.</scn>
What happens if I want to introduce asynchrony (non-blocking operations) such as:
public interface OrderService { @OneWay
void startNewOrder();
@OneWay
void orderApples(..);
@OneWay
void orderPlums(..);
}
Does the proposal require interfaces to have a synchronous operation to start a conversation?
2. Clarification on service operation signatures
I'm unclear if by the following the proposal intends to require use of CallableReference for conversational interactions:
A simple extension to the model already proposed can solve both these problems. A conversation would be initiated by the service creating a CallableReference and returning it to the client. This CallableReference contains an identity for the conversation. This client then makes multiple calls through this CallableReference instance. Because these calls all carry the same identity, a conversation-scoped service will dispatch all of them to the same instance.
I'm assuming this is just for illustrative purposes and it would be possible for a conversation to be initiated in response to the following client code, which does not use the CallableReference API:
public class OrderClient ... {
@Reference
protected OrderService service;
public void doIt() {
service.orderApples(...);
service.orderPlums(...); // routed to the same target instance
}
}
Is this correct?
<scn>In a word, No. All conversations would need to be initiated by the proposed mechanism of having the server return a CallableReference to the client. This allows the conversation identity to be generated by the server, not the client. Several people (e.g., Anish and Mike) have called this out as an issue with the current mechanism for conversations.</scn>
Sorry for being so thick, but I don't see why the above could not be supported using "server" generation of conversation ids. We should be careful here to specify what we mean by "server", and whether invocations are flowing through a wire or a client external to the domain. I don't think the term "server" should necessarily mean "the runtime hosting the service provider." Sometimes this may be the (degenerate) case, but not always.
For communications flowing through wires, the only thing we can likely say is "conversation ids are generated by the domain". I would imagine most domain implementations would chose an efficient id generation scheme that can be done without context switching on every invocation (e.g. UUID generation provided by the JDK). However, in cases where this efficient identity generation is not possible, I believe SCA infrastructure can support the above code. In this case, the reference proxies would be responsible for making some type of out-of-band request to generate a conversation id to some piece of domain "infrastructure". The important thing is how conversation id generation happens does not bleed into the programming model and is transparent to the application. In other words, we not should require anything more complex than this when the OrderClient is wired to an OrderService:
public class OrderClient ... {
@Reference
protected OrderService service;
public void doIt() {
service.orderApples(...);
service.orderPlums(...); // routed to the same target instance
}
}
I will also note that other conversational programming models provided by JBoss Seam, the JAX-WS RI, and even BEA Workshop are this easy. SCA should not be any more complex than those alternatives.
For communications originating from a client external to the domain, I think the following are going to vary:
1. Whether the id is generated by the client or the service provider host 2. The "shape" of the id and how it is represented to clients. For example, in JAX-WS clients, I believe the id would need to be encapsulated in a javax.xml.ws.wsaddressing.W3CEndpointReference.
I think it would be a useful exercise to understand what the proposal would require when using a JAX-WS, JMS, or .NET Web Services client. For example, what would a JMS client be required to do to invoke a conversational service? Specifically, how would it obtain a handle to the conversational service? The same issues could be examined in the context of .NET or JAX-WS.
3. Clarification on how a conversation can be ended
The proposal states:
This combination of having the service initiate a conversation and the client decide when to end it provides the correct combination of semantics and resolves Anish's first concern..
Would it still be possible to have the forward conversation end as a result of a callback as in onFullfilled(..) being called on the following interface, without the client having to do something specific in response?
public interface OrderCallback {
@EndsConversation
void onFullfilled(..t);
}
<scn>I am not sure whether this would be possible. The client already possesses a CallableReference object containing a conversation identity, and it could continue to make calls through this reference whatever calls have been made from the service provider to the client. With my proposal, there would not be an active conversation in the callback direction, only in the forward direction. The service provider could do something locally to end the conversation that it has initiated, but making the client aware of which conversation was ended (if the client has multiple conversations in progress with the same service) would require more mechanism than I am proposing. This is because the @endsConversation callback cannot identify which client-side CallableReference object was used to make the forward call to which the @endsConversation callback is related.</scn>
The example would actually end the conversation in the forward direction. This is a pretty common use case (a callback ending the forward conversation between a client and a provider) that I believe needs to be enabled. In addition to using @EndsConversation, a provider can end the current conversation via the RequestContext API. If both of those are not allowed in the proposal, it would mean the only way to end a conversation is for the client to do so or for the conversation to time out, assuming @ConversationAttributes was used. If I understand proposed API correctly, I don't think this would be a good thing.
4. Clarification on @Callback
Could the following example:
@Context
RequestContext requestContext;
public void orderApples(int quantity) {
boolean success = placeOrder("apples", quantity);
requestContext.getCallback().reportResult(success);
}
be re-implemented as:
@Callback
protected OrderCallback callback;
public void orderApples(int quantity) {
boolean success = placeOrder("apples", quantity);
callback.reportResult(success);
}
The re-written version (which can be done with the existing calback mechanisms) is IMO less complex and much easier to test.
<scn>The simpler version would work for stateless providers, or for conversational providers whose clients don't use per-call corellation identities, but not for conversational providers whose clients use per-call corellation identities. This is because the @Callback was injected into a conversational instance when the instance was created at the start of the conversation. If the orders for apples and plums carry different client-side corellation identities (the equivalent of having different CallbackIDs with the current APIs), then the callback proxy can't be obtained from an injected field in the conversational instance because it may not carry the correct client correlation identity for the actual call being made. The same issue exists today with the current APIs, and this was the conclusion we reached some months ago when this was discussed.</scn>
Sorry I'm not following this and it may be because I lack some context or am making incorrect assumptions. A conversation is defined as shared context between a client and a provider. Only one client has visibility to that shared context and hence the correct callback correlation will always be known. If the shared context is implemented in a runtime as routing to the same instance, then injection of the callback proxy can safely be done when the instance is created and cached for the lifetime of the instance.
For composite-scoped components, a smart callback proxy can be used to always dispatch to the correct client, although that would be beyond what the spec allows (which we did discuss several months back).
----------------
As a general comment, I consider it an anti-pattern to put infrastructure-specific types in service contracts (e.g. CallableReference) since it ties a priori both provider and client implementations to specific infrastructure choices and makes testing much more difficult (APIs have to be mocked out). I'm almost tempted to say CallableReferences should almost never be passed around as part of service signatures (use strong types and pass proxies instead). I'm sure that may be controversial.
<scn>Given that SCA does currently support putting CallableReference in a business signature, I think it's reasonable to use this mechanism to initiate conversations. I'm open to other ideas for how to pass the conversation information from provider to client when a conversation is started, so that the client can create a CallableReference from that information.</scn>
How about the current API:
public class OrderClient ... {
@Context
protected ComponentContext context;
public void doIt() {
service.orderApples(...);
CallableReference<OrderService> reference = context.cast(service); //... }
}
Or, we could change "cast(..)" to
public interface ComponentContext {
<T> CallableReference<T> getCallableReference(T proxy); } I believe this to be similar in spirit to working with message-based correlation ids (e.g. JMS) where the forward message id is not available until after the message has been enqueued.
The above approach would avoid polluting client code which does not need to make use of CallableReference and, more importantly, the signature of OrderService which would not be forced to carry SCA types. If we require services to use CallableReference, we will be forcing all providers and all clients to always use SCA infrastructure. If in the future we need to change the CallableReference API in some incompatible way (EJB made significant changes three times), we run the risk of causing migration havoc for end users given service contracts will need to be changed. This will also make interoperability with other middleware problematic. For example, how would an interface signature with CallableReference be represented to a .NET or JMS client?
Thanks,
Jim
On Jul 18, 2008, at 5:45 AM, Simon Nash wrote:
In Wednesday's discussion at the F2F, Anish made the point that using the proposed CallableReference model to handle conversations has the problem that it requires the client to create the "conversation ID", but this responsibility should be with the server. Anish was also concerned that combining the "callback ID" and "conversation ID" concepts changes the callback programming model depending whether or not a conversation is in progress.
A simple extension to the model already proposed can solve both these problems. A conversation would be initiated by the service creating a CallableReference and returning it to the client. This CallableReference contains an identity for the conversation. This client then makes multiple calls through this CallableReference instance. Because these calls all carry the same identity, a conversation-scoped service will dispatch all of them to the same instance.
This combination of having the service initiate a conversation and the client decide when to end it provides the correct combination of semantics and resolves Anish's first concern..
To resolve Anish's second concern, a bit more programming or an extra mechanism is needed. If the client is using a conversational CallableReference that was created by the service, it can't add its own correlation ID to this CallableReference because CallableReferences are immutable. If it needs this per-call correlation, it would need to create a new CallableReference and pass this on the call. One way to do this would be to use business data. The client code would look like this:
CallableReference<OrderService> myConversation = myService.startNewOrder(); // returns an ID for the entire fruit order
CallableReference<OrderService> myAppleOrder = myConversation.createCallableReference(); // create an ID for the apple order request
myConversation.orderApples(12, myAppleOrder); // the infrastructure sends an ID for myConversation
CallableReference<OrderService> myPlumOrder = myConversation.createCallableReference(); // create an ID for the plum order request
myConversation.orderPlums(6, myPlumOrder); // the infrastructure sends an ID for myConversation
The service interface looks like this:
public interface OrderService {
public void orderApples(int quantity, CallableReference<OrderService>);
public void orderPlums(int quantity, CallableReference<OrderService>);
}
The service provider code looks like this:
public void orderApples(int quantity, CallableReference<OrderService> myClient) {
boolean success = placeOrder("apples", quantity);
myClient.getCallback().reportResult(success);
}
This works, but it makes the business interface rather cumbersome, and the business interface and service provider programming model are different from the non-conversational case. If we want to optimize the business interface for this case further, and make the service provider programming model the same for the conversational and non-conversational cases, we could allow CallableReferences to be linked together so that the CallableReference for the callback delegates its forward call identity to a second CallableReference for the forward call. Using this approach, the client code would look like this:
CallableReference<OrderService> myConversation = myService.startNewOrder(); // returns an ID for the entire fruit order
CallableReference<OrderService> myAppleOrder = myConversation.createCallbackReference(); // myAppleOrder is linked to myConversation
myAppleOrder.orderApples(12); // the infrastructure sends IDs for both myConversation and myAppleOrder
CallableReference<OrderService> myPlumOrder = myConversation.createCallbackReference(); // myPlumOrder is linked to myConversation
myPlumOrder.orderPlums(6); // the infrastructure sends IDs for both myConversation and myPlumOrder
The service interface now looks like this:
public interface OrderService {
public void orderApples(int quantity);
public void orderPlums(int quantity);
}
The service provider code is now identical to the non-conversational case and looks like this:
@Context
RequestContext requestContext;
public void orderApples(int quantity) {
boolean success = placeOrder("apples", quantity);
requestContext.getCallback().reportResult(success);
}
Simon
Simon C. Nash, IBM Distinguished Engineer
Member of the IBM Academy of Technology
Tel. +44-1962-815156 Fax +44-1962-818999
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
|