In general, I agree but I’m also still trying to see how the “new” pieces may fit or lead to something really new.
BTW, what you describe is the rationale for the SOA-RM distinguishing between the underlying capability — often legacy that can’t easily be refactored — and the access mechanism that allows the underlying capability to be opaque and the access mechanism to expose RWE without requiring detailed knowledge of how to invoke the capability.
Ken
------------------------------------------------------------------------------ Dr. Kenneth Laskey MITRE Corporation, M/S F510 phone: 703-983-7934 7515 Colshire Drive fax: 703-983-1379 McLean VA 22102-7508
How we have adopted legacy systems in SOA before (we could not extract functionality and data from them)? - We used them as data sources: we have built a service on the top of such system; the service was responsible for communication with other services and , on its other 'end', requested system's functionality to perform operations, i.e. analyse Big Data. The system was sitting with/on the Big Data. The surevice could participate in any number of orchestrations and choreographies shielding the the system from others. This later was called a data service together with other (though similar) data services that performed only simple CRUD operations (but dealing with the data schema mapping). Thus, the legacy system was never promoted into services (except by s...d developers) but actively used in SO Ecosystem.
I see the solution in that the applications capable to work with Big Data remain in the status of applications and shielded/wrpped by "SOA services" to be included in the SO Ecosystem. Whether these applications comprised MS or not is immaterial. If a body of a MS leads to an analytical system sitting with Big Data, it is the same system I mentiond before.
- Michael
If I have a data service to retrieve data from a source so it can undergo some other processing (maybe by another service), I think of this as moving the data. For large amounts of data, this could run into bandwidth constraints. It is a deployment concern of where I put the data vs. where I put the processing. From a SOA perspective, I really don’t care. As a consumer, I invoke the processing and the processing can invoke the data services.
My thoughts related to microservices were that it might be easier to move the manifest of a VM closer to the data. Is that a fundamental difference with SOA. Maybe not. I use a service for its RWE, whether that RWE is a data access to a remote source or a spinning up of the processing apparatus somewhere less remote.
Ken
Gentlemen, this is the second message in response to the discussion.
Ken says, "Doing a bit more reading, a consideration could be whether it is easier to move the data to the processing or the processing to the data." Also, "... if I generate terabytes of data a day, it may be more efficient to bring the processing to the data rather than the data to the processing. ...about deploying on-premise infrastructure vs. cloud vs. what combination? What SOA principles are still useful to guide us? The house of the SOA paradigm may still basically function "
Regarding ‘moving data’, I’d like to put your attention on the fact that business and utility services do not encapsulate data as objects do. To access actual data, SO Ecosystem uses so-called Data Services. The major different between the Data Services and ‘data source drivers-on-HTTP-steroids’ is in business semantic mapping ability of the Data Services. Instead of encapsulating actual data, “SOA-style” services encapsulate business semantic meta-data about data. That is, ANY data they meet the sematic rules of the meta-date (owned by the service) can be used decoupling “SOA-style” services from the inflexible and separately owned data sources.
I am not clear about mentioned ‘moving data’: imagine we have a service that is supposed to process data on the same physical server where the database with that data situates. So what? They are still in different SW/HW processes. We can minimise the traffic via HTTP between the service’s body/implementation and the database, but this had been possible always before as well! Simply instead of HTTP communication protocol any programmatic interface to data store might be used. What ‘moving’ we talk about here? The only difference we need to consider is that a special service implementation should be used to process terabytes of data.
I think that no new SO principles are needed and SOA paradigm is still fine. Let’s assume we collect terabytes of data in a Cloud. As any Cloud, a Cloud storage is a … service, Oups! And, as a service, it likely would not share its service platform (storage environment) with another service (in Cloud or on-premise infrastructure), i.e. the processing entity will be another service in any case. A problem with Cloud is in that it uses only HTTP, which might be not suitable for moving the terabytes of data from a storage-service to the processor-service regardless its location.
A solution may be in that those two service merge – the data collector provides the processing and vice versa. We have seen such combinations, e.g. in ESB vendors (Web Services + messaging + data transformation). This is a violation of a Principle of Separation of Concerns, but it is not a conceptual violation – when the speed of “HTTP” would be adequate to the terabytes of data, these services can split again, if it would be needed. However, this also means we do not need to move services anywhere – mentioned merge will be a new breed of services.
Cheers,
- Michael
Ken, Martin, This is turning into an interesting conversation, and one that needs to be made outside of the app provider/paas context. Their self-interest tends to get in the way. I think that's why our ongoing work on SOA is relevent and, to use Ken's metaphor, the kitchen may need to be remodeled. I think the key issues revolve around ownership boundaries and governance, hence the difficulty with maintaining data integrity across concurrent micro-service iterations housed in containers in differing execution contexts, ie. public v. private clouds, inter-enterprise infrastructure, governmental systems. I can foresee data service providers coming into existence to service these cross-environment apps/microservices that use common datasources. I also look forward to hearing from a micro-services developer. Rex
On 5/30/2016 8:50 PM, Martin Smith wrote:
Ken -- Always good to open the aperture: referring to your idea of moving the app to the data, etc.
However. You identified one requirement for making this work: very robust config/patch/revision control to keep all the copies of the code in synch. Not trivial.
Backing further up the development process I think there's another hard problem: where do the requirements for the micro-service come from? Is the motivation for someone to make a micro-service to meet the needs of a single, specific containerized application? If so, it seems a remote possibility that that specification would closely meet the needs of other applications. Result: minimal re-use, unless the whole application is copied. (I admit I am assuming what provides the motivation for creation of a micro-service.)
I keep thinking: how does someone creating a micro-service differ from someone adding a new function to an open-source project on Github? The latter seems more valuable both because it's likely to be written in contemplation of re-use by others, and also the code is probably more efficient because it probably uses in-process communication vs. sockets.
Looking forward to hearing from a micro-services developer!!
Martin
PS--Also just starting to noodle around with AWS Lambda ("serverless" framework.)
--
Rex Brooks
Starbourne Communications Design
Email: rexb@starbourne.com
GeoAddress:
1361 Addison St. Apt. A
Berkeley, CA 94702
Phone: 510-898-0670
--------------------------------------------------------------------- To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail. Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php
|