OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

soa-rm message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]


Subject: Re: moving the data or the processing [was: [soa-rm] Microservices re-use?]


In general, I agree but I’m also still trying to see how the “new” pieces may fit or lead to something really new.

BTW, what you describe is the rationale for the SOA-RM distinguishing between the underlying capability — often legacy that can’t easily be refactored — and the access mechanism that allows the underlying capability to be opaque and the access mechanism to expose RWE without requiring detailed knowledge of how to invoke the capability.

Ken

------------------------------------------------------------------------------
Dr. Kenneth Laskey
MITRE Corporation, M/S F510          phone: 703-983-7934
7515 Colshire Drive                           fax: 703-983-1379
McLean VA 22102-7508

On Jun 17, 2016, at 6:26 AM, Mike Poulin <mpoulin@usa.com> wrote:

How we have adopted legacy systems in SOA before (we could not extract functionality and data from them)? - We used them as data sources: we have built a service on the top of such system; the service was responsible for communication with other services and , on its other 'end', requested system's functionality to perform operations, i.e. analyse Big Data. The system was sitting with/on the Big Data. The surevice could participate in any number of orchestrations and choreographies shielding the the system from others. This later was called a data service together with other (though similar) data services that performed only simple CRUD operations (but dealing with the data schema mapping). Thus, the legacy system was never promoted into services (except by s...d developers) but actively used in SO Ecosystem. 
 
I see the solution in that the applications capable to work with Big Data remain in the status of applications and shielded/wrpped by "SOA services" to be included in the SO Ecosystem. Whether these applications comprised MS or not is immaterial. If a body of a MS leads to an analytical system sitting with Big Data, it is the same system I mentiond before.
 
- Michael 
 
Sent: Thursday, June 16, 2016 at 9:20 PM
From: "Ken Laskey" <klaskey@mitre.org>
To: "Mike Poulin" <mpoulin@usa.com>
Cc: soa-rm@lists.oasis-open.org
Subject: moving the data or the processing [was: [soa-rm] Microservices re-use?]
If I have a data service to retrieve data from a source so it can undergo some other processing (maybe by another service), I think of this as moving the data.  For large amounts of data, this could run into bandwidth constraints.  It is a deployment concern of where I put the data vs. where I put the processing.  From a SOA perspective, I really don’t care.  As a consumer, I invoke the processing and the processing can invoke the data services.
 
My thoughts related to microservices were that it might be easier to move the manifest of a VM closer to the data.  Is that a fundamental  difference with SOA.  Maybe not.  I use a service for its RWE, whether that RWE is a data access to a remote source or a spinning up of the processing apparatus somewhere less remote.
 
Ken
 
On Jun 3, 2016, at 1:22 PM, Mike Poulin <mpoulin@usa.com> wrote:
 
Gentlemen, this is the second message in response to the discussion. 
 
Ken says, "Doing a bit more reading, a consideration could be whether it is easier to move the data to the processing or the processing to the data." Also, "... if I generate terabytes of data a day, it may be more efficient to bring the processing to the data rather than the data to the processing. ...about deploying on-premise infrastructure vs. cloud vs. what combination?  What SOA principles are still useful to guide us?  The house of the SOA paradigm may still basically function "
 
Regarding ‘moving data’, I’d like to put your attention on the fact that business and utility services do not encapsulate data as objects do. To access actual data, SO Ecosystem uses so-called Data Services. The major different between the Data Services and ‘data source drivers-on-HTTP-steroids’ is in business semantic mapping ability of the Data Services. Instead of encapsulating actual data, “SOA-style” services encapsulate business semantic meta-data about data. That is, ANY data they meet the sematic rules of the meta-date (owned by the service) can be used decoupling “SOA-style” services from the inflexible and separately owned data sources.
 
I am not clear about mentioned ‘moving data’: imagine we have a service that is supposed to process data on the same physical server where the database with that data situates. So what? They are still in different SW/HW processes. We can minimise the traffic via HTTP between the service’s body/implementation and the database, but this had been possible always before as well! Simply instead of HTTP communication protocol any programmatic interface to data store might be used. What ‘moving’ we talk about here? The only difference we need to consider is that a special service implementation should be used to process terabytes of data.
 
I think that no new SO principles are needed and SOA paradigm is still fine. Let’s assume we collect terabytes of data in a Cloud. As any Cloud, a Cloud storage is a … service, Oups! And, as a service, it likely would not share its service platform (storage environment) with another service (in Cloud or on-premise infrastructure), i.e. the processing entity will be another service in any case. A problem with Cloud is in that it uses only HTTP, which might be not suitable for moving the terabytes of data from a storage-service to the processor-service regardless its location.
A solution may be in that those two service merge – the data collector provides the processing and vice versa. We have seen such combinations, e.g. in ESB vendors (Web Services + messaging + data transformation). This is a violation of a Principle of Separation of Concerns, but it is not a conceptual violation – when the speed of “HTTP” would be adequate to the terabytes of data, these services can split again, if it would be needed. However, this also means we do not need to move services anywhere – mentioned merge will be a new breed of services.
 
Cheers,
- Michael
 
Sent: Wednesday, June 01, 2016 at 12:57 AM
From: rexbroo <rexb@starbourne.com>
To: soa-rm@lists.oasis-open.org
Subject: Re: [soa-rm] Microservices re-use?

Ken, Martin,

This is turning into an interesting conversation, and one that needs to be made outside of the app provider/paas context. Their self-interest tends to get in the way. I think that's why our ongoing work on SOA is relevent and, to use Ken's metaphor, the kitchen may need to be remodeled.

I think the key issues revolve around ownership boundaries and governance, hence the difficulty with maintaining data integrity across concurrent micro-service iterations housed in containers in differing execution contexts, ie. public v. private clouds, inter-enterprise infrastructure, governmental systems. I can foresee data service providers coming into existence to service these cross-environment apps/microservices that use common datasources.

I also look forward to hearing from a micro-services developer.

Rex

On 5/30/2016 8:50 PM, Martin Smith wrote:
Ken -- Always good to open the aperture: referring to your idea of moving the app to the data, etc. 
 
However.  You identified one requirement for making this work: very robust config/patch/revision control to keep all the copies of the code in synch.  Not trivial.  
 
Backing further up the development process I think there's another hard problem: where do the requirements for the micro-service come from? Is the motivation for someone to make a micro-service to meet the needs of a single, specific containerized application? If so, it seems a remote possibility that that specification would closely meet the needs of other applications. Result: minimal re-use, unless the whole application is copied. (I admit I am assuming what provides the motivation for creation of a micro-service.) 
 
I keep thinking: how does someone creating a micro-service differ from someone adding a new function to an open-source project on Github? The latter seems more valuable both because it's likely to be written in contemplation of re-use by others, and also the code is probably more efficient because it probably uses in-process communication vs. sockets. 
 
Looking forward to hearing from a micro-services developer!!
 
Martin
 
PS--Also just starting to noodle around with AWS Lambda ("serverless" framework.) 
 
 
 
 
On Fri, May 27, 2016 at 4:45 PM, Ken Laskey <klaskey@mitre.org> wrote:
Doing a bit more reading, a consideration could be whether it is easier to move the data to the processing or the processing to the data.
 
In our work on describing SOA, we assumed crossing ownership boundaries was a critical part of the paradigm: the consumer used services where they were offered rather than having a private copy.  Part of the rationale was that private copies held by consumers (who were effectively also forcing themselves to be providers) often meant enumerable modified copies that became one-of-a-kind albatrosses to maintain.  Under the SOA paradigm, the consumer and provider interacted under agreed upon terms (the execution context) with neither under the other’s control.  (What control really existed before is another topic for discussion.)
 
But with VMs and containers, we should be able to manage exact copies, including making sure the copies have a known, controlled configuration.  So now if I generate terabytes of data a day, it may be more efficient to bring the processing to the data rather than the data to the processing.  My configuration management makes sure the “manifest” for the new VM/container is up to date at all needed locations, and things like applications needing to be deployed are already pre-staged to any place that may need that application.
 
What does this say about deploying on-premise infrastructure vs. cloud vs. what combination?  What SOA principles are still useful to guide us?  The house of the SOA paradigm may still basically function but it may need to have the kitchen remodeled.
 
Ken
------------------------------------------------------------------------------
Dr. Kenneth Laskey
MITRE Corporation, M/S F510          phone: 703-983-7934
7515 Colshire Drive                           fax: 703-983-1379
McLean VA 22102-7508
 
On May 25, 2016, at 10:58 AM, Martin Smith <bfc.mclean@gmail.com> wrote:
 
This is a request for input on how re-use is accomplished in the microservices world.  Please provide info or comment via this list or via the public comment list for this TC:  soa-rm-comment@lists.oasis-open.org 
 
Please also pass this request to others in your network who may have microservices implementation experience.
 
 
In the April 13, 2016, meeting of the TC there was a discussion of comparison of characteristics of SOA architecture/design principles vs. "microservices" architecture (with specific reference to NIST Special Publication 800-180 (DRAFT) NIST Definition of Microservices, Application Containers and System Virtual Machines  of Feb 2016.) 
 
One issue discussed was how re-use is accomplished in a microservices-style implementation.  
 
It was suggested that the NIST SP seemed to imply that a microservice instance is confined inside a single container. If so, this would be a significant distinction from an “SOA-style” service, in that it would mean that the only way a microservice could be re-used outside the container would be to copy the code and run an entirely separate instance in another container or elsewhere. This “confinement” also implies that microservces are not “multi-user” (or “multi-tennant”) whereas multi-tenant capability might be considered an intrinsic characteristic of SOA-style services. These apparent characteristics of microservices would seem to give away most of the potential efficiencies of a scaleable, easily maintainable shared service.
 
Can anyone with microservices implementation experience explain how re-use is accomplished in this architectural style?  
 
Martin
  
 
 
 
--
Martin F Smith, Principal
BFC Consulting, LLC
McLean, Va 22102
 
 
--
Martin F Smith, Principal
BFC Consulting, LLC
McLean, Va 22102
703 506-0159
703 389-3224 mobile
 
-- 
Rex Brooks
Starbourne Communications Design
Email: rexb@starbourne.com
GeoAddress:
1361 Addison St. Apt. A
Berkeley, CA 94702
Phone: 510-898-0670 
--------------------------------------------------------------------- To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail. Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php



[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]