|
I noticed that one of Martin's comments isn't included in the
thread below, although it is in my own email thread for this
Subject, so I wanted to include it here because it appears that
Michael and Martin are both showing the beginning of efforts to
propose methods for developing decision-making tools/heuristics
that I think are valuable. I have bolded what I think are the most
relevant portions of their respective posts (the first two
messages below) and inserted my comments.
Cheers,
Rex
Here's what Martin wrote:
Rex, all- I think this might be getting interesting!
We seem to be coming up with a set of heuristics that bear on
appropriate module (service, microservice) size.
The heuristics might not all imply the same optimal scope, but
the most relevant heuristic might be a function of the specific
facts of the development task.
Heuristics:
1. Select a scope that is as small as possible with a local
minimum of number of interactions outside the component;
[I think this needs to be as small as
feasible, but no smaller, since I think it is possible to be too
small, so we need to define what would be too small for SOA and
too small for MSA. In both cases I think what we are looking for
OPTIMAL]
2. "Gather
together those things that change for the same reason, and
separate those things that change for different reasons.” Related
to requirements and also to business processes mapped to
software. Maybe a strength of micro-services because of the
principle of scoping to a single business owner.
[This was originally coined in relation to
the development of TCP bu tI think it relates most strongly to
Martin's notion of ownership, so I think I think it needs to
included in the development of a decision-making matrix such
as Michael suggests below.]
3.
The two-week rule: more generally, the good project-scoping rule
of delivering value incrementally to customers/sponsors vs. the
"big bang" approach.
4.
And something related to keeping teams small enough to have
close interaction, which implies relatively small scope for a
component/service.
The
first of these I could actually imagine being able to quantify
with a combination of a profiler, an optimization
algorithm and a network/node visualizer.
Martin
On 12/22/2016 3:00 AM, Mike Poulin
wrote:
Very interesting comments, Rex.
First, about transactional performance: I am not sure
that Microservices or Web Services are the appropriate
technologies for transactional behaviour and we have to
measure them afainst transactional performance. Nonetheless,
I agree with the 'balance' statement (for performance) and
service granularity comments. It is also known that regular
services, business services, do not provide "the transaction needs
ACID database"; that those services do not include
any data sources in thier models at all. For the purpose of
delaing with data sources, there as so-called Data Access
Services that, in essence, represent "smart data source
drivers on steroids" - their business functionality is in
the business information mapping between physiscal data
sources (supporting ACID) and business data models used by
business services (i.e. thier meta-data informational
models).
I do not see performance as the major objective for
Services - this depends on the business functionality behind
the interfaces. At the same time, Single Responsibility Principle of
a Principle of Separation of Concerns (I'd add - business
concerns) lead to the dual model of: 1) leave-services of
the finest granularity (in particualr business EC), and 2)
relatively stable coarser granular intermediary
combinations of leave services.
For example, A
company produces SW systems for financial market and
different sub-systems can accept customer payments. A
business utility composit service such as Card Payment
Service can be created and used in each sub-system. The
Card Payment Service comprises two temporary aggregates -
Credit Card "sub"-Service and Debit Card "sub"-Service
while each of them can vary depending on the card-reader
devise capabilities (with or without pin, with or without
contactless feature, etc.). These "sub"-Services are those
coarser granular intermediary combinations of leave
services. Following this service logic, the organisation
of business operations and IT development/support teams
can mirror the service relationships: a single division
providing Payment Service, sub-Divisions providing
different types of payment services including the Card
Payment Service, Cheque Payment Service, Mail Order
Payment Service, etc., and several teams including those
who orchestrate Credit Card "sub"-Service and Debit Card
"sub"-Service as well as several leave service teams.
At the same
time, a concept of orientation on service and afore
mentioned principles require maximisation of
inter-service interactsion (dependencies ) and
minimisation of service's functionality. I thionk this
lead us to the measurable matrix of granularity. It is
not easy or feasible talking about granularity maturity
becuase it depends on particular business task, but, as
a recommendation, I would point to creation of
relatively stable coarser granular intermediary
combinations of leave services as a Best Practice.
I think we would be better off to require require maximisation
optimization of inter-service interaction (dependencies ). I
think both Martin and Michael are onto something with the
combined concept of a measurable matrix of granularity and
a combination of a profiler, an optimization algorithm and a
network/node analyzer. What we need is to figure
out is what optimal is for size/complexity for SOA and for MSA.
I find "Gather
together those things that change for the same reason, and
separate those things that change for different reasons"
too mechanical. Since each service may have its owner and
the owners may be totally independent and even belong to
different businesses and legal authorities, some things
might be needed/permitted to change while other things
will not regardless that the changes root from the same
reasons. For example, if a service has its suppliers from
different organisations working in the legal regime and
there is anew regulation issued in this regime(mandatory
to everyone), some suppliers might adopt it many months
earlier than others. Mentioned statement is a useful
consideration, but still not a real driver for particular
combinations.
Regards,
- Michael
Another
rule of thumb that I ran across in another
O'Reilly Book from NGINX, Building
Microservices by Sam Newman, noted,
"Jon Eaves at RealEstate.com.au in Australia
characterizes a microservice as something that
could be rewritten in two weeks, a rule of thumb
that makes sense for his particular context."
Also, under Paradigm Principles for MSA I added, Single
Responsibility Principle; Robert C.
Martin’s definition states “Gather together those
things that change for the same reason, and
separate those things that change for different
reasons.” Both Eaves' rule of thumb and Martin's
principle could be thought of as qualitative
measures for ease of change.
Cheers,
Rex
On 12/19/2016 3:07 PM,
Ken Laskey wrote:
So what quantitative and/or qualitative
things are we looking towards for our metrics?
Ken
The Book
I recommended on SOA v.
Microservices by Mark Richards
suggests that the number of
inter-service calls required to
complete a single business/service
transaction is a key performance
factor in choosing between building a
larger, coarse-grained SOA service
with several component services to
handle the transaction and smaller,
typically faster Microservices used in
sequence to process the single overall
transaction. Another factor is the
importance of whether the transaction
needs ACID database
reliability/freshness or can withstand
looser BASE eventual consistency in
the database reliability/freshness.
However, unlike our previous
conclusion, I'm coming around to think
that service granularity is important.
I think it is less a case of choosing
between SOA and MSA for a whole system
and selecting component services based
on the performance parameters that
need to be fulfilled. I think it will
evolve toward blended architectures.
Cheers,
Rex
On 12/19/2016
12:59 PM, Ken Laskey wrote:
I am not proposing this as a
well thought out metric, but if the
optimal team size is 5.4 persons (or
something around that) and Conway’s Law
says the software product mirrors the
organization that creates it, and
microservices emphasize bounded context
(i.e. the scope), then does this tell us
something about service
granularity/size?
Or maybe it’s time for another cup
of coffee.
Ken
Hi All,
I havea couple of
comments to Ken's and
Bob's notes.
@Ken
<<Microservices
and business talks a lot
about the old SOA
aligning with business.
I think this makes a
lot of sense as long as
you look at business in
the broadest context and
realize most deliveries
are an amalgamation of
businesses.>>
The exceptional value
of our RAF is that it
allowes formulation of
Business SOA with no
changes in the
specification. When you
go this way as I did, a
statement "the old SOA
aligning with business"
becomes a tautology: any
business organisation is
the old, classic SOA by
definition and
construction, and there
are no exceptions for
centuries from this
rule.
You will be the firts
if you can point to any
business organisation in
the world, which would
not use orientation on
service to its customers
(including the
non-profit and
Government
organisations).
Orientation on serviceis
is one of the
fundamental principles
of capitalism.
@Bob
<<"Netflix
and Amazon for example
structure themselves
around multiple small
teams, each one with
responsibility for a
small part of the
overall system. These
independent teams can
own the whole
lifecycle of the
services they create,
affording them a
greater degree of
autonomy than is
possible for larger
teams with more
monolithic codebases.
These services with
their independent
concerns can change
and evolve separately
from one another,
resulting in the
ability to deliver
changes to production
faster.">>
This exact idea has been
described in my book
"Ladder to SOE" in 2009.
This is about new
service-oriented
organisation of business
with no vertical
command/accountability
subordination lines. In
essence, such
organisation is an
"internal market" (from
that book) that has
shared source of funding
and should be inthe
constant competition
with analogous external
services/providers. The
described model was
about normal regular
services while internals
of those service - what
they were built of - was
immaterial
(microservice,
miniservices,
nanoservices, or
anything else).
<<If
these organizations
had adopted larger
team sizes, the larger
monolithic systems
that would have
emerged would not have
given them the same
ability to experiment,
adapt, and ultimately
keep their customers
happy.”>>
This is only 50% true.
If the <<larger
team sizes>>
would be used as a basic
size they were either
end-up in the
decompositions into the
smaller size services
_or_ nothing wrong would
come from them: a
service/functionality
decomposition is the
natural business process
and it stops when the
smaller (more granular)
functionality does not
make sense for
particular business
model _or_ the current
(a bit less granular)
size is OK because it
serves the needs of the
business model of the
organisation already.
Moreover, saying that
it <<would
not have given them
the same ability to
experiment, adapt, and
ultimately keep their
customers happy>>
is fundamentally wrong:
there is no universal
scale to compare
granularity of servives,
i.e. size of the teams,
because each
organisation has its
optimal smallest size of
'basic' services (and it
depends on the corporate
business model, chosen
corporate strategy and
market dynamics).
Also, I believe that
the author has omited
the second part of the
organisation based on
services: it is
impossible to stay in
the market on the basic
services only - the
services usually need to
be dynamically
re-composed into bigger
less granular
combinations for solving
current market tasks.
These tasks change more
and more dynamically
nowadays and this leads
to creation of dynamic
mid-size service
compositions, which than
composed in the products
for internal and
external consumenrs. As
a result, the middle and
even top level
management appears in a
constant movement where
positions/roles exist
only until corresponding
coarse-granualr
compbination of basic
services is needed to
habdle the market state.
[As you, probably, have
noticed, this
explanation deviates
from IT and filly
replicates SO into
Business. This is what I
write since 2012 and
mean when referring to
COS. ]
Cheers,
- Michael
My first
thought in seeing
“miniservices" was
they’ve found a
way to equivocate
to the worst of
all worlds. There
may be goodness
here but it will
take more than a
Gartner broad
brush to make that
clear.
The
underpinning of
Conway’s Law is
people do what
is familiar,
whether that’s a
go idea or not.
Conway’s Law is
a rationale for
change
throughout an
organization and
not just in
selective spots.
Packaging of
microservices is
an enabler of a
lot of other
stuff folks want
to do. There is
a packaging
component to
structure
development and
delivery.
Microservices
and business
talks a lot
about the old
SOA aligning
with business.
I think this
makes a lot of
sense as long as
you look at
business in the
broadest context
and realize most
deliveries are
an amalgamation
of businesses.
IT
infrastructure
is a legitimate
business in an
organization
delivering
software even if
it doesn’t
directly reflect
the money-making
products the
organization
produces.
Middleware
fails the
money-making
business when it
becomes a
disconnected
business itself.
Until
tomorrow.
Ken
------------------------------------------------------------------------------
Dr.
Kenneth Laskey
MITRE
Corporation,
M/S F510
phone: 703-983-7934
7515
Colshire Drive
fax:
703-983-1379
McLean
VA 22102-7508
Hi
folks,
I
attended a
Gartner
webcast on “12
Core
Principles of
Application
Architecture
for Digital
Business and
IoT” one day
last week and
the speaker
made some
interesting
remarks about
microservices
architecture
(and this
generally
relates to
section 3 of
the
presentation …
I can provide
a copy of the
deck to those
who might
need/want it).
He emphasized
the packaging
construct with
special
reference to
the hyperscale
service
provider class
where the pace
of innovation
requires them
to deliver
“dozens to
hundreds of
times per day”
… he noted
that
“microservices
are all about
agility,
autonomy, and
discipline” in
that context,
at least. He
argued that
not many
organizations
are in the
hyperscale
category and
others should
focus on
“miniservices”,
which resemble
microservices
but with
“limited
autonomy”
support
relative to
microservices
in the
hyperscale
context.
Beyond that,
neither the
voice track
nor the slides
provided
sufficient
detail to
distinguish
the two.
“Organizations
for a few
years now have
understood
this link
between
organizational
structure and
software they
create, and
have been
embracing new
structures in
order to
achieve the
outcome they
want. Netflix
and Amazon for
example
structure
themselves
around
multiple small
teams, each
one with
responsibility
for a small
part of the
overall
system. These
independent
teams can own
the whole
lifecycle of
the services
they create,
affording them
a greater
degree of
autonomy than
is possible
for larger
teams with
more
monolithic
codebases.
These services
with their
independent
concerns can
change and
evolve
separately
from one
another,
resulting in
the ability to
deliver
changes to
production
faster. If
these
organizations
had adopted
larger team
sizes, the
larger
monolithic
systems that
would have
emerged would
not have given
them the same
ability to
experiment,
adapt, and
ultimately
keep their
customers
happy.”
The
article goes
on to note:
“… the
growing
popularity of
Microservices,
which are
increasingly
being adopted
by
organizations
looking to
improve the
autonomy of
their teams
and increase
the speed of
change … these
architectures
allow
organizations
much more
flexibility in
aligning the
architecture
of their
systems to the
structure of
their teams in
order to
ensure that
Conway’s law
works for
you.”
So, I’ve
expanded my
view about the
drivers behind
a shift to
microservices
architecture
to include
not only the
packaging
aspect but its
connection
with
operations and
organizational
structure/communication as well.
Still
need to get
some clarity
on reuse. I
am concerned
that we’ll see
benefits in
the short term
but end up
with an
unwieldy set
of near
duplicates
when there is
no time
invested to
discover if
someone else
has already
solved your
problem,
especially
when their
solution will
be better than
yours.
To what extent
is MS just
packaging and
reuse is you
spin up a copy
of my machine
image so you
get to use my
solution while
avoiding the
remote
networking
overhead? The
challenge is
for me to
notify you
when I
change/update
my image and
then you
staying in
synch with me
to the extent
that updating
is necessary
and provides
you value.
Ken
------------------------------------------------------------------------------
Dr. Kenneth
Laskey
MITRE
Corporation,
M/S F510 phone: 703-983-7934
7515 Colshire
Drive
fax:
703-983-1379
McLean VA
22102-7508
I was formulating a response to Ken’s query _on
behalf of_
a hypothetical
microservices
evangelist …
i.e., adopting
the mindset of
such a person
… in that
context, I
believe that
the definition
I offered
holds.
We can disparage it all we want, but if the hyperscale
players
continue to
optimize for
the packaging
benefits of
the
microservices
model along
the
Agile/DevOps
movement, we
might have to
accept it and
make the best
of it … which
certainly
offers new
opportunities
for SOA and
COS
principles,
designs, etc.
IMHO.
if I do not use any - no one
microservice,
I do not have
an application
do I?
The statement "An application
is a
composition of
one or more
microservices
and other
implementation
mechanisms
that provides
a coherent
grouping of
business
functionality.”That’s
quick n’
dirty... "-
no, is not
dirty, it is
shi..y!
(Pardon my
French)
To you
original
query: “My
biggest
problem is I
have yet to
see a good
definition of
“application”. Is it just the user interface that calls microservices
under the
hood?” …
Taking
the
perspective of
a
microservices
architecture
evangelist,
I’d answer “An
application is
a composition
of one or more
microservices
and other
implementation
mechanisms
that provides
a coherent
grouping of
business
functionality.”
That’s
quick n’ dirty
but conveys
the sense as I
understand (in
the
microservices
context).
and while the focus is really
on OSGi, its
discussion of
microservices
includes the
following:
The
first model is
the microservices
model. With
this model,
components are
defined as
independent
microservices
that any
application
can use. They
also have
stateless
behavior so
they can be
replaced and
scaled as
needed.
Additionally,
they are
independent of
each other and
of
applications
that use them,
so
deployment/redeployment
of a
microservice
doesn't affect
applications
it serves.
But microservices might
be the biggest
revolution in
componentization. A microservice is a logic component deployed in
RESTful form,
designed to be
accessed
through a URL.
Microservices
easily address
issues of
component
dependencies
and avalanches
of
redeployments
due to small
component
changes
because
microservices
are
independent as
long as the
API call
formats are
maintained.
Microservices
won't change
the modularity
of JVM or
provide an
efficient way
of managing
remote-versus-local
components,
but they could
significantly
reduce the
burden
of component
management for
distributed
components.
My biggest problem is I have
yet to see a
good
definition of
“application”.
Is it just
the user
interface that
calls
microservices
under the
hood?
Any favorite (attributable)
definitions of
application?
------------------------------------------------------------------------------
Dr. Kenneth
Laskey
MITRE
Corporation,
M/S F510 phone: 703-983-7934
7515 Colshire
Drive
fax:
703-983-1379
McLean VA
22102-7508
--
Rex Brooks
Starbourne Communications Design
Email: rexb@starbourne.com
GeoAddress:
1361 Addison St. Apt. A
Berkeley, CA 94702
Phone: 510-898-0670
--
Rex Brooks
Starbourne Communications Design
Email: rexb@starbourne.com
GeoAddress:
1361 Addison St. Apt. A
Berkeley, CA 94702
Phone: 510-898-0670
--
Rex Brooks
Starbourne Communications Design
Email: rexb@starbourne.com
GeoAddress:
1361 Addison St. Apt. A
Berkeley, CA 94702
Phone: 510-898-0670
|