ࡱ > @ E bjbjFF "x , , : X X X n B6 B6 B6 B6 | 6 L S 7 ,7 ,7 ,7 ,7 8 " )8 58 R R R R R R R $ T R W T R @9 8 8 @9 @9 R ,7 ,7 R A A A @9 ,7 ,7 R A @9 R A A O h P ,7
7 h} B6 = L P R R 0 S ,P gW @A gW P gW P =8 " _8 A w8 8 =8 =8 =8 R R $ ~ A ~ UIMA Behavioral Metadata Proposal
June 20, 2007
Adam Lally, David Ferrucci
Introduction
The Behavioral Metadata of an analytic declaratively describes what the analytic does; for example, what types of CASs it can process, what elements in a CAS it analyzes, and what sorts of effects it may have on CAS contents as a result of its application.
In this document we elaborate on, clarify, and extend the original proposal for UIMA Behavioral Metadata in the research report [1].
We state the Goals of the Behavioral Metadata and then consider what elements of the Behavioral Metadata would satisfy these goals. Based on these elements we present a preliminary proposal for an XML representation and begin to define the mapping from this XML representation to a formal language such as OCL.
Goals
1. Composition: Support composition either by a human developer or an automated process.
a. Analytics should be able to declare what they do in enough detail to assist manual and/or automated processes in considering their role in an application or in the composition of aggregate analytics.
b. Through their Behavioral Metadata, Analytics should be able to declare enough detail as to enable an application or aggregate to detect invalid compositions/workflows (e.g., a workflow where it can be determined that one of the Analytics preconditions can never be satisfied by the preceding Analytic).
2. Efficiency: Facilitate efficient sharing of CAS content among cooperating analytics. If analytics declare which elements of the CAS (e.g., views) they need to receive and which elements they do not need to receive, the CAS can be filtered or split prior to sending it to target analytics, to achieve transport and parallelization efficiencies respectively.
3. Discovery: Enable both human developers and automated processes to search a repository and locate components that provide a particular function (i.e., works on certain input, produces certain output)
NOTE: The draft specification lists another goal: "We should provide a mechanism for the caller of an Analytic to direct that Analytic to operate on a subset of the CAS. For example, an Analytic's Behavioral Metadata says it can analyze any object of type TextDocument (that is, TextDocument is the type of Sofa that it accepts). The CAS may contain multiple TextDocuments. The caller may want to direct the analytic to process only one of them." We plan to move this to the Abstract Interfaces. The idea is that the caller of an analytic should be able to direct the analytic to the element of the CAS the caller wants the analytic to "analyze" - this ultimately should be subsumed by the analytics 'analyzes' predicate else the Behavioral Metadata would be violated.
Elements of Behavioral Metadata
We take each of three goals in turn and consider what elements of the Behavioral Metadata would address these goals.
Composition
For composition, it is sufficient to express the behavioral metadata of an analytic simply as a precondition and a postcondition.
A Precondition is a predicate that qualifies CASs that the analytic considers valid input. More precisely the analytic's behavior would be considered unspecified for any CAS that did not satisfy the pre-condition. The pre-condition may be used by a framework or application to filter or skip CASs routed to an analytic whose pre-condition is not satisfied by the CASs. A human assembler or automated composition process can interpret the pre-conditions to determine if the analytic is suitable for playing a role in some aggregate composition.
A Postcondition is a predicate that is declared to be true of any CAS after having been processed by the analytic, assuming that the CAS satisfied the precondition when it was input to the analytic.
For example, if the pre-condition requires that valid input CASs contain People, Places and Organizations, but the Postconditions of the previously run Analytic asserts that the CAS will not contain all of these objects, then the composition is clearly invalid.
Efficiency
For the goal of efficiency we require that the Behavioral Metadata enable analytics to declare which objects in the CAS the analytic will consider to perform its function. The implication is that objects not in this set could be filtered in order to optimize transport.
This is not addressed by preconditions or postconditions. Instead we define a third type of condition called a projection condition.
A Projection Condition is a predicate that is evaluated over a CAS and which evaluates to a subset of the objects in the CAS. This is the set of objects that the Analytic declares that it will consider to perform its function.
Discovery
The three types of formal predicates, postcondition, precondition, and projection condition, are sufficient for the runtime to compute/validate compositions and to provide optimized CAS transport. However, they are not likely to be particularly intuitive to read or write.
For the goal of discovery, as well as to provide an easier model for the analytic developer, we propose a breakdown of the Behavioral Metadata into the following categories. We will later define how to generate formal preconditions, postconditions, and projection conditions from these expressions.
Analyzes: Identifies the subjects of analysis (sofas) that the analytic can analyze (that is, the objects it intends to produce annotations over).
Required Inputs: Identifies the objects which must be present in the CAS for the analytic to perform its analysis.
Optional Inputs: Identifies the objects which are not required in the CAS, but which the analytic might inspect while performing its analysis, if they were present.
Creates: Identifies objects that an analytic may create as a result of its analysis. For example, an analytic may declare it creates instances of type Organization with their sector feature equal to "Financial".
Modifies: Identifies objects and or slots that an analytic may modify.
Deletes: Identifies objects that an analytic may delete.
XML Representation
There are multiple levels of expressivity that we might want the XML representation to allow. Below we present examples at several different levels. There are two dimensions to this. One dimension is the level of specificity that is allowed Types, Features, or Constraints over Feature Values. A separate dimension is whether the concept of Views is used.
We initially consider the breakdown along the first dimension:
Level of Specificity
Types Only
This means that the analytic analyzes a Sofa of type TextDocument, requires objects of type Person, and will inspect objects of type Place if they are present. It may create objects of type At.
Types and Features
For the modifies predicate it might make sense in some cases to allow an analytic to declare which features it modifies. I think this is the only predicate where you might want to do this.
Constraints over Feature Values
The next step of complexity is to allow constraints on the values of features. For example, the analyzes predicate might specify objects of type Sofa with feature mimeTypeMajor = text, and inspect objects of type Person with feature age >= 18. At this point we might consider falling back on actual OCL expressions. For example:
select(s | s.oclKindOf(ex::Sofa) and s.mimeTypeMajor = "text")/>
select(p | p.oclKindOf(ex::Person) and p.age >= 18)"/>
Also, for the modifies predicate we might need some way to specify a constraint on the object after the modification has taken place, as well as before.
Views
An issue with the above examples is the lack of any relationship of the Sofa to the Annotations. Consider the example in Section REF _Ref169668839 \r \h 3.1.1 (an Analytic that analyzes a TextDocument, requires as input Person and Place annotation, and creates At annotations). Does the analytic require that the Person and Place annotations refer to a TextDocument Sofa? Will the At annotations that it creates refer to a TextDocument, and will it be the same TextDocument as the Persons and Places? Things become completely unclear for analytics that works with multiple Sofas.
One way to remove this ambiguity is to allow the behavioral specification to refer to a View, where a View collects all annotations referring to a particular Sofa.
ex:text:Token
ex:text:Person
ex:audio:SpeakerBoundary
ex:audio:Person
This example can process either a TextDocument Sofa or a RawAudio Sofa (or both we may need a notion of required vs. optional views). It has different input and output types for the different Sofas.
We also need a way for an analytic to create a View along with an associated Sofa and annotations. For example, this Analytic transcribes audio to text, and also outputs Person annotations over that text:
ex:audio:SpeakerBoundary
ex:audio:Person
ex:text:Person
Deriving Preconditions, Postconditions, and Projection Conditions
To give a formal meaning to the analyzes, required inputs, optional inputs, creates, modifies, and deletes expressions, we will define how these map into formal preconditions, postconditions, and projection conditions. In this section we give informal suggestions for how this would be done. The actual specification should formally define this as transformations from the XML Behavioral Metadata representation to OCL expressions.
Example:
As noted previously there is no difference between the analyzes and requiredInputs declarations for this purpose. Informally, they would generate a Precondition: "The input CAS must contain at least one instance of ex:TextDocument and at least one instance of ex:Person."
The creates declaration would generate a Postcondition: "For each object O in the output CAS, it must be the case that either O was present in the input CAS, or O is of type ex:At."
NOTE: this impacts the "constant IDs" issue from the Abstract Interfaces discussion because that example Postcondition would be difficult or perhaps impossible to evaluate without constant IDs. Without constant IDs there is not a clear definition for "O was present in the input CAS".
Along these same lines:
A "deletes" predicate translates into a postcondition that for each object O in the input CAS, if O does not satisfy the "deletes" predicate, then O is present in the output CAS.
A "modifies" predicate translates into a postcondition that for each object O in the input CAS, if O does not satisfy the "modifies" predicate, and if O is present in the output CAS (i.e. it was not deleted), then O has the same values for all of its slots.
For views, we add the additional constraint that objects are members of that View (and therefore annotations refer to the Views sofa). For example:
ex:text:Token
Translates into a precondition that the input CAS must contain a view V where V.sofa is of type TextDocument and V.members contains at least one object of type Token.
Conclusion
We propose that at a formal level, behavioral metadata can be viewed as consisting of three parts: a precondition, a postcondition, and a projection condition. These would be specified in a language such as OCL.
While analytic developers could write such conditions manually, this would be inconvenient and would be difficult to use for discovery. Therefore we propose the following more intuitive behavioral metadata elements: analyzes, requiredInputs, optionalInputs, creates, modifies, and deletes. These could possibly be further organized under views. Analytic developers would express their components behavioral metadata in this form, and the UIMA specification would define the precise mapping from this format to the formal preconditions, postconditions, and projection conditions in OCL.
Open Issues
Flow Controllers. Do we attempt to model the behavioral metadata of Flow Controllers? The spec element currently claims that Flow Controller behavioral metadata is limited to a precondition and in inspects predicate (since we have agreed that Flow Controllers cannot modify the CAS), and that we do not attempt to describe a Flow Controllers behavior (how it makes its routing decision) in any declarative way.
CAS Multipliers. This section contains no mention of how behavioral metadata relates to CAS Multipliers. Actions such as modify and delete dont seem to apply to the output CASes of a CAS Multiplier. Perhaps only a postcondition is useful in this case? This needs more thought.
Sets of Capabilities (box on pg. 46). Should we allow an analytic to declare multiple (precondition, capability, postcondition) triples? That is, if a Precondition1 is satisfied then the analytic can take one action, whereas if Precondition2 is satisfied the analytic can perform a different action. Historically Apache UIMA has allowed this but it has never gotten much use. The draft spec mentions some ambiguities that this raises, such as what happens if a CAS satisfies more than one precondition.
References
Ferrucci et al., Towards an Interoperability Standard for Text and Multi-modal Analytics. IBM Research Report RC24122. November 28, 2006.
For this example consider that the commonly used capability declaration that an analytic creates type X (and only type X) can be recast as a formal postcondition stating that if the CAS did not contain objects of type Y != X prior to its being processed by this analytic, then the CAS will not contain objects of type Y after being processed by this analytic.
There might be a better name than projection condition. The name was taken from database terminology, the idea being that a precondition might be thought of as a selection of CASes from a set of CASes, and from each such CAS the projection condition forms a new sub-CAS that can be thought of as a projection of the original CAS.
It is an open question whether we need an analyzes predicate that is different than the required inputs predicate. Functionally, there is not much difference between them, but the distinction might be useful for discovery.
! " J K X w |
c
d
. E Q S U i |
ᵭ|qi| h+_ CJ aJ h+- 6CJ ]aJ h CJ aJ h+- 56CJ aJ hvL CJ aJ h`# CJ aJ h+- h+- 56CJ aJ h+- CJ aJ h+- h &