Simple worklow diagrams and text (Sun's process)

[Kevin Looney <Kevin.T.Looney@Sun.COM>]

Hi Everyone,

     I dug these diagrams / descriptions up, and hacked them a bit to 
look more like how we describe things in TAG.

    These descriptions are wordy - but I expect that we can distill them 
down a bit more.  

    One thing to note, as well as process workflow, this description 
also describes some 'roles' (eg Specification Lead, Test Lead) as well 
as other 'nouns' in the process that aren't directly related to TAG 
(eg.  Test Binding Lists, Coverage Reports, Colored Specifications).

Hope this helps,  regards,
Kevin L

Title: SpecTrac 1.4 Tutorial: workflow

Specification Analysis Workflow

1. General Workflow
2. Original Specification workflow
3. Revision Specification Workflow

General Workflow

figure 1: General workflow for analyzing a specification

Figure 1 describes the overall workflow between a Spec Lead and Test Lead, for the production of a Test Suite based on a specification.

• The Spec Lead starts by producing an original specification.
• This specification gets analyzed [1] by a Test Lead, who produces analysis and tests [2].
• The Test Lead may generate a 'colored spec' (markup) and pass this back to the Spec Lead [3] for further revision.
• At any given time, the specification may be revised, and the spec is passed back to the Test Lead [4], to refine specification analysis and produce further tests [5].

A Test Lead reconciles the analysis and tests of the original specification [2], and revision specification [5]. A Test Lead can also export this analysis [3] to use as a way to communicate the 'quality of a specification and testing' to a Spec Lead

Original Specification Workflow

A Test Lead is given a new specification for initial analysis. This is the context of the original specification workflow.

figure 2: Workflow for analyzing a new specification

To do this analysis, a Test Lead will do the following tasks:

• [1] Identify TAs
• [2] Annotate TAs
• [3] Write Tests / Bind TAs to Tests
• [4] Create Coverage Report
• [5] Export Markup / Coverage
(additional task, if the Test Lead reviews results with the Spec Lead)

One outcome of this workflow is that a Test Lead will have a marked-up version of the spec describing TAs and their meta-data. Test Leads identify Test Cases from the TAs, to produce a new suite of tests for the specification.

A second outcome of this workflow is that a Test Lead will have a clear metric describing how completely a specification has been tested (test coverage, breadth). Coverage reports will describe the percentage of TAs that have a test that tests the specified behavior. Coverage reports can also be configured to look at percentage of coverage within definable partitions of the specification.

Revised Specification Workflow

In this second workflow, a Test Lead is given a revised version of the original specification for analysis. This is the context of the revision specification workflow.

figure 3: Workflow for analyzing a revision specification

To do this revision, a Test Lead will do the following tasks:

• [1]Identify Specification differences
• [1]Resolve Differences
• [3]Identify Test Changes / Update Coverage
• [4]Create Coverage Report
• [5]Export Markup / Coverage
(additional task, if the Test Lead reviews results with the Spec Lead)

One outcome of this workflow is that a Test Lead will have a revision of the specification markup accounting for additional assertions, missing assertions and changed assertions. The Test Lead resolves differences between original TAs and TAs that are in the revised specification. A Test Lead subsequently re-establishes previous TA markup and test bindings. The Test Lead may also create additional meta data for new or original TAs.

A second outcome of this workflow is that a Test Lead will have a revised metric describing how testing coverage has changed, given the specification revisions. The Test Lead will be able to use coverage reports to drive additional test development that addresses the revisions.