[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [cti-users] Java STIX 2.x Libary + Taxii-springboot-bpmn + Taxii-worker
Indeed, major kudos, Stephen! ^_^
Cheers,
Trey
On 25.01.2019 09:00:43, Terry MacDonald wrote:
> Hi Stephen,
>
> This is very cool indeed. Well done!
>
> Cheers
> Terry MacDonald
> Cosive
>
> On Fri, 25 Jan 2019, 08:38 Stephen Russett <stephen@digitalstate.ca wrote:
>
> > Hello all!
> >
> > new update for java based Taxii Server TAXII-springboot-bpmn!
> >
> > Just finished the first draft of the âTAXII-Workerâ,
> > https://github.com/StephenOTT/TAXII-Worker.
> >
> > The TAXII-Worker is a âExternal Task Workerâ, which interacts with the
> > Taxii serverâs workflow engine. Whenever any work needs to be executed,
> > rather than executing on the Taxii server, it is tasked for fetching by the
> > cluster of Workers. This is all based on Vertx so you get clustering,
> > non-blocking, scaling, etc etc.
> >
> > The extra flavour for the worker is, it also can execute on GraalVM and
> > use the polymorphic language support. What is nice about this is it means
> > you can have the automation execute in your language of choice (
> > https://www.graalvm.org/docs/). Example: you can have The STIX JSON be
> > parsed by the OASIS STIX Python lib instead of the STIX-java lib. Or you
> > can have your STIX 1.x json get upgraded to 2.x using The STIX elevator,
> > but when it fails, the Workflow engine will trigger a manual task for human
> > review of the specific STIX object that failed to âelevateâ to 2.x, or if
> > you have some scripts that execute custom manipulations of inbound data,
> > you can easily drop this into the automation without custom standup of new
> > systems. (Such as you can easily pass your data into a Node app and have it
> > return back to the taxii server without have to build any âextrasâ )
> >
> > Another example would be if you wanted to parse data from STIX into some
> > other non-STIX data format. You can use the workflow engine and the Graal
> > execution to convert using your language of choice into the end format of
> > your choice.
> >
> > You can also use this setup to execute work on other systems, such as if a
> > STIX cyber observable is evaluated and determined that some sort of script
> > should be executed as mitigation or prevention. No need for extra layers
> > of systems, you can execute this with ease, and with your language of
> > choice. This also plays well into OpenC2 style of requirements, where the
> > worker becomes a micro app which is the executor, and the workflow engine
> > is the upstream Command system. The actual openC2 spec is just a light
> > json layer on top.
> >
> > Enjoy!
> >
> >
> >
> >
> >
> >
> >
> > From: Stephen Russett <stephen@digitalstate.ca> <stephen@digitalstate.ca>
> > Reply: Stephen Russett <stephen@digitalstate.ca> <stephen@digitalstate.ca>
> > Date: January 15, 2019 at 1:05:35 PM
> > To: cti-users@lists.oasis-open.org <cti-users@lists.oasis-open.org>
> > <cti-users@lists.oasis-open.org>
> > Subject: Re: [cti-users] Java STIX 2.x Libary
> >
> > Hi Everyone
> >
> > Wanted to share a new update for the Java Stix Lib and now the Java
> > *Taxii* lib!
> >
> > So there was the original library we discussed previously:
> > https://github.com/StephenOTT/STIX-Java
> >
> > To complement and enable this library, there is the TAXII library that
> > provides a *Java Springboot implementation of TAXII using the STIX-Java
> > library*.
> >
> > https://github.com/StephenOTT/TAXII-springboot-bpmn
> >
> > The extra flavour with this TAXII server implementation is itâs built-in
> > automation engine with BPMN workflows.
> >
> > Encourage you to look at some example usage and explanation of
> > configurations here:
> > https://github.com/StephenOTT/TAXII-springboot-bpmn#example-usage
> >
> > This gives us a scalable TAXII server that can implement any level of
> > processing rules and configurations through Business and Analyst friendly
> > configurations without having to write new taxii server or post-taxii
> > server/middleware code. automation actions can be executed on the taxii
> > server as Java, Groovy, Ruby, *Python* or javascript. Or the workflows
> > can be configured to use External Task patterns, where external apps /
> > âWorkersâ claim the work, execute, and return the result through http. (so
> > actual execution of actions do not occur on the workflow engine); workers
> > can be written in any language.
> > Another interesting angle we will look to enable in the future is a
> > generic worker using https://www.graalvm.org/docs/ allowing the STIX-Java
> > lib to be used by any other language to provide a type-safe unified
> > interface and execution env for workers.
> >
> > The readme contains some further examples that may interest anyone around
> > the usage of Course of Action automation where you can trigger workflows
> > within the network to action a mitigation or âwhatever you likeâ based on
> > the existence or content of a bundle or any object within the bundle or the
> > STIX storage itself (stream listeners, and events based on DB changes).
> >
> > The Taxii implementation was also setup with multi-tenant configurations
> > allowing the API-ROOTs of Taxii to represent Tenants if you choose to use
> > it that way.
> >
> > Additionally you can use the Taxii Server to harvest from other servers
> > (Taxii or otherwise) that may not have Channels/ pub/sub abilities; this
> > functionality uses the BPMN configurations with Timers allowing you to
> > configure a recurring scape of data and process as you see fit.
> >
> >
> > Enjoy!
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > From: Stephen Russett <stephen@digitalstate.ca> <stephen@digitalstate.ca>
> > Reply: Stephen Russett <stephen@digitalstate.ca> <stephen@digitalstate.ca>
> > Date: December 10, 2018 at 3:53:30 PM
> > To: cti-users@lists.oasis-open.org <cti-users@lists.oasis-open.org>
> > <cti-users@lists.oasis-open.org>
> > Subject: Re: [cti-users] Java STIX 2.x Libary
> >
> > Good afternoon all!
> >
> > Wanted to share a nice update for the Java Stix Library.
> >
> > Rollout of immutables and full JSON Serialization/Deserialization has been
> > completed.
> >
> > As mentioned, you get builds with full spec logic coverage and nice
> > âwith*()â methods allowing you to create a new version of the immutable
> > with the addition N properties. such as:
> >
> > AttackPattern attackPattern123 = AttackPattern.builder()
> > .name("Some Attack Pattern 1")
> > .build()
> >
> > AttackPattern someNewAttackPattern = attackPattern123.withKillChainPhases(KillChainPhase.builder()
> > .killChainName("Some Name")
> > .phaseName("Some Phase")
> > .build())
> >
> >
> > âsomeNewAttackPatternâ is a full copy + the new KillChain information
> > added. All methods match the properties defined in the spec.
> >
> > Here is a more complex example of usage from Generation of objects,
> > generation of JSON, and parsing json back objects. Note how you do not
> > need to know/enforce which type of object you are parsing, the parser will
> > figure it out based on the Type and IDs, validate it, and return the needed
> > info.
> >
> > class BundleSpec extends Specification {
> >
> > def "Basic 'uses' Relationship object and addition to bundle"(){
> > when: "Create a Relationship with Attack Pattern and Malware"
> >
> > Relationship usesRel = Relationship.builder()
> > .relationshipType("uses")
> > .created(Instant.now())
> > .sourceRef(AttackPattern.builder()
> > .name("Some Attack Pattern 1")
> > .build())
> > .targetRef(Malware.builder()
> > .name("dog")
> > .addLabels("worm")
> > .build())
> > .build()
> >
> > then: "print the JSON string version of the created relationship object"
> > println usesRel.toJsonString()
> >
> > then: "parse the string back into a relationship object"
> > BundleableObject parsedRelationship = StixParsers.parseBundleableObject(usesRel.toJsonString())
> > assert parsedRelationship instanceof Relationship
> >
> > Relationship typedRelation = (Relationship)parsedRelationship
> >
> > and: "print the parsed relation"
> > println typedRelation
> >
> > then: "ensure the original JSON matches the new JSON"
> > assert usesRel.toJsonString() == typedRelation.toJsonString()
> >
> > then: "add the relationship into a bundle"
> > Bundle bundle = Bundle.builder()
> > .addObjects(usesRel)
> > .build()
> >
> > and: "print the bundle json"
> > println bundle.toJsonString()
> >
> > then: "parse json bundle back into object"
> > BundleObject parsedBundle = StixParsers.parseBundle(bundle.toJsonString())
> > assert parsedBundle instanceof Bundle
> > Bundle typedBundle = (Bundle)parsedBundle
> >
> > then: "ensure original bundle and parsed bundles match in their json forms"
> > assert bundle.toJsonString() == typedBundle.toJsonString()
> >
> > }
> >
> > def "bundleable object parsing"(){
> > when: "setup parser and object"
> > String attackPatternString = AttackPattern.builder()
> > .name("Some Attack Pattern 1")
> > .build().toJsonString()
> >
> > then: "can parse the json back into a attack Pattern"
> > BundleableObject parsedAttackPatternBo = StixParsers.parseBundleableObject(attackPatternString)
> > assert parsedAttackPatternBo instanceof AttackPattern
> >
> > AttackPattern parsedAttackPattern = (AttackPattern)StixParsers.parseBundleableObject(attackPatternString)
> > println parsedAttackPattern.toJsonString()
> >
> > }
> > }
> >
> >
> >
> >
> > Enjoy!!
> >
> >
> >
> >
> > From: Stephen Russett <stephen@digitalstate.ca> <stephen@digitalstate.ca>
> > Reply: Stephen Russett <stephen@digitalstate.ca> <stephen@digitalstate.ca>
> > Date: December 5, 2018 at 2:33:06 PM
> > To: cti-users@lists.oasis-open.org <cti-users@lists.oasis-open.org>
> > <cti-users@lists.oasis-open.org>
> > Subject: Re: [cti-users] Java STIX 2.x Libary
> >
> > Hey Everyone
> >
> > just little update for the java lib.
> >
> > I have started cleanup refactor now that all of the angles were generally
> > covered.
> > All of the objects have been implemented as Immutable and you get a nice
> > builder interface for everything:
> >
> > when:
> > Relationship derived = Relationship.builder()
> > .relationshipType("derived-from")
> > .sourceRef(AttackPattern.builder()
> > .name("Some Attack Pattern 1")
> > .build())
> > .targetRef(AttackPattern.builder()
> > .name("Some Other Attack Patter 2")
> > .build())
> > .build()
> >
> > Bundle stixBundleObject = Bundle.builder()
> > .addObjects(derived)
> > .build()
> >
> >
> > There are lots of helper methods that come with the builders as well,
> > allowing you to transfer from one Immutable to another. All objects
> > support two modes of Hydration: dehydrated: Type+ID or Hydrated: normal
> > attribute requirements. With dehydrated this is a object that only
> > contains a type and a Id (such as when deserializing a object with a
> > relationship). A interesting factor here is the easy ability extend this
> > to any fields and add additional layers, as its all enforced with
> > validation groups:
> >
> > @NotBlank(groups = {Default.class, ValidateIdOnly.class}, message = "Type is required")
> > @JsonProperty("type")
> > default String getType(){
> > return typeValue();
> > }
> >
> > vs another attribute such as:
> >
> > @NotNull
> > @JsonProperty("created")
> > @Value.Default
> > default Instant getCreated(){
> > return Instant.now();
> > }
> >
> >
> > The example above allows anyone to easily extend or configure hydration or
> > dehydrated attribute requirements with a single groups = {Default.class,
> > ValidateIdOnly.class}
> > which says âenforce this validation for the Default group and the
> > ValidateIdOnlyGroup. When no group is provided the default group
> > enforcement is used.
> >
> > You also get Annotation validators, allowing really easy extension:
> > So consider where Malware SDO reuses the Labels but enforces (or with a
> > âSHOULDâ) of a specific vocabulary:
> >
> > you get to use Annotations!
> >
> > @Override
> > @Vocab(MalwareLabels.class)
> > @NotNull @Size(min = 1, message = "At least one label from malware-label-ov must be used")
> > Set<@Size(min = 1) String> getLabels();
> >
> >
> > @Vocab can be used on Strings and on Sets of Strings. Allowing default
> > and custom vocab usage. There is also @RelationshipTypeLimit
> >
> > @RelationshipTypeLimit(source = MalwareSdo.class, relationshipTypes = {"targets", "uses", "variant-of"})
> >
> > This can be applied on a Class to enforce a Relationshipâs
> > relationshipTypes based on the Source SDO.
> >
> > and then there is @RelationshipLimit:
> >
> > @RelationshipLimit(source = MalwareSdo.class, relationshipType = "targets", target = {IdentitySdo.class, VulnerabilitySdo.class})
> > @RelationshipLimit(source = MalwareSdo.class, relationshipType = "uses", target = {ToolSdo.class})
> > @RelationshipLimit(source = MalwareSdo.class, relationshipType = "variant-of", target = {MalwareSdo.class})
> >
> > or:
> >
> > @RelationshipLimit(source = DomainObject.class, relationshipType = "derived-from", target = {DomainObject.class}, classEquality = true)
> >
> >
> > This allows you to defined Target-ref restrictions for relationship Types
> > and Source SDOs.
> >
> > Enjoy!
> >
> >
> >
> >
> >
> >
> > From: Stephen Russett <stephen@digitalstate.ca> <stephen@digitalstate.ca>
> > Reply: Stephen Russett <stephen@digitalstate.ca> <stephen@digitalstate.ca>
> > Date: November 29, 2018 at 8:48:11 PM
> > To: cti-users@lists.oasis-open.org <cti-users@lists.oasis-open.org>
> > <cti-users@lists.oasis-open.org>
> > Subject: Re: [cti-users] Java STIX 2.x Libary
> >
> > Hey everyone
> >
> > just update on the lib: All of the spec has been added minus the cyber
> > observables and pattern checking support.
> > I am now adding unit tests to ensure full coverage of the spec and
> > everything works well.
> >
> > Lots of helpers as well such as AttackPattern myAP =
> > AttackPattern.parse(someJsonString). .parse() can be used on all objects
> > including a bundle.
> > and the ability to autopopulate the bundle based on relationships: example
> > the attack pattern has a âtargetsâ or a related_to. auto-populate will
> > detect any linked objects and add them to the bundle.
> > In the same light, there is also âauto-hydrationâ, which is used to
> > populate a object with actual pointers to objects rather than just the
> > string of the ID.
> > This has interesting use cases for bundle processing: where each object is
> > processed into a java object and then you âhydrateâ the objects where each
> > java object will look back in the bundle of java objects so it can set the
> > links between objects. This same function can also be used when processing
> > STIX data to allow data enrichment: Allowing someone to receive a bundle
> > that has relationships/Ids but not the objects for those Ids. You can use
> > the hydration hooks to look up the Ids in another system(s) to populate the
> > java object.
> >
> > If anyone has some common issues they run into for stix data quality
> > issues, I would be interested to hear them, so i can cover it in the tests
> > and add helpers to prevent the errors or âauto-correctâ them.
> >
> > Thanks!
> >
> > Steve
> >
> >
> >
> > From: Stephen Russett <stephen@digitalstate.ca> <stephen@digitalstate.ca>
> > Reply: Stephen Russett <stephen@digitalstate.ca> <stephen@digitalstate.ca>
> > Date: November 22, 2018 at 5:38:24 PM
> > To: cti-users@lists.oasis-open.org <cti-users@lists.oasis-open.org>
> > <cti-users@lists.oasis-open.org>
> > Subject: Re: [cti-users] Java STIX 2.x Libary
> >
> > Hi All!
> >
> > I have just added support for relationships with STIX in the Java lib.
> >
> > example
> >
> > ```
> > {
> > "type": "bundle",
> > "id": "bundle--f428f07f-4efe-4980-9f47-74d946826c69",
> > "spec_version": "2.0",
> > "objects": [
> > {
> > "type": "attack-pattern",
> > "id": "attack-pattern--cb3010c8-f36c-4dc3-bf3d-8c1ffeb5e1cf",
> > "created": "2018-11-22T22:26:43.159Z",
> > "modified": "2018-11-25T22:26:43.159Z",
> > "revoked": false,
> > "object_marking_refs": [
> > "marking-definition--6139cfd0-7d2f-4389-b7e3-e97836888268"
> > ],
> > "granular_markings": [
> > {
> > "selectors": [
> > "pattern1",
> > "pattern2",
> > "pattern3"
> > ],
> > "marking_ref":
> > "marking-definition--b4ab8f5b-f812-49c5-a2b2-b168a0ba236d"
> > }
> > ],
> > "name": "some pattern",
> > "kill_chain_phases": [
> > {
> > "kill_chain_name": "Chain1",
> > "phase_name": "phase1"
> > },
> > {
> > "kill_chain_name": "Chain1",
> > "phase_name": "phase2"
> > }
> > ],
> > "x_someCustomKey": "My custom value",
> > "x_someOtherCustom_key": 3939
> > },
> > {
> > "type": "observed-data",
> > "id": "observed-data--e3d14217-fc58-47bb-b5fb-b67d6ca78db3",
> > "created": "2018-11-22T22:26:43.221Z",
> > "modified": "2018-11-22T22:26:43.221Z",
> > "revoked": false,
> > "object_marking_refs": [
> > "marking-definition--39ceb120-7777-4e13-888e-95efb6c99a31"
> > ],
> > "first_observed": "2018-11-22T22:26:43.209Z",
> > "last_observed": "2018-11-22T22:26:43.209Z",
> > "number_observed": 3,
> > "objects": {
> > "some artifact": {
> > "type": "artifact",
> > "url": "someURL"
> > },
> > "some AS": {
> > "type": "autonomous-system",
> > "number": 5,
> > "rir": "someRIR"
> > }
> > }
> > },
> > {
> > "type": "marking-definition",
> > "id": "marking-definition--39ceb120-7777-4e13-888e-95efb6c99a31",
> > "created": "2018-11-22T22:26:43.208Z",
> > "granular_markings": [
> > {
> > "selectors": [
> > "marking-pattern1",
> > "pattern2",
> > "pattern3"
> > ],
> > "marking_ref":
> > "marking-definition--b4ab8f5b-f812-49c5-a2b2-b168a0ba236d"
> > }
> > ],
> > "definition_type": "statement",
> > "definition": {
> > "statement": "Internal review of data allows for sharing as per
> > ABC-009 Standard"
> > }
> > },
> > {
> > "type": "marking-definition",
> > "id": "marking-definition--b4ab8f5b-f812-49c5-a2b2-b168a0ba236d",
> > "created": "2018-11-22T22:26:43.199Z",
> > "definition_type": "tlp",
> > "definition": {
> > "tlp": "red"
> > }
> > },
> > {
> > "type": "relationship",
> > "id": "relationship--fb64d173-3478-4eb9-abae-c580cf92454c",
> > "created": "2018-11-22T22:26:43.242Z",
> > "modified": "2018-11-22T22:26:43.242Z",
> > "revoked": false,
> > "relationship_type": "targets",
> > "source": "attack-pattern--cb3010c8-f36c-4dc3-bf3d-8c1ffeb5e1cf",
> > "target": "identity--16b669a9-91ef-492d-852c-9249695a09f4"
> > }
> > ]
> > }
> > ```
> >
> > I am hoping some of the community can provide some interesting
> > âcomplexitiesâ for relationships for me to test again. If you have some
> > scenarios of complexity please share!
> >
> > All of the relationships are typed checked and validated per SDO. So it
> > should generated fully spec compliant relationships.
> >
> > as a extra helper to maintain sanity I have added the
> > "bundle.autoDetectBundleObjects()â method. This will traverse the nested
> > relationships and objects inside of each objects in the bundle and add any
> > nested content into the bundle at the parent level as per the spec. The
> > âsanityâ aspect is that if you are building a complex object with many
> > relationships and nested objects, you can build these all inline without
> > the need to generate them as individual bundle items. So you could have a
> > âattack-patternâ with 5 ârelated-toâ, 5 object markings, and 5 granular
> > markings. you would just have to write
> > âbundle.addObjects(myAttackPattern)â and run the
> > âbundle.autoDetectBundleObjects()â method, and it will detect all of the
> > nested 15 objects and add them into the bundle as per the spec.
> >
> >
> > Thanks
> >
> >
> >
> > From: Trey Darley <trey@kingfisherops.com> <trey@kingfisherops.com>
> > Reply: Trey Darley <trey@kingfisherops.com> <trey@kingfisherops.com>
> > Date: November 21, 2018 at 5:50:17 PM
> > To: Stephen Russett <stephen@digitalstate.ca> <stephen@digitalstate.ca>
> > Cc: cti-users@lists.oasis-open.org <cti-users@lists.oasis-open.org>
> > <cti-users@lists.oasis-open.org>
> > Subject: Re: [cti-users] Java STIX 2.x Libary
> >
> > On 21.11.2018 12:06:54, Stephen Russett wrote:
> > >
> > > Wanted to share this project I have been working on that provides a
> > > few different CTI management tooling, but importantly for this list:
> > > a STIX 2.x Java library.
> > >
> >
> > Nice work, Stephen! Thank you for your efforts on behalf of the wider
> > CTI community. ^_^
> >
> > --
> > Cheers,
> > Trey Darley
> > Co-chair, CTI TC
> >
> > ++--------------------------------------------------------------------------++
> > mobile: +32/494.766.080
> > gpg fingerprint: 85F3 5F54 4A2A B4CD 33C4 5B9B B30D DD6E 62C8 6C1D
> >
> > ++--------------------------------------------------------------------------++
> > --
> > "When you choose the lesser of two evils, always remember that it is
> > still an evil." --Max Lerner
> >
> >
--
++--------------------------------------------------------------------------++
Kingfisher Operations, sprl
gpg fingerprint: 85F3 5F54 4A2A B4CD 33C4 5B9B B30D DD6E 62C8 6C1D
++--------------------------------------------------------------------------++
--
"Repetition does not transform a lie into a truth." --Franklin Delano
Roosevelt
Attachment:
signature.asc
Description: PGP signature
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]