[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [cti-users] Java STIX 2.x Libary + Taxii-springboot-bpmn + Taxii-worker
Indeed, major kudos, Stephen! ^_^ Cheers, Trey On 25.01.2019 09:00:43, Terry MacDonald wrote: > Hi Stephen, > > This is very cool indeed. Well done! > > Cheers > Terry MacDonald > Cosive > > On Fri, 25 Jan 2019, 08:38 Stephen Russett <stephen@digitalstate.ca wrote: > > > Hello all! > > > > new update for java based Taxii Server TAXII-springboot-bpmn! > > > > Just finished the first draft of the âTAXII-Workerâ, > > https://github.com/StephenOTT/TAXII-Worker. > > > > The TAXII-Worker is a âExternal Task Workerâ, which interacts with the > > Taxii serverâs workflow engine. Whenever any work needs to be executed, > > rather than executing on the Taxii server, it is tasked for fetching by the > > cluster of Workers. This is all based on Vertx so you get clustering, > > non-blocking, scaling, etc etc. > > > > The extra flavour for the worker is, it also can execute on GraalVM and > > use the polymorphic language support. What is nice about this is it means > > you can have the automation execute in your language of choice ( > > https://www.graalvm.org/docs/). Example: you can have The STIX JSON be > > parsed by the OASIS STIX Python lib instead of the STIX-java lib. Or you > > can have your STIX 1.x json get upgraded to 2.x using The STIX elevator, > > but when it fails, the Workflow engine will trigger a manual task for human > > review of the specific STIX object that failed to âelevateâ to 2.x, or if > > you have some scripts that execute custom manipulations of inbound data, > > you can easily drop this into the automation without custom standup of new > > systems. (Such as you can easily pass your data into a Node app and have it > > return back to the taxii server without have to build any âextrasâ ) > > > > Another example would be if you wanted to parse data from STIX into some > > other non-STIX data format. You can use the workflow engine and the Graal > > execution to convert using your language of choice into the end format of > > your choice. > > > > You can also use this setup to execute work on other systems, such as if a > > STIX cyber observable is evaluated and determined that some sort of script > > should be executed as mitigation or prevention. No need for extra layers > > of systems, you can execute this with ease, and with your language of > > choice. This also plays well into OpenC2 style of requirements, where the > > worker becomes a micro app which is the executor, and the workflow engine > > is the upstream Command system. The actual openC2 spec is just a light > > json layer on top. > > > > Enjoy! > > > > > > > > > > > > > > > > From: Stephen Russett <stephen@digitalstate.ca> <stephen@digitalstate.ca> > > Reply: Stephen Russett <stephen@digitalstate.ca> <stephen@digitalstate.ca> > > Date: January 15, 2019 at 1:05:35 PM > > To: cti-users@lists.oasis-open.org <cti-users@lists.oasis-open.org> > > <cti-users@lists.oasis-open.org> > > Subject: Re: [cti-users] Java STIX 2.x Libary > > > > Hi Everyone > > > > Wanted to share a new update for the Java Stix Lib and now the Java > > *Taxii* lib! > > > > So there was the original library we discussed previously: > > https://github.com/StephenOTT/STIX-Java > > > > To complement and enable this library, there is the TAXII library that > > provides a *Java Springboot implementation of TAXII using the STIX-Java > > library*. > > > > https://github.com/StephenOTT/TAXII-springboot-bpmn > > > > The extra flavour with this TAXII server implementation is itâs built-in > > automation engine with BPMN workflows. > > > > Encourage you to look at some example usage and explanation of > > configurations here: > > https://github.com/StephenOTT/TAXII-springboot-bpmn#example-usage > > > > This gives us a scalable TAXII server that can implement any level of > > processing rules and configurations through Business and Analyst friendly > > configurations without having to write new taxii server or post-taxii > > server/middleware code. automation actions can be executed on the taxii > > server as Java, Groovy, Ruby, *Python* or javascript. Or the workflows > > can be configured to use External Task patterns, where external apps / > > âWorkersâ claim the work, execute, and return the result through http. (so > > actual execution of actions do not occur on the workflow engine); workers > > can be written in any language. > > Another interesting angle we will look to enable in the future is a > > generic worker using https://www.graalvm.org/docs/ allowing the STIX-Java > > lib to be used by any other language to provide a type-safe unified > > interface and execution env for workers. > > > > The readme contains some further examples that may interest anyone around > > the usage of Course of Action automation where you can trigger workflows > > within the network to action a mitigation or âwhatever you likeâ based on > > the existence or content of a bundle or any object within the bundle or the > > STIX storage itself (stream listeners, and events based on DB changes). > > > > The Taxii implementation was also setup with multi-tenant configurations > > allowing the API-ROOTs of Taxii to represent Tenants if you choose to use > > it that way. > > > > Additionally you can use the Taxii Server to harvest from other servers > > (Taxii or otherwise) that may not have Channels/ pub/sub abilities; this > > functionality uses the BPMN configurations with Timers allowing you to > > configure a recurring scape of data and process as you see fit. > > > > > > Enjoy! > > > > > > > > > > > > > > > > > > > > From: Stephen Russett <stephen@digitalstate.ca> <stephen@digitalstate.ca> > > Reply: Stephen Russett <stephen@digitalstate.ca> <stephen@digitalstate.ca> > > Date: December 10, 2018 at 3:53:30 PM > > To: cti-users@lists.oasis-open.org <cti-users@lists.oasis-open.org> > > <cti-users@lists.oasis-open.org> > > Subject: Re: [cti-users] Java STIX 2.x Libary > > > > Good afternoon all! > > > > Wanted to share a nice update for the Java Stix Library. > > > > Rollout of immutables and full JSON Serialization/Deserialization has been > > completed. > > > > As mentioned, you get builds with full spec logic coverage and nice > > âwith*()â methods allowing you to create a new version of the immutable > > with the addition N properties. such as: > > > > AttackPattern attackPattern123 = AttackPattern.builder() > > .name("Some Attack Pattern 1") > > .build() > > > > AttackPattern someNewAttackPattern = attackPattern123.withKillChainPhases(KillChainPhase.builder() > > .killChainName("Some Name") > > .phaseName("Some Phase") > > .build()) > > > > > > âsomeNewAttackPatternâ is a full copy + the new KillChain information > > added. All methods match the properties defined in the spec. > > > > Here is a more complex example of usage from Generation of objects, > > generation of JSON, and parsing json back objects. Note how you do not > > need to know/enforce which type of object you are parsing, the parser will > > figure it out based on the Type and IDs, validate it, and return the needed > > info. > > > > class BundleSpec extends Specification { > > > > def "Basic 'uses' Relationship object and addition to bundle"(){ > > when: "Create a Relationship with Attack Pattern and Malware" > > > > Relationship usesRel = Relationship.builder() > > .relationshipType("uses") > > .created(Instant.now()) > > .sourceRef(AttackPattern.builder() > > .name("Some Attack Pattern 1") > > .build()) > > .targetRef(Malware.builder() > > .name("dog") > > .addLabels("worm") > > .build()) > > .build() > > > > then: "print the JSON string version of the created relationship object" > > println usesRel.toJsonString() > > > > then: "parse the string back into a relationship object" > > BundleableObject parsedRelationship = StixParsers.parseBundleableObject(usesRel.toJsonString()) > > assert parsedRelationship instanceof Relationship > > > > Relationship typedRelation = (Relationship)parsedRelationship > > > > and: "print the parsed relation" > > println typedRelation > > > > then: "ensure the original JSON matches the new JSON" > > assert usesRel.toJsonString() == typedRelation.toJsonString() > > > > then: "add the relationship into a bundle" > > Bundle bundle = Bundle.builder() > > .addObjects(usesRel) > > .build() > > > > and: "print the bundle json" > > println bundle.toJsonString() > > > > then: "parse json bundle back into object" > > BundleObject parsedBundle = StixParsers.parseBundle(bundle.toJsonString()) > > assert parsedBundle instanceof Bundle > > Bundle typedBundle = (Bundle)parsedBundle > > > > then: "ensure original bundle and parsed bundles match in their json forms" > > assert bundle.toJsonString() == typedBundle.toJsonString() > > > > } > > > > def "bundleable object parsing"(){ > > when: "setup parser and object" > > String attackPatternString = AttackPattern.builder() > > .name("Some Attack Pattern 1") > > .build().toJsonString() > > > > then: "can parse the json back into a attack Pattern" > > BundleableObject parsedAttackPatternBo = StixParsers.parseBundleableObject(attackPatternString) > > assert parsedAttackPatternBo instanceof AttackPattern > > > > AttackPattern parsedAttackPattern = (AttackPattern)StixParsers.parseBundleableObject(attackPatternString) > > println parsedAttackPattern.toJsonString() > > > > } > > } > > > > > > > > > > Enjoy!! > > > > > > > > > > From: Stephen Russett <stephen@digitalstate.ca> <stephen@digitalstate.ca> > > Reply: Stephen Russett <stephen@digitalstate.ca> <stephen@digitalstate.ca> > > Date: December 5, 2018 at 2:33:06 PM > > To: cti-users@lists.oasis-open.org <cti-users@lists.oasis-open.org> > > <cti-users@lists.oasis-open.org> > > Subject: Re: [cti-users] Java STIX 2.x Libary > > > > Hey Everyone > > > > just little update for the java lib. > > > > I have started cleanup refactor now that all of the angles were generally > > covered. > > All of the objects have been implemented as Immutable and you get a nice > > builder interface for everything: > > > > when: > > Relationship derived = Relationship.builder() > > .relationshipType("derived-from") > > .sourceRef(AttackPattern.builder() > > .name("Some Attack Pattern 1") > > .build()) > > .targetRef(AttackPattern.builder() > > .name("Some Other Attack Patter 2") > > .build()) > > .build() > > > > Bundle stixBundleObject = Bundle.builder() > > .addObjects(derived) > > .build() > > > > > > There are lots of helper methods that come with the builders as well, > > allowing you to transfer from one Immutable to another. All objects > > support two modes of Hydration: dehydrated: Type+ID or Hydrated: normal > > attribute requirements. With dehydrated this is a object that only > > contains a type and a Id (such as when deserializing a object with a > > relationship). A interesting factor here is the easy ability extend this > > to any fields and add additional layers, as its all enforced with > > validation groups: > > > > @NotBlank(groups = {Default.class, ValidateIdOnly.class}, message = "Type is required") > > @JsonProperty("type") > > default String getType(){ > > return typeValue(); > > } > > > > vs another attribute such as: > > > > @NotNull > > @JsonProperty("created") > > @Value.Default > > default Instant getCreated(){ > > return Instant.now(); > > } > > > > > > The example above allows anyone to easily extend or configure hydration or > > dehydrated attribute requirements with a single groups = {Default.class, > > ValidateIdOnly.class} > > which says âenforce this validation for the Default group and the > > ValidateIdOnlyGroup. When no group is provided the default group > > enforcement is used. > > > > You also get Annotation validators, allowing really easy extension: > > So consider where Malware SDO reuses the Labels but enforces (or with a > > âSHOULDâ) of a specific vocabulary: > > > > you get to use Annotations! > > > > @Override > > @Vocab(MalwareLabels.class) > > @NotNull @Size(min = 1, message = "At least one label from malware-label-ov must be used") > > Set<@Size(min = 1) String> getLabels(); > > > > > > @Vocab can be used on Strings and on Sets of Strings. Allowing default > > and custom vocab usage. There is also @RelationshipTypeLimit > > > > @RelationshipTypeLimit(source = MalwareSdo.class, relationshipTypes = {"targets", "uses", "variant-of"}) > > > > This can be applied on a Class to enforce a Relationshipâs > > relationshipTypes based on the Source SDO. > > > > and then there is @RelationshipLimit: > > > > @RelationshipLimit(source = MalwareSdo.class, relationshipType = "targets", target = {IdentitySdo.class, VulnerabilitySdo.class}) > > @RelationshipLimit(source = MalwareSdo.class, relationshipType = "uses", target = {ToolSdo.class}) > > @RelationshipLimit(source = MalwareSdo.class, relationshipType = "variant-of", target = {MalwareSdo.class}) > > > > or: > > > > @RelationshipLimit(source = DomainObject.class, relationshipType = "derived-from", target = {DomainObject.class}, classEquality = true) > > > > > > This allows you to defined Target-ref restrictions for relationship Types > > and Source SDOs. > > > > Enjoy! > > > > > > > > > > > > > > From: Stephen Russett <stephen@digitalstate.ca> <stephen@digitalstate.ca> > > Reply: Stephen Russett <stephen@digitalstate.ca> <stephen@digitalstate.ca> > > Date: November 29, 2018 at 8:48:11 PM > > To: cti-users@lists.oasis-open.org <cti-users@lists.oasis-open.org> > > <cti-users@lists.oasis-open.org> > > Subject: Re: [cti-users] Java STIX 2.x Libary > > > > Hey everyone > > > > just update on the lib: All of the spec has been added minus the cyber > > observables and pattern checking support. > > I am now adding unit tests to ensure full coverage of the spec and > > everything works well. > > > > Lots of helpers as well such as AttackPattern myAP = > > AttackPattern.parse(someJsonString). .parse() can be used on all objects > > including a bundle. > > and the ability to autopopulate the bundle based on relationships: example > > the attack pattern has a âtargetsâ or a related_to. auto-populate will > > detect any linked objects and add them to the bundle. > > In the same light, there is also âauto-hydrationâ, which is used to > > populate a object with actual pointers to objects rather than just the > > string of the ID. > > This has interesting use cases for bundle processing: where each object is > > processed into a java object and then you âhydrateâ the objects where each > > java object will look back in the bundle of java objects so it can set the > > links between objects. This same function can also be used when processing > > STIX data to allow data enrichment: Allowing someone to receive a bundle > > that has relationships/Ids but not the objects for those Ids. You can use > > the hydration hooks to look up the Ids in another system(s) to populate the > > java object. > > > > If anyone has some common issues they run into for stix data quality > > issues, I would be interested to hear them, so i can cover it in the tests > > and add helpers to prevent the errors or âauto-correctâ them. > > > > Thanks! > > > > Steve > > > > > > > > From: Stephen Russett <stephen@digitalstate.ca> <stephen@digitalstate.ca> > > Reply: Stephen Russett <stephen@digitalstate.ca> <stephen@digitalstate.ca> > > Date: November 22, 2018 at 5:38:24 PM > > To: cti-users@lists.oasis-open.org <cti-users@lists.oasis-open.org> > > <cti-users@lists.oasis-open.org> > > Subject: Re: [cti-users] Java STIX 2.x Libary > > > > Hi All! > > > > I have just added support for relationships with STIX in the Java lib. > > > > example > > > > ``` > > { > > "type": "bundle", > > "id": "bundle--f428f07f-4efe-4980-9f47-74d946826c69", > > "spec_version": "2.0", > > "objects": [ > > { > > "type": "attack-pattern", > > "id": "attack-pattern--cb3010c8-f36c-4dc3-bf3d-8c1ffeb5e1cf", > > "created": "2018-11-22T22:26:43.159Z", > > "modified": "2018-11-25T22:26:43.159Z", > > "revoked": false, > > "object_marking_refs": [ > > "marking-definition--6139cfd0-7d2f-4389-b7e3-e97836888268" > > ], > > "granular_markings": [ > > { > > "selectors": [ > > "pattern1", > > "pattern2", > > "pattern3" > > ], > > "marking_ref": > > "marking-definition--b4ab8f5b-f812-49c5-a2b2-b168a0ba236d" > > } > > ], > > "name": "some pattern", > > "kill_chain_phases": [ > > { > > "kill_chain_name": "Chain1", > > "phase_name": "phase1" > > }, > > { > > "kill_chain_name": "Chain1", > > "phase_name": "phase2" > > } > > ], > > "x_someCustomKey": "My custom value", > > "x_someOtherCustom_key": 3939 > > }, > > { > > "type": "observed-data", > > "id": "observed-data--e3d14217-fc58-47bb-b5fb-b67d6ca78db3", > > "created": "2018-11-22T22:26:43.221Z", > > "modified": "2018-11-22T22:26:43.221Z", > > "revoked": false, > > "object_marking_refs": [ > > "marking-definition--39ceb120-7777-4e13-888e-95efb6c99a31" > > ], > > "first_observed": "2018-11-22T22:26:43.209Z", > > "last_observed": "2018-11-22T22:26:43.209Z", > > "number_observed": 3, > > "objects": { > > "some artifact": { > > "type": "artifact", > > "url": "someURL" > > }, > > "some AS": { > > "type": "autonomous-system", > > "number": 5, > > "rir": "someRIR" > > } > > } > > }, > > { > > "type": "marking-definition", > > "id": "marking-definition--39ceb120-7777-4e13-888e-95efb6c99a31", > > "created": "2018-11-22T22:26:43.208Z", > > "granular_markings": [ > > { > > "selectors": [ > > "marking-pattern1", > > "pattern2", > > "pattern3" > > ], > > "marking_ref": > > "marking-definition--b4ab8f5b-f812-49c5-a2b2-b168a0ba236d" > > } > > ], > > "definition_type": "statement", > > "definition": { > > "statement": "Internal review of data allows for sharing as per > > ABC-009 Standard" > > } > > }, > > { > > "type": "marking-definition", > > "id": "marking-definition--b4ab8f5b-f812-49c5-a2b2-b168a0ba236d", > > "created": "2018-11-22T22:26:43.199Z", > > "definition_type": "tlp", > > "definition": { > > "tlp": "red" > > } > > }, > > { > > "type": "relationship", > > "id": "relationship--fb64d173-3478-4eb9-abae-c580cf92454c", > > "created": "2018-11-22T22:26:43.242Z", > > "modified": "2018-11-22T22:26:43.242Z", > > "revoked": false, > > "relationship_type": "targets", > > "source": "attack-pattern--cb3010c8-f36c-4dc3-bf3d-8c1ffeb5e1cf", > > "target": "identity--16b669a9-91ef-492d-852c-9249695a09f4" > > } > > ] > > } > > ``` > > > > I am hoping some of the community can provide some interesting > > âcomplexitiesâ for relationships for me to test again. If you have some > > scenarios of complexity please share! > > > > All of the relationships are typed checked and validated per SDO. So it > > should generated fully spec compliant relationships. > > > > as a extra helper to maintain sanity I have added the > > "bundle.autoDetectBundleObjects()â method. This will traverse the nested > > relationships and objects inside of each objects in the bundle and add any > > nested content into the bundle at the parent level as per the spec. The > > âsanityâ aspect is that if you are building a complex object with many > > relationships and nested objects, you can build these all inline without > > the need to generate them as individual bundle items. So you could have a > > âattack-patternâ with 5 ârelated-toâ, 5 object markings, and 5 granular > > markings. you would just have to write > > âbundle.addObjects(myAttackPattern)â and run the > > âbundle.autoDetectBundleObjects()â method, and it will detect all of the > > nested 15 objects and add them into the bundle as per the spec. > > > > > > Thanks > > > > > > > > From: Trey Darley <trey@kingfisherops.com> <trey@kingfisherops.com> > > Reply: Trey Darley <trey@kingfisherops.com> <trey@kingfisherops.com> > > Date: November 21, 2018 at 5:50:17 PM > > To: Stephen Russett <stephen@digitalstate.ca> <stephen@digitalstate.ca> > > Cc: cti-users@lists.oasis-open.org <cti-users@lists.oasis-open.org> > > <cti-users@lists.oasis-open.org> > > Subject: Re: [cti-users] Java STIX 2.x Libary > > > > On 21.11.2018 12:06:54, Stephen Russett wrote: > > > > > > Wanted to share this project I have been working on that provides a > > > few different CTI management tooling, but importantly for this list: > > > a STIX 2.x Java library. > > > > > > > Nice work, Stephen! Thank you for your efforts on behalf of the wider > > CTI community. ^_^ > > > > -- > > Cheers, > > Trey Darley > > Co-chair, CTI TC > > > > ++--------------------------------------------------------------------------++ > > mobile: +32/494.766.080 > > gpg fingerprint: 85F3 5F54 4A2A B4CD 33C4 5B9B B30D DD6E 62C8 6C1D > > > > ++--------------------------------------------------------------------------++ > > -- > > "When you choose the lesser of two evils, always remember that it is > > still an evil." --Max Lerner > > > > -- ++--------------------------------------------------------------------------++ Kingfisher Operations, sprl gpg fingerprint: 85F3 5F54 4A2A B4CD 33C4 5B9B B30D DD6E 62C8 6C1D ++--------------------------------------------------------------------------++ -- "Repetition does not transform a lie into a truth." --Franklin Delano Roosevelt
Attachment:
signature.asc
Description: PGP signature
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]