[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [cti-users] Java STIX 2.x Libary + Taxii-springboot-bpmn + Taxii-worker
Hello all!new update for java based Taxii Server TAXII-springboot-bpmn!Just finished the first draft of the âTAXII-Workerâ,Âhttps://github.com/StephenOTT/TAXII-Worker.The TAXII-Worker is a âExternal Task Workerâ, which interacts with the Taxii serverâs workflow engine. Whenever any work needs to be executed, rather than executing on the Taxii server, it is tasked for fetching by the cluster of Workers. This is all based on Vertx so you get clustering, non-blocking, scaling, etc etc. ÂThe extra flavour for the worker is, it also can execute on GraalVM and use the polymorphic language support. What is nice about this is it means you can have the automation execute in your language of choice (https://www.graalvm.org/docs/). ÂExample: you can Âhave The STIX JSON be parsed by the OASIS STIX Python lib instead of the STIX-java lib. Or you can have your STIX 1.x json get upgraded to 2.x using The STIX elevator, but when it fails, the Workflow engine will trigger a manual task for human review of the specific STIX object that failed to âelevateâ to 2.x, or if you have some scripts that execute custom manipulations of inbound data, you can easily drop this into the automation without custom standup of new systems. (Such as you can easily pass your data into a Node app and have it return back to the taxii server without have to build any âextrasâ )Another example would be if you wanted to parse data from STIX into some other non-STIX data format. You can use the workflow engine and the Graal execution to convert using your language of choice into the end format of your choice.ÂYou can also use this setup to execute work on other systems, such as if a STIX cyber observable is evaluated and determined that some sort of script should be executed as mitigation or prevention. ÂNo need for extra layers of systems, you can execute this with ease, and with your language of choice. This also plays well into OpenC2 style of requirements, where the worker becomes a micro app which is the executor, and the workflow engine is the upstream Command system. The actual openC2 spec is just a light json layer on top.Enjoy!
From:ÂStephen Russett <stephen@digitalstate.ca>
Reply:ÂStephen Russett <stephen@digitalstate.ca>
Date:ÂJanuary 15, 2019 at 1:05:35 PM
To:Âcti-users@lists.oasis-open.org <cti-users@lists.oasis-open.org>
Subject:Â Re: [cti-users] Java STIX 2.x Libary
Hi EveryoneWanted to share a new update for the Java Stix Lib and now the Java Taxii lib!So there was the original library we discussed previously:Âhttps://github.com/StephenOTT/STIX-JavaTo complement and enable this library, there is the TAXII library that provides a Java Springboot implementation of TAXII using the STIX-Java library.The extra flavour with this TAXII server implementation is itâs built-in automation engine with BPMN workflows.Encourage you to look at some example usage and explanation of configurations here:Âhttps://github.com/StephenOTT/TAXII-springboot-bpmn#example-usageThis gives us a scalable TAXII server that can implement any level of processing rules and configurations through Business and Analyst friendly configurations without having to write new taxii server or post-taxii server/middleware code. Âautomation actions can be executed on the taxii server as Java, Groovy, Ruby, Python or _javascript_. Or the workflows can be configured to use External Task patterns, where external apps / âWorkersâ claim the work, execute, and return the result through http. (so actual execution of actions do not occur on the workflow engine); workers can be written in any language. ÂAnother interesting angle we will look to enable in the future is a generic worker using https://www.graalvm.org/docs/Âallowing the STIX-Java lib to be used by any other language to provide a type-safe unified interface and execution env for workers.The readme contains some further examples that may interest anyone around the usage of Course of Action automation where you can trigger workflows within the network to action a mitigation or âwhatever you likeâ based on the existence or content of a bundle or any object within the bundle or the STIX storage itself (stream listeners, and events based on DB changes).The Taxii implementation was also setup with multi-tenant configurations allowing the API-ROOTs of Taxii to represent Tenants if you choose to use it that way.Additionally you can use the Taxii Server to harvest from other servers (Taxii or otherwise) that may not have Channels/ pub/sub abilities; this functionality uses the BPMN configurations with Timers allowing you to configure a recurring scape of data and process as you see fit.Enjoy!
From:ÂStephen Russett <stephen@digitalstate.ca>
Reply:ÂStephen Russett <stephen@digitalstate.ca>
Date:ÂDecember 10, 2018 at 3:53:30 PM
To:Âcti-users@lists.oasis-open.org <cti-users@lists.oasis-open.org>
Subject:Â Re: [cti-users] Java STIX 2.x Libary
Good afternoon all!Wanted to share a nice update for the Java Stix Library.Rollout of immutables and full JSON Serialization/Deserialization has been completed.As mentioned, you get builds with full spec logic coverage and nice âwith*()â methods allowing you to create a new version of the immutable with the addition N properties. Âsuch as:AttackPattern attackPattern123 = AttackPattern.builder()
.name("Some Attack Pattern 1")
.build()
AttackPattern someNewAttackPattern = attackPattern123.withKillChainPhases(KillChainPhase.builder()
.killChainName("Some Name")
.phaseName("Some Phase")
.build())âsomeNewAttackPatternâ is a full copy + the new KillChain information added. All methods match the properties defined in the spec.Here is a more complex example of usage from Generation of objects, generation of JSON, and parsing json back objects. Note how you do not need to know/enforce which type of object you are parsing, the parser will figure it out based on the Type and IDs, validate it, and return the needed info.class BundleSpec extends Specification {
def "Basic 'uses' Relationship object and addition to bundle"(){
when: "Create a Relationship with Attack Pattern and Malware"
Relationship usesRel = Relationship.builder()
.relationshipType("uses")
.created(Instant.now())
.sourceRef(AttackPattern.builder()
.name("Some Attack Pattern 1")
.build())
.targetRef(Malware.builder()
.name("dog")
.addLabels("worm")
.build())
.build()
then: "print the JSON string version of the created relationship object"
println usesRel.toJsonString()
then: "parse the string back into a relationship object"
BundleableObject parsedRelationship = StixParsers.parseBundleableObject(usesRel.toJsonString())
assert parsedRelationship instanceof Relationship
Relationship typedRelation = (Relationship)parsedRelationship
and: "print the parsed relation"
println typedRelation
then: "ensure the original JSON matches the new JSON"
assert usesRel.toJsonString() == typedRelation.toJsonString()
then: "add the relationship into a bundle"
Bundle bundle = Bundle.builder()
.addObjects(usesRel)
.build()
and: "print the bundle json"
println bundle.toJsonString()
then: "parse json bundle back into object"
BundleObject parsedBundle = StixParsers.parseBundle(bundle.toJsonString())
assert parsedBundle instanceof Bundle
Bundle typedBundle = (Bundle)parsedBundle
then: "ensure original bundle and parsed bundles match in their json forms"
assert bundle.toJsonString() == typedBundle.toJsonString()
}
def "bundleable object parsing"(){
when: "setup parser and object"
String attackPatternString = AttackPattern.builder()
.name("Some Attack Pattern 1")
.build().toJsonString()
then: "can parse the json back into a attack Pattern"
BundleableObject parsedAttackPatternBo = StixParsers.parseBundleableObject(attackPatternString)
assert parsedAttackPatternBo instanceof AttackPattern
AttackPattern parsedAttackPattern = (AttackPattern)StixParsers.parseBundleableObject(attackPatternString)
println parsedAttackPattern.toJsonString()
}
}Enjoy!!
From:ÂStephen Russett <stephen@digitalstate.ca>
Reply:ÂStephen Russett <stephen@digitalstate.ca>
Date:ÂDecember 5, 2018 at 2:33:06 PM
To:Âcti-users@lists.oasis-open.org <cti-users@lists.oasis-open.org>
Subject:Â Re: [cti-users] Java STIX 2.x Libary
Hey Everyonejust little update for the java lib.I have started cleanup refactor now that all of the angles were generally covered.All of the objects have been implemented as Immutable and you get a nice builder interface for everything:when:
Relationship derived = Relationship.builder()
.relationshipType("derived-from")
.sourceRef(AttackPattern.builder()
.name("Some Attack Pattern 1")
.build())
.targetRef(AttackPattern.builder()
.name("Some Other Attack Patter 2")
.build())
.build()
Bundle stixBundleObject = Bundle.builder()
.addObjects(derived)
.build()There are lots of helper methods that come with the builders as well, allowing you to transfer from one Immutable to another. All objects support two modes of Hydration: dehydrated: Type+ID or Hydrated: normal attribute requirements. With dehydrated this is a object that only contains a type and a Id (such as when deserializing a object with a relationship). A interesting factor here is the easy ability extend this to any fields and add additional layers, as its all enforced with validation groups:Â@NotBlank(groups = {Default.class, ValidateIdOnly.class}, message = "Type is required")
@JsonProperty("type")
default String getType(){
return typeValue();
}vs another attribute such as:@NotNull
@JsonProperty("created")
@Value.Default
default Instant getCreated(){
return Instant.now();
}The example above allows anyone to easily extend or configure hydration or dehydrated attribute requirements with a singleÂgroups = {Default.class, ValidateIdOnly.class}Âwhich says âenforce this validation for the Default group and the ValidateIdOnlyGroup. When no group is provided the default group enforcement is used.You also get Annotation validators, allowing really easy extension:So consider where Malware SDO reuses the Labels but enforces (or with a âSHOULDâ) of a specific vocabulary:you get to use Annotations!@Override
@Vocab(MalwareLabels.class)
@NotNull @Size(min = 1, message = "At least one label from malware-label-ov must be used")
Set<@Size(min = 1) String> getLabels();@Vocab can be used on Strings and on Sets of Strings. Allowing default and custom vocab usage. There is also Â@RelationshipTypeLimitThis can be applied on a Class to enforce a Relationshipâs relationshipTypes based on the Source SDO.@RelationshipTypeLimit(source = MalwareSdo.class, relationshipTypes = {"targets", "uses", "variant-of"})and then there is @RelationshipLimit:@RelationshipLimit(source = MalwareSdo.class, relationshipType = "targets", target = {IdentitySdo.class, VulnerabilitySdo.class})
@RelationshipLimit(source = MalwareSdo.class, relationshipType = "uses", target = {ToolSdo.class})
@RelationshipLimit(source = MalwareSdo.class, relationshipType = "variant-of", target = {MalwareSdo.class})or:Â@RelationshipLimit(source = DomainObject.class, relationshipType = "derived-from", target = {DomainObject.class}, classEquality = true)This allows you to defined Target-ref restrictions for relationship Types and Source SDOs.Enjoy!
From:ÂStephen Russett <stephen@digitalstate.ca>
Reply:ÂStephen Russett <stephen@digitalstate.ca>
Date:ÂNovember 29, 2018 at 8:48:11 PM
To:Âcti-users@lists.oasis-open.org <cti-users@lists.oasis-open.org>
Subject:Â Re: [cti-users] Java STIX 2.x Libary
Hey everyonejust update on the lib: All of the spec has been added minus the cyber observables and pattern checking support.I am now adding unit tests to ensure full coverage of the spec and everything works well.Lots of helpers as well such as AttackPattern myAP = AttackPattern.parse(someJsonString). Â.parse() can be used on all objects including a bundle.and the ability to autopopulate the bundle based on relationships: example the attack pattern has a âtargetsâ or a related_to. Âauto-populate will detect any linked objects and add them to the bundle.In the same light, there is also âauto-hydrationâ, which is used to populate a object with actual pointers to objects rather than just the string of the ID.This has interesting use cases for bundle processing: where each object is processed into a java object and then you âhydrateâ the objects where each java object will look back in the bundle of java objects so it can set the links between objects. This same function can also be used when processing STIX data to allow data enrichment: Allowing someone to receive a bundle that has relationships/Ids but not the objects for those Ids. You can use the hydration hooks to look up the Ids in another system(s) to populate the java object.If anyone has some common issues they run into for stix data quality issues, I would be interested to hear them, so i can cover it in the tests and add helpers to prevent the errors or âauto-correctâ them.Thanks!SteveFrom:ÂStephen Russett <stephen@digitalstate.ca>
Reply:ÂStephen Russett <stephen@digitalstate.ca>
Date:ÂNovember 22, 2018 at 5:38:24 PM
To:Âcti-users@lists.oasis-open.org <cti-users@lists.oasis-open.org>
Subject:Â Re: [cti-users] Java STIX 2.x Libary
Hi All!I have just added support for relationships with STIX in the Java lib.example```{ "type": "bundle", "id": "bundle--f428f07f-4efe-4980-9f47-74d946826c69", "spec_version": "2.0", "objects": [  {   "type": "attack-pattern",   "id": "attack-pattern--cb3010c8-f36c-4dc3-bf3d-8c1ffeb5e1cf",   "created": "2018-11-22T22:26:43.159Z",   "modified": "2018-11-25T22:26:43.159Z",   "revoked": false,   "object_marking_refs": [    "marking-definition--6139cfd0-7d2f-4389-b7e3-e97836888268"   ],   "granular_markings": [    {     "selectors": [      "pattern1",      "pattern2",      "pattern3"     ],     "marking_ref": "marking-definition--b4ab8f5b-f812-49c5-a2b2-b168a0ba236d"    }   ],   "name": "some pattern",   "kill_chain_phases": [    {     "kill_chain_name": "Chain1",     "phase_name": "phase1"    },    {     "kill_chain_name": "Chain1",     "phase_name": "phase2"    }   ],   "x_someCustomKey": "My custom value",   "x_someOtherCustom_key": 3939  },  {   "type": "observed-data",   "id": "observed-data--e3d14217-fc58-47bb-b5fb-b67d6ca78db3",   "created": "2018-11-22T22:26:43.221Z",   "modified": "2018-11-22T22:26:43.221Z",   "revoked": false,   "object_marking_refs": [    "marking-definition--39ceb120-7777-4e13-888e-95efb6c99a31"   ],   "first_observed": "2018-11-22T22:26:43.209Z",   "last_observed": "2018-11-22T22:26:43.209Z",   "number_observed": 3,   "objects": {    "some artifact": {     "type": "artifact",     "url": "someURL"    },    "some AS": {     "type": "autonomous-system",     "number": 5,     "rir": "someRIR"    }   }  },  {   "type": "marking-definition",   "id": "marking-definition--39ceb120-7777-4e13-888e-95efb6c99a31",   "created": "2018-11-22T22:26:43.208Z",   "granular_markings": [    {     "selectors": [      "marking-pattern1",      "pattern2",      "pattern3"     ],     "marking_ref": "marking-definition--b4ab8f5b-f812-49c5-a2b2-b168a0ba236d"    }   ],   "definition_type": "statement",   "definition": {    "statement": "Internal review of data allows for sharing as per ABC-009 Standard"   }  },  {   "type": "marking-definition",   "id": "marking-definition--b4ab8f5b-f812-49c5-a2b2-b168a0ba236d",   "created": "2018-11-22T22:26:43.199Z",   "definition_type": "tlp",   "definition": {    "tlp": "red"   }  },  {   "type": "relationship",   "id": "relationship--fb64d173-3478-4eb9-abae-c580cf92454c",   "created": "2018-11-22T22:26:43.242Z",   "modified": "2018-11-22T22:26:43.242Z",   "revoked": false,   "relationship_type": "targets",   "source": "attack-pattern--cb3010c8-f36c-4dc3-bf3d-8c1ffeb5e1cf",   "target": "identity--16b669a9-91ef-492d-852c-9249695a09f4"  } ]}```I am hoping some of the community can provide some interesting âcomplexitiesâ for relationships for me to test again. ÂIf you have some scenarios of complexity please share!All of the relationships are typed checked and validated per SDO. So it should generated fully spec compliant relationships.as a extra helper to maintain sanity I have added the "bundle.autoDetectBundleObjects()â method. This will traverse the nested relationships and objects inside of each objects in the bundle and add any nested content into the bundle at the parent level as per the spec. The âsanityâ aspect is that if you are building a complex object with many relationships and nested objects, you can build these all inline without the need to generate them as individual bundle items. So you could have a âattack-patternâ with 5 ârelated-toâ, 5 object markings, and 5 granular markings. Âyou would just have to write âbundle.addObjects(myAttackPattern)â and run the âbundle.autoDetectBundleObjects()â method, and it will detect all of the nested 15 objects and add them into the bundle as per the spec.Thanks
From:ÂTrey Darley <trey@kingfisherops.com>
Reply:ÂTrey Darley <trey@kingfisherops.com>
Date:ÂNovember 21, 2018 at 5:50:17 PM
To:ÂStephen Russett <stephen@digitalstate.ca>
Cc:Âcti-users@lists.oasis-open.org <cti-users@lists.oasis-open.org>
Subject:Â Re: [cti-users] Java STIX 2.x Libary
On 21.11.2018 12:06:54, Stephen Russett wrote:
>
> Wanted to share this project I have been working on that provides a
> few different CTI management tooling, but importantly for this list:
> a STIX 2.x Java library.
>
Nice work, Stephen! Thank you for your efforts on behalf of the wider
CTI community. ^_^
--
Cheers,
Trey Darley
Co-chair, CTI TC
++--------------------------------------------------------------------------++
mobile: +32/494.766.080
gpg fingerprint: 85F3 5F54 4A2A B4CD 33C4 5B9B B30D DD6E 62C8 6C1D
++--------------------------------------------------------------------------++
--
"When you choose the lesser of two evils, always remember that it is
still an evil." --Max Lerner
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]