I think you guys have raised some good points and questions here that relate more generally to data validation and workflows:
- Where should CybOX data validation occur? Should it happen at the instance generation level or at the consumption level, or perhaps both?
- If the former, should it be accomplished through the schema/data model (not likely with JSON as Jerome pointed out), or through a library?
- If the latter, does it even make sense to have any kind of data validation at the instance generation level?
- What are the consumer-oriented workflows around IP address data?
My thoughts:
- This probably deserves its own discussion thread, so these are just some quick thoughts. Right now validation occurs through XML schema at the instance level, though because there’s no strong type validation (all CybOX Object
fields are fundamentally strings), the onus is still on consumers to validate CybOX instance data. Going to JSON won’t change this paradigm, which seems OK to me, as long as this expectation is clearly communicated. However, in our specifications we can help
consumers understand how to validate instance data, e.g., by specifying corresponding regular expressions (such as for IPv4 addresses). I don’t think expecting our python libraries to validate all kinds of CybOX data is realistic, as this places an additional
burden on our codebase.
- This probably a large list, so these are just a few examples off the top of my head:
- Filtering/parsing, potentially validating, and looking up the IP address data in a database to see if it
has been before
- Filtering/parsing, potentially validating, and enriching the IP address data with other data (e.g., geolocation)
- Filtering/parsing, potentially validating, and sending off the IP address data to another tool/appliance
(e.g., IDS)
I could see an explicit version/type string (either as a separate field or as part of the field name) being useful in the filtering step, for instance,
if you only care about IPv4 addresses and want to ignore anything IPv6. A simple string comparison operation should be less computationally expensive than ipv4/ipv6 validation with a regex (for instance).
Regards,
Ivan
From: Jerome Athias
Date: Thursday, October 29, 2015 at 10:31 AM
To: John Anderson
Cc: Sean Barnum, John Wunder, Mark Davidson, Trey Darley, Terry MacDonald, Bret Jordan, Ivan Kirillov, " cti-cybox@lists.oasis-open.org"
Subject: Re: [cti-cybox] CybOX 3.0: Address Object Refactoring
XSD prevents that
And you can have regex in XSD (feedback loop of data errors: send a malformed xml to my TAXII server)
(That's why the question how do u do in Json?)
Like Sean, I strongly discourage trying to mix the ip addresses. You're trying to define (just few) elements of an Asset's composition
On Thursday, 29 October 2015, John Anderson < janderson@soltra.com> wrote:
The bonus point was for identifying the problem with this data:
Do you see it?
Now, what if a Producer sends that to an automated system? If that system is not already doing input validation, you get KABOOM!
One lessen that the Bad Guys have taught us is that you always validate input. Since we are already validating it, we will always know what type of IP address it is--regardless of what the Producer has said. Therefore, we do not need to mark the type
of IP Address.
The only thing we get by allowing a Producer-authored "type" value is this: We know when the Producer gave us bad data. But, who cares? My system is probably just going to throw that away. I have yet to see a good feedback loop in TAXII/STIX/CybOX for telling
the Producer about data errors.
Or, perhaps, we could have the CybOX libraries validate the "type" value and raise an Exception when it's wrong. But, again, the libraries will have to determine the type of IP Address from the data format. So, I don't see any gain, but rather a lot of lost
computing cycles.
JSA
I agree.
While the format could be distinguished from the string without explicit labeling this puts the onus on the consumer to do that analysis on the data field. This seems like an unnecessary burden for use cases where the field may be used in processing, orchestration
or workflow routing decisions.
sean
What’s the upside of not requiring the producer to specify the version? If it’s distinguishable by format, coudn’t the producer just do that and always provide the format?
I still hold that IP addresses are distinguishable by format alone. However, I acknowledge that it might be nice to know that the value was "assessed" by the producer (however, whether that's really useful is debatable).
Hence, may I suggest:
Generic IP Address field:
{
"ip_address": "127.0.0.1"
}
or, if the producer knows the version:
{
"ip_address": {"127.0.0.1": "ipv4"}
}
or, if that much dynamism gives you heartburn:
{
"ip_address": {"value": "127.0.0.1", "type": "ipv4"}
}
Then, for things with multiple IP Addresses:
{
"inbound_ip_address": "1.2.3.4",
"outbound_ip_address": "2001:0db8:0a0b:12f0::0001",
"another_ip_address": {"127.1.1.1": "ipv4"},
"ip_address_list": [
"5.6.7.8",
{"9.8.7.6": "ipv6"}
]
}
Now, for the white elephant: Are we moving to JSON, officially? Or should we be discussing this with XML examples instead?
JSA
PS-Bonus points! Did you spot the data inconsistency? Do you see the problem that it exposes?
________________________________________
From:
cti-cybox@lists.oasis-open.org < cti-cybox@lists.oasis-open.org> on behalf of Davidson II, Mark S < mdavidson@mitre.org>
Sent: Thursday, October 29, 2015 9:26 AM
To: Trey Darley; Terry MacDonald
Cc: Jordan, Bret; Kirillov, Ivan A.;
cti-cybox@lists.oasis-open.org
Subject: RE: [cti-cybox] CybOX 3.0: Address Object Refactoring
I like this form also.
As to representing a host with multiple network interfaces, I wonder if that's a slightly different discussion. In that case, would you have multiple IP address objects with a relationship to e.g., a host object?
-Mark
-----Original Message-----
From:
cti-cybox@lists.oasis-open.org [ mailto:cti-cybox@lists.oasis-open.org] On Behalf Of Trey Darley
Sent: Thursday, October 29, 2015 5:22 AM
To: Terry MacDonald < terry@soltra.com>
Cc: Jordan, Bret < bret.jordan@bluecoat.com>; Kirillov, Ivan A. < ikirillov@mitre.org>;
cti-cybox@lists.oasis-open.org
Subject: Re: [cti-cybox] CybOX 3.0: Address Object Refactoring
On 28.10.2015 22:27:02, Terry MacDonald wrote:
>
> I probably prefer this one:
>
> {
> "ipv4Address": "128.25.213.19",
> "ipv6Address": "fe80::3e07:54ff:fe6c:6d13"
> }
>
+1
--
Cheers,
Trey
--
Trey Darley
Senior Security Engineer
4DAA 0A88 34BC 27C9 FD2B A97E D3C6 5C74 0FB7 E430
Soltra | An FS-ISAC & DTCC Company
www.soltra.com
--
"Good, Fast, Cheap: Pick any two (you can't have all three)." --RFC 1925
|