[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [cti-cybox] CybOX 3.0: File Object Refactoring
Just a heads up that we’ve updated the File Object Refactoring proposal [1] to take into account some of the great points brought up around file metadata and masquerading discussion during our last SC call. Let us know what you think.
FileMetadataExtension
A default extension point for capturing general classes of file metadata. A sub-class of the
FileMismatchEnum
Regards,
Ivan
From: <cti-cybox@lists.oasis-open.org> on behalf of Jason Keirstead <Jason.Keirstead@ca.ibm.com>
Date: Tuesday, December 15, 2015 at 11:16 AM To: Ivan Kirillov <ikirillov@mitre.org> Cc: John Wunder <jwunder@mitre.org>, Jerome Athias <athiasjerome@gmail.com>, Patrick Maroney <Pmaroney@specere.org>, Sean Barnum <sbarnum@mitre.org>, John Anderson <janderson@soltra.com>, Paul Patrick <ppatrick@isightpartners.com>, Bret Jordan <bret.jordan@bluecoat.com>, "cti-cybox@lists.oasis-open.org" <cti-cybox@lists.oasis-open.org>, Terry MacDonald <terry@soltra.com> Subject: Re: [cti-cybox] CybOX 3.0: File Object Refactoring I tend to agree.. I am trying to figure out who is setting "is masqueraded" and under what circumstances and what the use case is for this field. I think “is_packed” will likely be included in the next release, perhaps as part of the metadata extension. I’m on the fence as far as explicitly capturing that a file is masqueraded – as John pointed out below, this can already be done implicitly by capturing the file name/extension and MIME type. If the two do not match, then it is likely a case of masquerading. I know that we have this property in the existing File Object, but I’ve always considered it a product of analysis rather than pure observation. IMO, we should leave analytical findings to other places where they make more sense (probably STIX), and leave CybOX to “just the facts”. Regards, Ivan From: John Wunder <jwunder@mitre.org> Date: Tuesday, December 15, 2015 at 9:48 AM To: Jerome Athias <athiasjerome@gmail.com> Cc: Patrick Maroney <Pmaroney@specere.org>, Jason Keirstead <Jason.Keirstead@ca.ibm.com>, Sean Barnum <sbarnum@mitre.org>, Ivan Kirillov <ikirillov@mitre.org>, John Anderson <janderson@soltra.com>, Paul Patrick <ppatrick@isightpartners.com>, Bret Jordan <bret.jordan@bluecoat.com>, "cti-cybox@lists.oasis-open.org" <cti-cybox@lists.oasis-open.org>, Terry MacDonald <terry@soltra.com> Subject: Re: [cti-cybox] CybOX 3.0: File Object Refactoring Yep, that’s one mechanism. Are there others? I’m just suggesting that we should be precise and encode these specific mechanisms rather than some generic field. In the case of a mis-named file, I think we’re all set (the content type will not match the extension in the file name) so maybe there’s nothing to add. From: Jerome Athias <athiasjerome@gmail.com> Date: Tuesday, December 15, 2015 at 11:44 AM To: "Wunder, John A." <jwunder@mitre.org> Cc: Patrick Maroney <Pmaroney@specere.org>, Jason Keirstead <Jason.Keirstead@ca.ibm.com>, Sean Barnum <sbarnum@mitre.org>, Ivan Kirillov <ikirillov@mitre.org>, John Anderson <janderson@soltra.com>, Paul Patrick <ppatrick@isightpartners.com>, "Jordan, Bret" <bret.jordan@bluecoat.com>, "cti-cybox@lists.oasis-open.org" <cti-cybox@lists.oasis-open.org>, Terry MacDonald <terry@soltra.com> Subject: Re: [cti-cybox] CybOX 3.0: File Object Refactoring e.g. (maybe not applicable use case): I am uploading a nicepic.jpg file to a website via an upload form accepting only the extension .jpg But, I renamed webshell.php to picture.jpg just before uploading My .php pretended to be a .jpg 2015-12-15 19:41 GMT+03:00 Wunder, John A. <jwunder@mitre.org>:
It seems like we should capture those specific mechanisms rather than a general-purpose field for “this file claims to be X”. I would guess the majority are via the extension, which is already covered in the name. From: <cti-cybox@lists.oasis-open.org> on behalf of Patrick Maroney <Pmaroney@Specere.org> Date: Tuesday, December 15, 2015 at 11:36 AM To: Jason Keirstead <Jason.Keirstead@ca.ibm.com>, Jerome Athias <athiasjerome@gmail.com> Cc: Sean Barnum <sbarnum@mitre.org>, Ivan Kirillov <ikirillov@mitre.org>, John Anderson <janderson@soltra.com>, Paul Patrick <ppatrick@isightpartners.com>, "Jordan, Bret" <bret.jordan@bluecoat.com>, "cti-cybox@lists.oasis-open.org" <cti-cybox@lists.oasis-open.org>, Terry MacDonald <terry@soltra.com> Subject: Re: [cti-cybox] CybOX 3.0: File Object Refactoring Quick comment - I did not see any reference to masquerading: distinguishing between what a file/content actually "is" vs. what "it" claims to be. Both forms need to be represented unambiguously. Patrick Maroney Office: (856)983-0001 Cell: (609)841-5104 President Integrated Networking Technologies, Inc. PO Box 569 Marlton, NJ 08053 From: <cti-cybox@lists.oasis-open.org> on behalf of Jason Keirstead <Jason.Keirstead@ca.ibm.com> Date: Tuesday, December 15, 2015 at 11:31 AM To: Jerome Athias <athiasjerome@gmail.com> Cc: Sean Barnum <sbarnum@mitre.org>, Ivan Kirillov <ikirillov@mitre.org>, John Anderson <janderson@soltra.com>, Paul Patrick <ppatrick@isightpartners.com>, Bret Jordan <bret.jordan@bluecoat.com>, "cti-cybox@lists.oasis-open.org" <cti-cybox@lists.oasis-open.org>, Terry MacDonald <terry@soltra.com> Subject: Re: [cti-cybox] CybOX 3.0: File Object Refactoring I think yes the IANA mime types as a baseline is a must. Controlled Vocabulary based on IANA? https://www.iana.org/assignments/media-types/media-types.xhtml 2015-12-15 18:34 GMT+03:00 Barnum, Sean D. <sbarnum@mitre.org>:
+1 for explicit is_directory property sean From: <cti-cybox@lists.oasis-open.org> on behalf of Steve Cell <ikirillov@mitre.org> Date: Tuesday, December 15, 2015 at 10:23 AM To: John Anderson <janderson@soltra.com>, Jerome Athias <athiasjerome@gmail.com>, "ppatrick@isightpartners.com" <ppatrick@isightpartners.com> Cc: Jason Keirstead <Jason.Keirstead@ca.ibm.com>, "Jordan, Bret" <bret.jordan@bluecoat.com>, "cti-cybox@lists.oasis-open.org" <cti-cybox@lists.oasis-open.org>, Terry MacDonald <terry@soltra.com> Subject: Re: [cti-cybox] CybOX 3.0: File Object Refactoring +1 for MIMEType as well – I think this would be semantically less ambiguous than “content type”, and so it would be my preference. This would likely be a property that we would add into the default “file metadata” extension; I’ll update the proposal accordingly. There are likely other properties that would fit in here as well – things like entropy. Is there a sense in the community as far as other common file metadata related properties we should be including? As far as characterizing directories, as mentioned in the writeup below, the current plan is allow for this through the use of the file_path field without the file_name field. E.g, the following would be a directory: { "file_system_properties":{"file_path": {"delimiter":"\\", "components":["C:","windows"]}} } This goes along with the notion, as Mark pointed out, that files and directories are treated the same in many languages and also operating systems. However, Paul has a good point that this is less explicit than having a separate directory object. We’ve thought about this in the past and the discussion has always come back to the fact that directories are usually analogous to files in most places, just not in Windows. Therefore, perhaps what we can do is: "file_system_properties":{“is_directory": True, "file_path": {"delimiter":"\\", "components":["C:","windows"]}} } What do you think? Regards, Ivan From: John Anderson <janderson@soltra.com> Date: Tuesday, December 15, 2015 at 7:09 AM To: Jerome Athias <athiasjerome@gmail.com>, Paul Patrick <ppatrick@isightpartners.com> Cc: Jason Keirstead <Jason.Keirstead@ca.ibm.com>, Ivan Kirillov <ikirillov@mitre.org>, Bret Jordan <bret.jordan@bluecoat.com>, "cti-cybox@lists.oasis-open.org" <cti-cybox@lists.oasis-open.org>, Terry MacDonald <terry@soltra.com> Subject: Re: [cti-cybox] CybOX 3.0: File Object Refactoring +1 for having an attribute that holds a MIME Type value. (And maybe "mimetype" is the right attribute name.) Random use-case: An executable that has a ".txt" extension is still executable on Linux, if the right bits are set. If the MIME type is known, then that might make it easier for automated systems to pay attention.
From: Jerome Athias <athiasjerome@gmail.com> Sent: Tuesday, December 15, 2015 8:28 AM To: Paul Patrick Cc: Jason Keirstead; Kirillov, Ivan A.; Jordan, Bret; cti-cybox@lists.oasis-open.org; John Anderson; Terry MacDonald Subject: Re: [cti-cybox] CybOX 3.0: File Object Refactoring MIMEType is used in Malware Metadata Exchange Format (MMDEF), which is used in MAEC Ref. https://standards.ieee.org/develop/indconn/icsg/mmdef.html
2015-12-15 16:19 GMT+03:00 Paul Patrick <ppatrick@isightpartners.com>: Interesting thought. When thinking about this from a malware analysis point of view, having a content_type attribute would actually be very useful. Paul Patrick From: <cti-cybox@lists.oasis-open.org> on behalf of Jason Keirstead <Jason.Keirstead@ca.ibm.com> Date: Tuesday, December 15, 2015 at 8:00 AM To: Ivan Kirillov <ikirillov@mitre.org> Cc: "Jordan, Bret" <bret.jordan@bluecoat.com>, "cti-cybox@lists.oasis-open.org" <cti-cybox@lists.oasis-open.org>, John Anderson <janderson@soltra.com>, Terry MacDonald <terry@soltra.com> Subject: Re: [cti-cybox] CybOX 3.0: File Object Refactoring
It is not safe (nor always possible) to assume the content type of a file based on it's 3 letter extension. By including content_type, we would allow query mechanisms operating over cybox to make a pattern that matches any zip file. Jason Keirstead Product Architect, Security Intelligence, IBM Security Systems www.ibm.com/security | www.securityintelligence.com Without data, all you are is just another person with an opinion - Unknown "Kirillov, Ivan A." ---12/14/2015 06:26:31 PM---We’ve updated the proposal to take into account the new file_name field: Field Type Multiplicit From: "Kirillov, Ivan A." <ikirillov@mitre.org> To: Jason Keirstead/CanEast/IBM@IBMCA Cc: "Jordan, Bret" <bret.jordan@bluecoat.com>, "cti-cybox@lists.oasis-open.org" <cti-cybox@lists.oasis-open.org>, "John Anderson" <janderson@soltra.com>, Terry MacDonald <terry@soltra.com> Date: 12/14/2015 06:26 PM Subject: Re: [cti-cybox] CybOX 3.0: File Object Refactoring Sent by: <cti-cybox@lists.oasis-open.org> We’ve updated the proposal to take into account the new file_name field:
Thanks, Ivan From: <cti-cybox@lists.oasis-open.org> on behalf of Ivan Kirillov <ikirillov@mitre.org> Date: Friday, November 20, 2015 at 7:38 AM To: Jason Keirstead <Jason.Keirstead@ca.ibm.com> Cc: Bret Jordan <bret.jordan@bluecoat.com>, "cti-cybox@lists.oasis-open.org" <cti-cybox@lists.oasis-open.org>, John Anderson <janderson@soltra.com>, Terry MacDonald <terry@soltra.com> Subject: Re: [cti-cybox] CybOX 3.0: File Object Refactoring Thanks Jason and John! Trey and I will chat about this and update the proposal accordingly. Regards, Ivan From: Jason Keirstead Date: Friday, November 20, 2015 at 9:01 AM To: Ivan Kirillov Cc: Bret Jordan, "cti-cybox@lists.oasis-open.org", John Anderson, Terry MacDonald Subject: Re: [cti-cybox] CybOX 3.0: File Object Refactoring I agree with this and IMO it's a good approach. Definitely agree that searching by hash/fuzzy hash is a common and useful practice, and the latter is why SSDEEP is one of the standard values in the HashTypeEnum. Going back to Jason’s point, tokenizing a file name/path string means that you need to employ some extra logic to understand where the file name actually resides, so that’s well taken. I think we were going back and forth as to whether we should have a separate field for file name vs. file path, and wanted to avoid the headaches associated with the current object :) However, I can see the utility in patterning and querying around separate path vs. file name fields, especially in terms of having clear semantics. For instance, a pattern such as: file.file_name.equals(“f00bar.dll”) Is clearly written against only the file name. Whereas the following has the potential of matching against a directory with that name: file.file_path.contains(“f00bar.dll”) Anyhow, if we do go down this road, this is probably the most I’d want to split the file name/path related fields. Also, unlike the current File Object, file_path would ONLY hold the path element, and would not be permitted to encompass the file name as well. Does this seem reasonable? Regards, Ivan From: <cti-cybox@lists.oasis-open.org> on behalf of Bret Jordan Date: Thursday, November 19, 2015 at 5:06 PM To: Terry MacDonald Cc: Jason Keirstead, John Anderson, "cti-cybox@lists.oasis-open.org", Ivan Kirillov Subject: Re: [cti-cybox] CybOX 3.0: File Object Refactoring From our experience searching by file hash and a fuzzy hash is really valuable. It is also nice when you can say things like show me a fuzzy hashes that are 80% similar. Thanks, Bret Bret Jordan CISSP Director of Security Architecture and Standards | Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050 "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."
As most malware randomizes names upon each drop, we’ll most likely be searching for it with an MD5/SHA1/SHA256 Hash, or even a ‘fuzzy hash’ of some kind (e.g. ssdeep or any other Context Trigger Piecewise Hashing program) rather than File Name. Fuzzy hashes are increasingly important as more malware supports polymorphism. IMHO File Name will be used but won’t be as useful as hashes. CTPH original paper for those interested. http://dfrws.org/2006/proceedings/12-Kornblum.pdf Cheers Terry MacDonald Senior STIX Subject Matter Expert SOLTRA | An FS-ISAC and DTCC Company +61 (407) 203 206 | terry@soltra.com From: cti-cybox@lists.oasis-open.org [mailto:cti-cybox@lists.oasis-open.org] On Behalf Of Jason Keirstead Sent: Friday, 20 November 2015 7:07 AM To: John Anderson <janderson@soltra.com> Cc: cti-cybox@lists.oasis-open.org; Kirillov, Ivan A. <ikirillov@mitre.org> Subject: Re: [cti-cybox] CybOX 3.0: File Object Refactoring :) File globs are friendlier than regex. <image002.png> Some examples: https://github.com/cyberdelia/django-pipeline/issues/208 From: cti-cybox@lists.oasis-open.org <cti-cybox@lists.oasis-open.org> on behalf of Jason Keirstead <Jason.Keirstead@ca.ibm.com> Sent: Thursday, November 19, 2015 2:53 PM To: Kirillov, Ivan A. Cc: cti-cybox@lists.oasis-open.org Subject: Re: [cti-cybox] CybOX 3.0: File Object Refactoring - regex searching is extremely expensive over large amounts of data, so we should try to avoid the need for software to do it during design if possible. - I was more referring to this optional part of the proposal
FileName
- Jason Keirstead Product Architect, Security Intelligence, IBM Security Systems www.ibm.com/security | www.securityintelligence.com Without data, all you are is just another person with an opinion - Unknown "Kirillov, Ivan A." ---11/19/2015 03:45:06 PM---That’s a fair point, Jason – my only counter-argument is that most queries such as these can easily From: "Kirillov, Ivan A." <ikirillov@mitre.org> To: Jason Keirstead/CanEast/IBM@IBMCA Cc: "cti-cybox@lists.oasis-open.org" <cti-cybox@lists.oasis-open.org> Date: 11/19/2015 03:45 PM Subject: Re: [cti-cybox] CybOX 3.0: File Object Refactoring Sent by: <cti-cybox@lists.oasis-open.org> That’s a fair point, Jason – my only counter-argument is that most queries such as these can easily be expressed with a regular _expression_. E.g., for "find all observables that <match other params> and are explorer.exe” you’d have: file_name.regex = "explorer\.exe$” As far as John’s point about file extensions, I’d completely agree that they’re largely superfluous today. It’s also worth noting that our concept of “extensions” has to do with extending the File Object with context/domain-specific data and NOT with regards to “.dll”, “.exe” and so forth. Perhaps we need another name for it :) Regards, Ivan From: Jason Keirstead Date: Thursday, November 19, 2015 at 2:37 PM To: Ivan Kirillov Cc: "cti-cybox@lists.oasis-open.org" Subject: Re: [cti-cybox] CybOX 3.0: File Object Refactoring My only comment - and I have not decided where I sit on the fence - is that if you remove "file extension" and "file name" properties, and consolidate them all into one value called "path", this will make filtering and QUERY more difficult against your data. IE "find all observables that <match other params> and are DLL" or "find all observables that <match other params> and are explorer.exe" - Jason Keirstead Product Architect, Security Intelligence, IBM Security Systems www.ibm.com/security | www.securityintelligence.com Without data, all you are is just another person with an opinion - Unknown "Kirillov, Ivan A." ---11/19/2015 01:20:31 PM---All, As Trey mentioned in his previous email, we’ve been thinking about how to refactor and fix the From: "Kirillov, Ivan A." <ikirillov@mitre.org> To: "cti-cybox@lists.oasis-open.org" <cti-cybox@lists.oasis-open.org> Date: 11/19/2015 01:20 PM Subject: [cti-cybox] CybOX 3.0: File Object Refactoring Sent by: <cti-cybox@lists.oasis-open.org> All, As Trey mentioned in his previous email, we’ve been thinking about how to refactor and fix the issues associated with the File Object (and its subclasses). Accordingly, we’ve put together a page that outlines the existing issues and our ideas on addressing them: https://github.com/CybOXProject/schemas/wiki/CybOX-3.0:-File-Object-Refactoring We’ll be discussing this during today’s call, but we’d love to get your input here (and/or on Slack) as well – generally on your feelings with regards to these changes, but also on:
· Are there any other properties for the default extensions that we should be adding?
Regards, Ivan and Trey --------------------------------------------------------------------- To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail. Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php [attachment "graycol.gif" deleted by Jason Keirstead/CanEast/IBM] --------------------------------------------------------------------- To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail. Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php [attachment "graycol.gif" deleted by Jason Keirstead/CanEast/IBM] --------------------------------------------------------------------- To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail. Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php --------------------------------------------------------------------- To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail. Follow this link to all your TCs in OASIS at: https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php [attachment "graycol.gif" deleted by Jason Keirstead/CanEast/IBM] [attachment "C690F973-64C5-4C00-889B-C1A5BB4A2A0B[11].png" deleted by Jason Keirstead/CanEast/IBM] |
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]