| I think what you are calling out, represents a lot of the way people think of things. For example, if you know the event happened it December 2015, but you were not sure of the day, then you would probably do:
timestamp = 2015-12-00T00:00:00Z precision = month
But that can be interpreted as 2015-11 - 2016-01
The other problem with precision is how do you say, I know it happened the first few days of January around noon.
Thanks,
Bret Bret Jordan CISSPDirector of Security Architecture and Standards | Office of the CTO Blue Coat Systems PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050 "Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."
<typing with my analyst hat on> If I say “an incident occurred on Tuesday Jan 19”, I mean it occurred on Tuesday, between midnight and midnight. I would represent this as: Timestamp: 2016-01-19T00:00:00Z Timestamp_precision: day If I am reporting details on phishing campaigns reported during December 2015, I mean those reported between midnight Dec 1, 2015 through midnight Jan 1, 2016. I would represent this as: Timestamp: 2015-12-01T00:00:00Z Timestamp_precision: month This would be the equivalent of the following SQL (or something similar without whatever syntax errors I unwittingly included): Select * from tbl_incidents where incidentDateTime > 2015-12-01 and incidentDateTime < 2016-01-01 Hope this makes sense, Chris Ricard FS-ISAC This is another violent agreement, “yes, and” situations. Yes, this is how the data gets generated in the wild. The problem is that unless we put our foot down and chose whether the time is the midpoint of the bucket or the bottom of the bucket, the consumer HAS NO CLUE what the bucket is. It is really trivial if you wrote the producer and the consumer: they both will encode your world view. It is really hard for a multivendor solution to have the same interpretation of what the bucket is unless we specify it here. I really do NOT want to add yet another timestamp parameter. Precision is bad enough. “Error bars” or “count from the bottom” or “count from the middle” is really ugly. I would put the onus on the client and specify one and only one way to express time stamps. So, we could do it that way - which would require the producer to take the equivalent of 100% of their known precision and adjust their timestamps downward accordingly. I would argue strongly though that this is pretty much *never* how this is done in industry and would result in confusion. Normally the onus is on the consumer of information to interpret the producers information as they see fit when they know the precision.
Here is the difference:
- If I follow the specification below, and I read the time 12:00:00 off the clock and know my precision to be minute-level, then I would have to supply a timestamp of 11:59:00 with a precision of 1 minute ( note here the importance, that minute-level precision is not the same as 60 second precision - it actually requires a 2x the confidence interval time-boxing - this is important!). The consumer would then take that information and know "OK the time starts at 11:59:00 and ends between then at 12:01:00"
- The way it is normally done instead, is the producer of the time-sensitive information just sends whatever time came off of their information producing source. The consumer of that information then constructs the time-box around whatever rules they see fit. To carry forward the above example, the producer would send me 12:00:00 with 1 minute precision, and I would know implicitly, if I care about this at all, that that event could have occurred any time between 11:59:00 and 12:01:00.
I think that the second method is how pretty much all systems behave. I have never known a system to behave the first way.
-
Jason Keirstead
Product Architect, Security Intelligence, IBM Security Systems
www.ibm.com/security | www.securityintelligence.com
Without data, all you are is just another person with an opinion - Unknown
<graycol.gif>Eric Burger ---01/19/2016 10:06:48 PM---I would offer the important precision is not not hours, minutes, or seconds, but number of seconds.
From: Eric Burger <Eric.Burger@georgetown.edu>
To: cti@lists.oasis-open.org
Date: 01/19/2016 10:06 PM
Subject: Re: [cti] Timestamp Serialization Question
Sent by: <cti@lists.oasis-open.org> <6650AAD86713469292D0ED4A3C07F67E.png>
I would offer the important precision is not not hours, minutes, or seconds, but number of seconds. We also need to define whether the timestamp represents the middle of the range or the bottom.
For example, if I only transmit the hour portion of the timestamp of an event, then 12:00:00Z means anything from 12:00:00.000000000 to 12:59:59.999999999. However, if I transmit the closest hour portion of the timestamp of an event, then 12:00:00Z means anything from 11:30:00.000000000 to 12:29:59.9999999999.
Note that a typical data collection window is tenths of minutes. That is six seconds, not ‘seconds.’ I.e. 12:00:00Z means either 12:00:00Z - 12:00:05.999999999 or 11:59:57Z - 12:00:02.999999999.
My suggestion is (1) the timestamp represents the bottom of the range of the bucket and (2) the number is that for precision of less than a second (i.e., granularity of more than a second) is a bucket of seconds starting from the timestamp value with a precision number of seconds. So, examples would be:
12:00:00Z = event happened between 12:00:00Z - 12:00:00.9999999999Z [default precision = 1s]
12:00:00Z (precision = 60s) = event happened between 12:00:00Z - 12:00:59Z [formerly known as “minute precision”]
12:00:00Z (precision = 3600s) = event happened between 12:00:00Z - 12:59:59Z [formerly known as “hour precision”]
12:00:00Z (precision = 6s) = event happened between 12:00:00Z - 12:00:05.999999999Z [how else would you specify “tenth of a minute”?]
The use case as I understand it at a high level is so that when someone submits a timestamp of 12:00:00 zulu, we know the difference between if they truely mean exactly 12:00:00 on the button, or if they only have second level precision available to them. And this is required because we aren't mandating a fixed format, but RFC 3339 which is variable.
Eric Burger --- Re: [cti] Timestamp Serialization Question --- From: | | To: | | Date: | Tue, Jan 19, 2016 9:30 PM | Subject: | Re: [cti] Timestamp Serialization Question |
<63A539EA131E4378984BB48524099357.png>
I’m still clueless as to the use case.
Not a negative statement, but I would like to see the concise reason we need ‘precision’ before I weigh in, if at all.
On Jan 19, 2016, at 3:22 PM, Jordan, Bret <bret.jordan@BLUECOAT.COM> wrote:
There tends to be two options for dealing with objects that have multiple timestamps and their corresponding precision. Sean and I have been talking through the pros and cons of these. We would like to get everyone's opinion. Which do you prefer, option 1 or option 2
Option 1:
This option put the burden on the JSON serialization format to add an extra "_precision" field to each timestamp enabled field. This is a much flatter and easier to parse and process representation, but the con is it requires unique field names.
{
"type": "incident",
"initial_compromise_time" : "2015-12-07T22:00:00Z",
"initial_compromise_time_precision": "hour",
"first_data_exfiltrated_time" : "2015-12-09T05:11:00Z",
"first_data_exfiltrated_time_precision" : "minute",
"incident_opened_time" : "2016-01-15T11:19:22Z",
"incident_closed_time" : "2016-01-19T17:24:017Z"
}
Option 2:
This option will require a nested object and struct to store this data and will have an extra layer of indirection for all of those times when the timestamp is at the default precision.
{
"type": "incident",
"initial_compromise_time" : {
"timestamp": "2015-12-07T22:00:00Z",
"timestamp_precision": "hour"
},
"first_data_exfiltrated_time" : {
"timestamp": "2015-12-09T05:11:00Z",
"timestamp_precision" : "minute"
},
"incident_opened_time" : {
"timestamp": "2016-01-15T11:19:22Z"
},
"incident_closed_time" : {
"timestamp": "2016-01-19T17:24:017Z"
}
}
Thanks,
Bret
Bret Jordan CISSP
Director of Security Architecture and Standards | Office of the CTO
Blue Coat Systems
PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."
|