Re: [EXT] [cti] [EXT] Re: [cti] TAXII Pagination Example Text

[Bret Jordan <Bret_Jordan@symantec.com>]

As I stated in a previous email, you should be able to easily do a date_added+.000001 for each record.  You are going to have to store these records on your system, and thus you will need some basic meta-data to store them anyway.  You probably are going to store which device you got them from, and a bunch of other things.  A simple timestamp should not be a significant challenge.

If you want to write up a proposal for TAXII 2.2, I am sure others, including me, would love to see it.  Please just make sure your proposal does not break existing use-cases.

Thanks
Bret


> On Sep 9, 2019, at 5:14 PM, Andras Iklody <andras.iklody@circl.lu> wrote:
> 
> The data is collected in bulk from the other sources. We could indeed
> attach a timestamp to the bulk collection, but that would be the same
> for all data collected in one shot exceeding any sane pagination limits.
> 
> I am somewhat confused that we're back at the data storage question
> though, TAXII/STIX should NOT have inherent needs when it comes to how
> data is stored as from what I understood it's purely a
> transport/exchange format / protocol. If TAXII basically requires us to
> move all the stored data to true micro second precision (instead of
> padding it on fetch for lower precision tools), then that will kill a
> lot of interactions with other tools out there.
> 
> Best regards,
> Andras
> 
> On 09.09.19 17:07, Bret Jordan wrote:
>> So if you are building a streaming collector that gathers data from other sensors, you have control over how that data gets added / stored on that intermediary streaming server.  Since you may need to paginate the data, you must be definition then need to temporarily store that data.  If you need to store data, then you can define what ever data structures you want.  Thus you can define the date added property with microsecond precision.   Keep in mind that the data added is NOT the date that the end / origin sensor used for its timestamp. 
>> 
>> I just do not see how this is broken or why it would not work. Please help meâ
>> 
>> Bret