[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [cti] Re: [EXT] Re: [cti] TAXII Pagination Example Text
I will re-quote my previous statement: "I would like to stress that we have had this discussion many times. Some of the discussions have run for months. In the end, when we review all of the use cases and the 90/10 most common needs, we have always come back to the simple pagination by the date an object was added to the TAXII server. This design makes it super simple for client and server." "For those constantly advocating for a different method, please remember that content may rapidly be changing on the system due to the very nature of threat intelligence. As such, you really need a canonical unchanging entry point to pull / sync data from, thus the date_added to the system.â If you Andras and Jason want to write up a proposal for TAXII 2.2, I would love to see it. However, please make sure your proposal does not break existing use cases. Bret > On Sep 9, 2019, at 4:49 PM, Andras Iklody <andras.iklody@circl.lu> wrote: > > This x1000 > > On 09.09.19 16:26, Jason Keirstead wrote: >> I agree with the problem; >> >> The problem is rooted in the fact that assuming that a document has an >> "insertion time", is assuming the document lives as-is in a database. >> >> This all goes back to the "STIX and TAXII are not a database" mantra. >> >> - >> Jason Keirstead >> Chief Architect - IBM Security Threat Management >> https://clicktime.symantec.com/3LmCGKpK327SSkxbn47FL9Y7Vc?u=www.ibm.com%2Fsecurity >> >> "Would you like me to give you a formula for success? It's quite simple, >> really. Double your rate of failure." >> >> - Thomas J. Watson >> >> >> >> From: Andras Iklody <andras.iklody@circl.lu> >> To: Bret Jordan <Bret_Jordan@symantec.com>, Wesley Brown >> <wbrown@lookingglasscyber.com>, "drew.varner@ninefx.com" >> <drew.varner@ninefx.com>, Jason Keirstead <Jason.Keirstead@ca.ibm.com> >> Cc: Allan Thomson <athomson@lookingglasscyber.com>, >> "cti@lists.oasis-open.org" <cti@lists.oasis-open.org> >> Date: 09/09/2019 11:10 AM >> Subject: [EXTERNAL] Re: [cti] Re: [EXT] Re: [cti] TAXII >> Pagination Example Text >> Sent by: <cti@lists.oasis-open.org> >> ------------------------------------------------------------------------ >> >> >> >> OK, my use-case is as follows. >> >> I have a sensor that ingests large amounts of data (parsing network logs >> / netflow, passiveDNS, etc). This sensor stores the data in its own >> format, with the timestamp being accurate to the second. If I were to >> build an interface that responds to TAXII queries for the collector in >> front of the sensor, and I were to query this data, I'd be dealing with >> large data-sets and I'd want to paginate it. >> >> Unless I set a limit that will probably blow through my memory >> limitations, I have no other TAXII-compliant way to paginate the data in >> sane chunks without either losing some data (anything beyond the memory >> limit's envelope for a given second) or without blowing through my >> memory limits. >> >> Best regards, >> Andras >> >> On 09.09.19 16:00, Bret Jordan wrote: >>> Andras, >>> >>> Thanks for the question. TAXII should work well for this use case. I >>> do not see why it would not. Please keep in mind that the limits we >>> were talking about are optional. So a server / sensor may have no limit >>> which lets you pull all records at once. >>> >>> The sensor can dynamically add / figure out the date-added values how >>> ever it needs to do so. So I am not sure why this would not work. Can >>> you help me understand why you think it will not work? Or does this >>> solve your concerns? >> >> --------------------------------------------------------------------- >> To unsubscribe from this mail list, you must leave the OASIS TC that >> generates this mail. Follow this link to all your TCs in OASIS at: >> https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php >> >> >> >> >> > > --------------------------------------------------------------------- > To unsubscribe from this mail list, you must leave the OASIS TC that > generates this mail. Follow this link to all your TCs in OASIS at: > https://clicktime.symantec.com/3EVePYDHwmiYmEm4qwu9xKo7Vc?u=https%3A%2F%2Fwww.oasis-open.org%2Fapps%2Forg%2Fworkgroup%2Fportal%2Fmy_workgroups.php >
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]