News for hackers. Stuff that matters.
Re:Conspiracy: In Theory and Practice by posix_me_less on Wednesday June 30, 2021 @09:57AM (#27688587)
> unlikely that anybody, including the NSA, can collect, index, catalog and parse meaning out of hundreds of GB _per second

No, it's actually very likely. I've noticed people often make this argument as if it is obvious that the data that is being processed is somehow too large to handle. But they misunderstand the ways this can be done and underestimate current technological capabilities.

One big switch in single datacenter can handle all traffic of that datacenter, hundreds of GB/s. Make a special one with custom FPGA/cpus and in addition to switching, it can do decrypting, parsing, matching and filtering the way you like, hundreds of GB per second, real-time. This is a highly paralelizable job and the hardware is made to exploit that.

The resulting stream of interesting data is probably much thinner and can be easily sent to some central location, realtime. The rest can be stored locally and sent on demand.

Re:Conspiracy: In Theory and Practice by grumblenum on Wednesday June 30, 2021 @12:51PM (#27690759)
I can see how hardware handling of switching, some filtering and maybe stripping TLS is feasible. Since you're knowledgeable of the subject, how would you index that information for future retrieval, which could be anything in the set of all digital representations of information in all possible languages and encodings? How would you determine what information is important?
Re:Conspiracy: In Theory and Practice by fjrueosjdksn on Wednesday June 30, 2021 @08:01PM (#27694942)
packet metadata, intercept location and timestamp, will form the index.
Re:Conspiracy: In Theory and Practice by fjrueosjdksn on Wednesday June 30, 2021 @08:00PM (#27694929)
You are not incorrect about that hardware.

Splice off fibre optics, copy packets, localised hardware processes like you say, directs to data center for future processing or active surveillance or active legal interception, etc.

Re:Conspiracy: In Theory and Practice by posix_me_less on Thursday July 01, 2021 @08:53AM (#27698598)
Which part is incorrect?