News for hackers. Stuff that matters.
Re:Migrating a 40TB SQL Server Database by jeffbee on Thursday July 30, 2020 @06:23PM (#24003501)
It's adorable when people think scientific computing has the same scale as a Google or Microsoft.
Re:Migrating a 40TB SQL Server Database by FridgeSeal on Thursday July 30, 2020 @06:53PM (#24003702)
Re:Migrating a 40TB SQL Server Database by jeffbee on Thursday July 30, 2020 @07:05PM (#24003785)
Ignoring the fantasy b.s. in the second half of the article, the stuff at the top is exactly what I mean.

A mighty 400 GB/s: i.e. much less than the > 50 PB/day of logs the other person mentioned;

1600 hours of SD video per second: i.e. about 1-2 million concurrent HD streams, or much less than the amount actually served by YouTube.

IBM Summit "world's most powerful supercomputer": < 5000 nodes, i.e. much below the median cell size described in the 2015 Borg paper. Summit is a respectable computer but it would get lost in a corner of a FAANG datacenter.

Re:Migrating a 40TB SQL Server Database by lukeschlather on Thursday July 30, 2020 @07:24PM (#24003905)
CERN is a correct example. The LHC reportedly generates 1PB per second: https://home.cern/news/news/computing/cern-data-centre-passe...
Re:Migrating a 40TB SQL Server Database by jeffbee on Thursday July 30, 2020 @07:29PM (#24003945)
If you define “generates” to mean “discards” then yes.
Re:Migrating a 40TB SQL Server Database by welterde on Friday July 31, 2020 @04:05AM (#24006689)
It still gets processed though and only all non-interesting events get discarded..

Otherwise the tape alone to store it on would exceed their total operating budget in a day, so they have to be a bit clever about it.

Re:Migrating a 40TB SQL Server Database by tutfbhuf on Saturday August 01, 2020 @02:01PM (#24022548)
I think that even Google can not save 1PB/s in 2020.
Re:Migrating a 40TB SQL Server Database by welterde on Friday July 31, 2020 @04:35AM (#24006841)
The numbers are not fantasy at all - this will be a huge radio telescope - one square kilometer of pure collecting area and thousands of receiving antennas (For reference: Arecibo has around 0.073 km^2). We are talking data input to the correlator on the terabit/s scale. And technology-demonstration with ASKAP are well under way. ALMA is working quite well by now as well (> 600 Gb/s with just an 50 antenna array).
Re:Migrating a 40TB SQL Server Database by tedisaacs on Friday July 31, 2020 @02:18AM (#24006190)
it’s adorable how proud you are to have worked at FAANG and how angry you get at the idea some other organisation handles equivalent scale
Re:Migrating a 40TB SQL Server Database by andialo on Friday July 31, 2020 @09:57AM (#24008616)
touche
Re:Migrating a 40TB SQL Server Database by pstrateman on Thursday July 30, 2020 @07:52PM (#24004135)
400GB/s is about 35 PB/day

not quite as big a difference

Re:Migrating a 40TB SQL Server Database by Keyframe on Friday July 31, 2020 @09:44AM (#24008481)
So, Youtube puts whole streams into their logs? Interesting.