Re:Ballista: Distributed compute platform implemented in Rus...
by cogman10 on Monday January 18, 2021 @02:24PM
(#25825455)
I think that's a fair description of pretty much every webapp :D
Most of them are doing nothing more than copying data from the database into an http stream.
Re:Ballista: Distributed compute platform implemented in Rus...
by ithkuil on Monday January 18, 2021 @02:34PM
(#25825599)
Well, all what CPUs do is copying memory from one memory location to another, perform some arithmetic and do some conditional jumps :-)
Re:Ballista: Distributed compute platform implemented in Rus...
by xiphias2 on Monday January 18, 2021 @03:15PM
(#25826077)
Most of the arithmetic is checking optional fields if they are empty or not before copying :)
Re:Ballista: Distributed compute platform implemented in Rus...
by cogman10 on Monday January 18, 2021 @02:49PM
(#25825795)
pop pop jump jump oh what a crunch it is.
(sung to this tune https://www.youtube.com/watch?v=iENQXIQ8wH0 )
Side note: It really is incredible what happened in the early days of computing when memory and computation were limited. How much care was taken in the precise layout of memory or even the timing of a calculation was insane.
Re:Ballista: Distributed compute platform implemented in Rus...
by munk-a on Monday January 18, 2021 @03:22PM
(#25826153)
Don't forget the validation!
Webapps are dumb middleware that pipes data from the database into an http stream - but it needs to determine which database calls to invoke and sanitize all the incoming junk.
Re:Ballista: Distributed compute platform implemented in Rus...
by lmeyerov on Tuesday January 19, 2021 @01:21AM
(#25830592)
We use arrow to do the validation 'for free': it has a typed columnar schema, so by passing the schema, we get the recordset validation without inspecting values.
we still need to do semantic validation, but w/ arrow, now we do that in bulk and on gpus :)
Re:Ballista: Distributed compute platform implemented in Rus...
by xiphias2 on Monday January 18, 2021 @03:12PM
(#25826043)
Sure, but when you are using C++, protobufs are not the best way to store data...but I guess it could be worse.
Most of what the CPUs at Google are doing is just copying fields from one protobuffer to another.