Make your programs run faster by better using the data cache (2020)

Re:Make your programs run faster by better using the data ca... by nathants on Friday June 23, 2023 @11:16PM (#36455145)

this is a great journey! what volume of data were you analyzing?

more you could try:

- span multiple machines, with lambda/ec2-spot and s3

- write c/cpp, call via python subprocess or cffi

- pytorch gpu on ec2-spot

[ Parent | on HN ]

Re:Make your programs run faster by better using the data ca... by behnamoh on Friday June 23, 2023 @11:36PM (#36455278)

Thanks for the input! I'll look them up.

Depending on the resolution of my parameters, I'm dealing with a 32-40GB-ish data.

[ Parent | on HN ]