Re:Processing 40TB of code from 10M projects with a dedicate...
by folkrav on Tuesday October 01, 2019 @09:22AM
(#21125012)
Why is it ridiculous? You'll need the related libraries either case, and by default virtualenv uses symlinks. The "full virtualenv" part makes it sound like it's heavy, or something like that.
[ Parent
| on HN ]
Re:Processing 40TB of code from 10M projects with a dedicate...
by theshrike79 on Tuesday October 01, 2019 @03:12PM
(#21129390)
If you do it in a language that provides static binaries or an equivalent, you don't need to pull dozens of libraries with possibly hundreds of files just to run a single script.
Imagine a situation where you'd need to perform an operation on a hundred servers, would you either transfer one static exacutable or build an virtualenv in every machine and download correct versions all of the related libraries?
Re:Processing 40TB of code from 10M projects with a dedicate...
by folkrav on Tuesday October 01, 2019 @04:52PM
(#21130488)
If I controlled the environment, I'd probably build a Docker image and deploy that. Otherwise yes, This kind of extreme example isn't very representative of the initial situation though. And you're also talking about switching up languages, which isn't always possible or easy, especially in large teams or codebases.