This reminds me of a horrible hack I once put into an internal ruby (sinatra) REST API server that had to aggregate data from a ton of SOAP servers in a pretty large environment, and return some computed results as JSON. The amount of XML that had to be parsed into objects to do the reporting was massive, and the web workers (being ruby) never gave the memory back to the OS. Running the code inside a unicorn process would basically cause it to start using several GB of ram, and over time all the unicorn workers would become bloated and the service would have to be restarted.

I eventually decided to just fork() the ruby process, do the processing, compute the JSON results (which were comparatively small) and hand the results back as a string over a socket to the parent process, which would cache them and respond. It pretty much solved the memory issue, since the forked process would just exit() gave all the memory back.

In fact, I think I ended up doing a GC.disable() inside the forked process (because I was going to exit anyway) and it sped the process up from taking >2 minutes to just a few seconds. Ruby's GC was utterly terrible back then.

It was an internal tool that never was touched by the general internet, which is why we could "afford" to have it do such intensive work behind a single request, but I always wonder if there was a better approach that wasn't so awful.