Re:An Interview with Jose Valim, Creator of Elixir
by joelbluminator on Friday October 18, 2019 @06:03AM
(#21290015)
But part of the idea behind a queue job is not to overwhelm the server with heavy jobs that aren't urgent. Even with a thread pool, in java, you may still overwhelm the server with heavy jobs like sending emails, if you do not clog the threads you may still spike the memory - do you really want your server and your background jobs to share the same memory and resources?. So the separate instance make sense to me - they are a feature in fact. I really want to know from someone from the java ecosystem more about how they handle heavy background jobs. I get that Elixir may have this part done in a nicer way, but still ,Sidekiq is a VERY polished tool.
Re:An Interview with Jose Valim, Creator of Elixir
by cpursley on Saturday October 19, 2019 @06:24AM
(#21297767)
> do you really want your server and your background jobs to share the same memory and resources
Erlang/Elixir processes don't share memory (Actor model) like Java threads do, giving it a distinct advantage (see Akka for Actor model in Java).
But yeah, for heavy stuff even in Elixir, you might want to outsource that to a Job server (which there are many libraries in Elixir for this).
Re:An Interview with Jose Valim, Creator of Elixir
by joelbluminator on Friday October 18, 2019 @05:59AM
(#21289985)
I'm really asking because I don't know - in java world background queues are actually rare?
Re:An Interview with Jose Valim, Creator of Elixir
by jonesetc on Sunday October 20, 2019 @08:09PM
(#21308238)
probably a bit of loose wording there. It's probably not what either of us would call objectively rare, but it is comparatively rare compared to many other runtimes. There's quite amount of this that can be avoided with some vertical or horizontal scaling depending on what you are doing with these events.
For instance you linked the spring application events integration with redis, while that is definitely an option it's not one you'd immediately reach for. Instead you'd more likely set up and use application events and/or domain events (if using spring data) handled locally with thread pool. The benefit here is that of course way less set up and complexity, but you can also handle these events sync or async.
Sure, you may want to move your Java/Erlang jobs to Redis/SQS/etc for different reasons, but it is not your starting point. And given you were explicitly talking about CRUD apps, being able to have everything you need with only your app + database, without a need for Redis, workers, or tasks queues, leads to operational simplicity.