News for hackers. Stuff that matters.
Re:My £4 a month server can handle 4.2M requests a day by SNosTrAnDbLe on Tuesday September 07, 2021 @10:44PM (#28452432)
I work in the Analytics space and been mostly on Java and I am so glad that other people feel the same. At this point, people have become afraid of suggesting something other than Spark. I see something written in Rust to be much better at problems like this. I love the JVM but it works well with transactional workloads and starts showing its age when its dealing with analytical loads. The worst thing is then people start doing weak references and weird off the heap processing usually by a senior engineer but really defeats the purpose of the JVM
Re:My £4 a month server can handle 4.2M requests a day by wiz21c on Wednesday September 08, 2021 @02:25AM (#28453748)
I guess your company is running on Java and running something else would cost a lot in training, recruiting, understanding, etc. But down the line, defeating the JVM will be understood only by the guy who did it... Then that guy will leave... Then the newcomers will rewrite the thing in Spark 'cos it will feel safer. Rinse-repeat...

(I'm totally speculating, but your story seems so true that it inspired me :-)

Re:My £4 a month server can handle 4.2M requests a day by SNosTrAnDbLe on Wednesday September 08, 2021 @04:17PM (#28462475)
Some of it is what you mentioned about training and hiring costs that but mostly its this creation of the narrative that it will scale someday in the future. This is usually done by that engineer(s) and they are good at selling so a different opinion is ignored or frowned upon by the leadership.

I have now seen this anti pattern in multiple places now

Re:My £4 a month server can handle 4.2M requests a day by robertlagrant on Wednesday September 08, 2021 @06:37AM (#28455287)
> I love the JVM but it works well with transactional workloads and starts showing its age when its dealing with analytical loads.

This is interesting. Can you elaborate a bit?

Re:My £4 a month server can handle 4.2M requests a day by SNosTrAnDbLe on Wednesday September 08, 2021 @04:30PM (#28462639)
Analytical loads deal with very large datasets in the order of terabytes even after you compress them. These workloads dont change much so keeping them in the heap eventually results in long processing pauses because JVM tries to recover this memory. However, in most cases, this data is not meant to be garbage collected. For transactions, once you have persisted the data, it should be garbage collected so the pattern works. There are a lot of other aspects that I can probably think of but the above one is the most important in my mind.