News for hackers. Stuff that matters.
Re:Fast, Safe, and Complete(ish) Web Service in Rust by Chyzwar on Tuesday March 27, 2018 @06:35PM (#16693429)
You can keep pushing bytes as you get them from async operations. In my nodejs am making 20000 DB queries and I can push data to connected clients when any these get completed.

In the threaded code, you need to get a window of time and memory. In my Java, I have a max heap size of 8gb and I can maybe create up to 8k threads. When making 20000 DB queries only 8k is being executed where most of these are waiting for the query to complete or CPU time.

Re:Fast, Safe, and Complete(ish) Web Service in Rust by vbezhenar on Wednesday March 28, 2018 @02:26AM (#16695806)
You're wrong about thread limit. I can easily create 100k threads on my PC and it takes around 6GB memory (and I didn't even bother to investigate that consumption, probably I could do with less). 8k threads is nothing. And beefy server machine should be able to deal with million of threads (but I guess that's around practical limit).
Re:Fast, Safe, and Complete(ish) Web Service in Rust by Chyzwar on Wednesday March 28, 2018 @03:07AM (#16695997)
This depends on thread implementation and how much is allocated for the stack. Java will by default allocate 1MB, you can change that if you know what are you doing.

I know that you can have millions of M:N/green threads but these are different. In any case, you need to compete for memory and CPU time to execute.

Since not many people are programming in Haskell and BEAM languages my simplifications still hold.

Re:Fast, Safe, and Complete(ish) Web Service in Rust by vbezhenar on Wednesday March 28, 2018 @04:13AM (#16696231)
I'm talking about real threads, not green threads and I used Java for my quick test. Here's code, you can run it yourself. masklinn is correct about virtual memory, every sane OS will only use few kilobytes for short stacks.

    public class Test {
        volatile static boolean stop = false;
        static AtomicInteger counter = new AtomicInteger();
        public static void main(String[] args) throws InterruptedException {
            Thread[] threads = new Thread[100000];
            for (int i = 0; i < threads.length; i++) {
                threads[i] = new Thread(() -> {
                    counter.incrementAndGet();
                    while (!stop) {
                        try {
                            Thread.sleep(1000);
                        } catch (InterruptedException ignored) {
                        }
                    }
                });
            }
            for (Thread thread : threads) {
                thread.start();
            }
            while (counter.get() < threads.length) {
                Thread.sleep(1000);
            }
            System.out.println("Threads are running");
            Thread.sleep(10000);
            stop = true;
            for (Thread thread : threads) {
                thread.join();
            }
        }
    }
Re:Fast, Safe, and Complete(ish) Web Service in Rust by masklinn on Wednesday March 28, 2018 @03:37AM (#16696103)
> This depends on thread implementation and how much is allocated for the stack. Java will by default allocate 1MB, you can change that if you know what are you doing.

Isn't that vmem? Unless it also writes to the entirety of that 1MB before starting the thread, it's not actually committed by default on most unices.