News for hackers. Stuff that matters.
Re:O(N^2) in CreateProcess by rightbyte on Monday April 22, 2019 @06:44AM (#19717833)
Yes, but he creates 1000 processes of the same executable with different arguments to run different tests as I understand it. With a really big executable.

That's not the same thing as it's slow to add normal sized processes.

Re:O(N^2) in CreateProcess by mort96 on Monday April 22, 2019 @06:55AM (#19717879)
That's not out of the ordinary though?

Any time you compile C or C++ code, you're creating a process from the same GCC/Clang/whatever executable but with different arguments for each source file. For a big project, it's not uncommon for there to be thousands or tens of thousands of small source files. Creating and maintaining processes is the core responsibility of an operating system.

Re:O(N^2) in CreateProcess by rightbyte on Monday April 22, 2019 @07:47AM (#19718126)
gcc is like 1Mb. It would be interesting if he did the same test on Debian or something with default options.

I would personally link in the tests as a shared lib or something. chrome.exe on my laptop is around 1Mb. That would probably speed up his tests more than his hacks to the validation routine.

Re:O(N^2) in CreateProcess by brucedawson on Tuesday April 23, 2019 @12:56PM (#19730465)
I have been told that moving the meat of unit_tests.exe to a DLL would avoid the repeated CFG initialization costs (they would be paid on the first CreateProcess call and then the CFG data would be shared). This wouldn't speed up the tests "more" than my change did (it would speed up the tests slightly less, actually), but it would be a possible fix.

But, such a change would be more work. And, honestly, it shouldn't be necessary. Windows could avoid this problem by using an O(n) algorithm for the initialization, or by sharing the CFG data between .exe files as well as .dll files. I am content with my current workaround and I'll consider reverting it when the underlying OS issue is fixed.

Note that the chrome executable on Linux contains all of the code so it is 50-100 MB. It is only an accident of history that chrome.exe on Windows puts all the code in chrome.dll/chrome_child.dll

Re:O(N^2) in CreateProcess by leeter on Monday April 22, 2019 @10:42AM (#19719278)
I have a sneaky suspicion you're right, windows generally assumes any 'shared' code will be in DLLs and is optimized around ensuring they are shared as much as possible in memory. It's likely that if the tests were in a DLL vs. the EXE then windows would just share the CFG page for that DLL read only between all the processes and call it done.
Re:O(N^2) in CreateProcess by KyleJ61782 on Monday April 22, 2019 @10:22AM (#19719129)
The size of the executable doesn't really matter after the first load as subsequent loads share the mapped section so it doesn't need to be loaded from disk again. Loading an executable multiple times after the first time really shouldn't take all that long from an OS perspective...just some mapping of shared pages.
Re:O(N^2) in CreateProcess by brucedawson on Monday April 22, 2019 @11:52AM (#19719876)
> The size of the executable doesn't really matter > after the first load

shouldn't really matter. If you have CFG enabled then it does. If they fix CFG's initialization then it will go back to not mattering, and even this huge process will be created in just a few ms.