News for hackers. Stuff that matters.
Re:Performance comparison: counting words in Python, Go, C++... by macgyverismo on Monday March 15, 2021 @09:28AM (#26465828)
I was suprised at the C++ outcome, so I started testing myself. I was very amazed to find a 4~5x speedup when reading directly from file (std::ifstream) instead of reading stdin. This was on a 10.5.7 MBP 2014

[edit]

more like 9~10x actually...

  time { ./build/Release/countwords_ifstream; } | tail
  ...
  real 0m1.337s
  user 0m1.332s
  sys 0m0.022s

  time { ./build/Release/countwords_original < kjvbible_x10.txt; } | tail
  ...
  real 0m12.184s
  user 0m12.138s
  sys 0m0.044s
changes between these two:

  6a7
  > #include <fstream>
  8a10
  >  std::ifstream inp( "/Users/macgyverismo/Desktop/test/kjvbible_x10.txt" );
  13c15
  <     while (std::cin >> word) {
  ---
  >     while (inp >> word) {
Re:Performance comparison: counting words in Python, Go, C++... by cornstalks on Monday March 15, 2021 @12:47PM (#26468330)
I dug into this once because reading from stdin on my Mac was so slow, but only when using std::cin. Using C APIs to read stdin was fast. Turns out libc++ on mac uses `getc` to read from stdin one character at a time. Madness.
Re:Performance comparison: counting words in Python, Go, C++... by darwingr on Tuesday March 16, 2021 @12:57AM (#26473179)
Huh, well that makes a lot of sense. Three years ago when I was in University practicing for competitive coding we came across a limit in the performance (ms) we could get on even simple problems. The professor leading the practice poked away at his own solution to some problem for days because he couldn't get a better time than the top score online. Eventually he implemented his own versions of cin and cout. Needless to say he was pretty frustrated but there's just some problems that are hard to let go of without a solution.
Re:Performance comparison: counting words in Python, Go, C++... by w-m on Monday March 15, 2021 @12:50PM (#26468361)
Similar here, reading the file from stdin on a 2020 MBP on macOS 11.2.2 is incredibly slow.

    int main() {
        std::ios::sync_with_stdio(false);
        std::string str((std::istreambuf_iterator<char>(std::cin)),
                         std::istreambuf_iterator<char>());
    }
takes around 8 seconds to run, no counting at all.

    int main() {
        std::ifstream f("kjvbible_x10.txt");
        std::string str((std::istreambuf_iterator<char>(f)),
                         std::istreambuf_iterator<char>());
    }
runs in ~270 miliseconds. Factor 30x, yikes.
Re:Performance comparison: counting words in Python, Go, C++... by c-cube on Monday March 15, 2021 @09:50AM (#26466072)
Could be a lock on stdin that is acquired in a loop? Rust has such a lock and typically one should lock stdin at the beginning and read from within the critical section.
Re:Performance comparison: counting words in Python, Go, C++... by FAANG_dream on Monday March 15, 2021 @09:52AM (#26466087)
Re:Performance comparison: counting words in Python, Go, C++... by macgyverismo on Monday March 15, 2021 @10:16AM (#26466334)
Asuming you mean 'std::ios::sync_with_stdio(false);', that was already present in the original code.
Re:Performance comparison: counting words in Python, Go, C++... by usefulcat on Monday March 15, 2021 @11:16AM (#26466987)
I tried that on Linux and got at best a 5% improvement.
Re:Performance comparison: counting words in Python, Go, C++... by benhoyt on Monday March 15, 2021 @01:20PM (#26468763)
Yeah, I'm on Linux too and tried that with no noticeable difference in results. There's something off about macOS's approach.
Re:Performance comparison: counting words in Python, Go, C++... by nerdponx on Monday March 15, 2021 @09:38AM (#26465940)
Is that due to something at the OS or filesystem driver level?