Tell HN: Python's print function is thread-unsafe

Posted by kerneloops on Monday January 01, 2024 @03:31AM
from the self-post dept.

Some days ago I found that Python's print function is by default thread-unsafe, as the underlying sys.stdout is by default a TextIOWrapper, which is not thread safe. That means, for the code below

  from threading import Thread
  def target(x):
      for _ in range(100000): print(x)
  Thread(target=target, args=['a' * 10]).start()
  Thread(target=target, args=['b' * 10]).start()

Python could print out not only interleaved bytes (aaabbbabab), but also null bytes and uninitialized memory, thanks to non-synchronized buffering.

( 6 comments | 4 points | on HN )

Re:Tell HN: Python's print function is thread-unsafe by kerneloops on Monday January 01, 2024 @03:35AM (#38830456)

A survey of some other languages:

C printf: MT-Safe locale.

C++ std::cout: safe, unless you call sync_with_stdio(false).

JVM System.out.println: safe in common JVMs.

C# Console.WriteLine: safe.

Go fmt.Printf: safe.

Rust println!: safe.

Ruby puts: safe.

So it seems that Python is the outlier here.

[ Parent | on HN ]

Re:Tell HN: Python's print function is thread-unsafe by vitaut on Monday January 01, 2024 @11:49AM (#38833141)

std::cout is thread-safe but because of the concatenation-based API parts of the message may interleave. C++23 std::print is safe (and the output doesn't interleave).

See https://vitaut.net/posts/2023/print-in-cpp23/.

[ Parent | on HN ]

Re:Tell HN: Python's print function is thread-unsafe by cratermoon on Monday January 01, 2024 @04:11AM (#38830595)

Why Threads Are a Bad Idea: https://news.ycombinator.com/item?id=17297325

The problem with threads: https://ieeexplore.ieee.org/document/1631937

[ Parent | on HN ]

Re:Tell HN: Python's print function is thread-unsafe by ansgri on Monday January 01, 2024 @08:57AM (#38831791)

Given threads are here to stay, thread-unsafe print is a really bad idea. Couldn’t even imagine it can crash instead of interleave, in Python of all things.

[ Parent | on HN ]

Re:Tell HN: Python's print function is thread-unsafe by antman on Monday January 01, 2024 @04:52AM (#38830765)

See this example:

https://superfastpython.com/thread-safe-print-in-python/

Python logging is thread safe if you want to use it for output

[ Parent | on HN ]

Re:Tell HN: Python's print function is thread-unsafe by eesmith on Monday January 01, 2024 @05:02AM (#38830809)

I was able to reproduce your failure (using Python 3.10) after a couple of runs:

  % grep -v '^aaaaaaaaaa$' kerneloops.out | grep -v '^bbbbbbbbbb$' | od -x
  0000000      8130    0db3    0001    0000    bd30    0dad    0001    0000
  *
  0006420      000a
  0006421

I was surprised as I expected possible interleaved bytes but not output other than "a"s, "b"s and new-lines, as I thought the GIL would come to the rescue.

[ Parent | on HN ]