News for hackers. Stuff that matters.
Re:Let's Remove the Global Interpreter Lock by kllrnohj on Monday August 14, 2017 @12:26PM (#15009897)
#1 & #2: Consumer CPUs are now pushing 16 cores & 32 threads. Python is limited to ~1/20th of what a single box is capable of. That's a pretty big bottleneck.

#4: Even if you're just talking message passing sending a message between threads is in the 10s of nanoseconds while between processes is 10s of microseconds. That's a ~1000x slowdown on core communication. Given that CPU cores are not getting any faster, that's a pretty big hit to efficiency to take. Similarly simply moving data between processes is expensive, while moving data between threads is free.

Re:Let's Remove the Global Interpreter Lock by breatheoften on Monday August 14, 2017 @02:15PM (#15011015)
Moving data between threads is only free to the extent that synchronization is free. Maybe you could say that moving immutable data between threads is free but I don't think you can say its free in general ... Doing so significantly undersells the complexity that comes with shared memory concurrency.
Re:Let's Remove the Global Interpreter Lock by kllrnohj on Monday August 14, 2017 @04:01PM (#15012281)
You seem to be conflating moving with sharing. Moving between threads is always free[1] regardless of if it's mutable or immutable, and there's no concurrency issues at all since it's a move.

Move means the sender no longer has a reference. As in, std::move, rust's ownership transfer, webworker's transferables, etc...

1: Yes there's a single synchronize point where the handoff happens, but this is part of sending a message at all. It's also independent to the size & complexity of the payload itself when we're talking multi-threaded instead of multi-process. You have that exact same sync point that costs the exact same regardless of whether your message consists of a single byte or a multi-gigabyte structure.

Re:Let's Remove the Global Interpreter Lock by breatheoften on Monday August 14, 2017 @08:34PM (#15014358)
Ah I see -- Can you actually describe ownership in python sufficiently well to be able to describe this move operation for any useful python data structures?
Re:Let's Remove the Global Interpreter Lock by kllrnohj on Tuesday August 15, 2017 @10:51AM (#15018702)
Generically ownership is simply who has a reference to the object.

So if a.foo = b, then a 'owns' b. A 'move' is simply handing a different object the reference, then dropping your own reference. For example:

  a.foo = b // a 'owns' b
  c.foo = a.foo // a & c share b, however if the next line is:
  a.foo = None // a has 'moved' b to c, since c now has the only reference to b.
Some languages have codified this to make the contract part of the language, but it doesn't need any first-class language support. It's just a pattern at the end of the day.
Re:Let's Remove the Global Interpreter Lock by sqeaky on Monday August 14, 2017 @12:50PM (#15010176)
You are right about the majority of what you said, but I am pedantically picking on one point. CPU cores are getting faster, but they aren't doing it with clock speed, they are dispatching more instructions per cycle or otherwise making the work faster.
Re:Let's Remove the Global Interpreter Lock by kllrnohj on Monday August 14, 2017 @01:23PM (#15010483)
IPC gains per generation are vanishingly tiny, if they exist at all. Skylake -> Kaby Lake, for example, had no IPC improvements at all. A very small clock bump to the various tiers was it.

Even if you look over a large generation gap there's only a ~20% IPC improvement going from an i7-2600K to an i7-7700K ( https://www.hardocp.com/article/2017/01/13/kaby_lake_7700k_v... )

6 years & a shrink from 32nm to 14nm and all it can muster is +20%. Cores are just not getting faster by any meaningful amount.

Re:Let's Remove the Global Interpreter Lock by sqeaky on Tuesday August 15, 2017 @10:22PM (#15024496)
My compiles have gotten more than 20% faster so something is making my newere machines faster than my older machines.

That it is not 10x as fast I blame on AMD for not being as competitive as they could have been.

Re:Let's Remove the Global Interpreter Lock by happycube on Monday August 14, 2017 @09:30PM (#15014618)
Yeah - I remember when five year old CPUs were basically useless!

(Kaby Lake is basically a new stepping of Skylake - if intel wasn't having problems with new process nodes it likely wouldn't have been released at all, and if it was it would've been used for a one-off chip in the same generation ala the 4770K)

Re:Let's Remove the Global Interpreter Lock by jononor on Monday August 14, 2017 @12:57PM (#15010251)
The efficiency hit is very dependent on how large your computation chunks are. If the computation per message batch is on order of 100 ms, it would be <10% loss.
Re:Let's Remove the Global Interpreter Lock by kllrnohj on Monday August 14, 2017 @01:27PM (#15010519)
Assuming very small messages that are rarely sent then yes, the hit of multi process is not going to be your biggest issue.