News for hackers. Stuff that matters.
Re:Why I’m Learning Perl 6 by StevePerkins on Wednesday July 26, 2017 @07:52AM (#14855378)
> Concurrency is hard and if you want M:N thread multiplexing (i.e. WEB SCALE CODE, where application threads aren’t pinned to pthreads) your options today are precisely Erlang, Go, .NET, and Perl 6.

Putting aside the "web scale" jokes (http://www.mongodb-is-web-scale.com/), this statement is still absurd.

Every major language, or at least the ones that matter for backend development, has support for thread multiplexing / coroutines / fibers, whatever. Perhaps not in the core language syntax or standard library. But it's easy to implement with native code, and so SOMEONE has implemented it in a library if the language has an FFI.

Java, and all of the other JVM-based languages in turn, have Quasar (http://docs.paralleluniverse.co/quasar/).

Ruby has support for primitive fibers baked into the standard language (https://ruby-doc.org/core-2.1.1/Fiber.html), and likewise community gems with more robust functionality like Quasar.

Python 3 likewise has this out of the box (https://www.python.org/dev/peps/pep-0492/).

The list goes on and on: https://en.wikipedia.org/wiki/Coroutine#Programming_language....

Re:Why I’m Learning Perl 6 by dragonwriter on Wednesday July 26, 2017 @09:11AM (#14855898)
> Every major language, or at least the ones that matter for backend development, has support for thread multiplexing / coroutines / fibers, whatever.

M:N threading is not, from a developer perspective, the same thing as coroutines/fibers. Coroutines are lower-level; it's possible to build something like M:N threading on top of coroutines, but it means doing a lot of work that's already done for you in a language that provides M:N threading out of the box, and if you have to do it yourself, that also means code you use from others isn't going to integrate well with it generally.

Re:Why I’m Learning Perl 6 by ajross on Wednesday July 26, 2017 @01:36PM (#14858284)
> Coroutines are lower-level [than M:N threading]

Not sure I understand what you're trying to say here, this sounds exactly backwards to my reading. A green threading library can be written in assembly language with no reference to anything but the hardware ISA specification and the ABI that defines the relevant calling convention. It's as "low level" as a software construct can be. The only lower level implementation I can think of are hardware-assisted context switching features like you see on some embedded architectures.

Coroutines, on the other hand, are artifacts of the language runtime and depend in sensitive ways on that whole stack. Much higher level.

Re:Why I’m Learning Perl 6 by dragonwriter on Wednesday July 26, 2017 @01:41PM (#14858326)
> Not sure I understand what you're trying to say here

The exact sense of the phrase you excerpted is explained in the next clause of the same sentence of the comment.

Re:Why I’m Learning Perl 6 by ajross on Wednesday July 26, 2017 @02:03PM (#14858541)
OK, then that's backwards, sorry. The fact that A is hard to implement in terms of B doesn't mean that A is a "lower level" (or even "higher level") construct in the sense people usually use those terms. It just means that the two metaphors are incompatible.

Honestly I was asking the question because I thought you had some deeper point in mind I couldn't see. But you just had it backwards.

Re:Why I’m Learning Perl 6 by simias on Wednesday July 26, 2017 @10:55AM (#14856671)
That's kind of the problem when the author uses meaningless words like "WEB SCALE", we don't really know what it means exactly.

If it's about serving a great number of concurrent connections, I'm not sure why M:N is the only way to go.

Re:Why I’m Learning Perl 6 by btilly on Wednesday July 26, 2017 @04:50PM (#14860215)
Most languages give you one of two easy options. The first is to use green threads, which makes 100,000 new threads cheap, but now a single slow function call anywhere can block other threads. The second is to use OS threads, which keeps a slow function call in one thread from interrupting your responsiveness, but now makes 100,000 new threads expensive.

What M:N threading does, at least in theory, is allows you to create 100,000 threads cheaply AND keep a slow function call in one thread from blocking others. In a language with those features you can build it out of continuations (NOT coroutines) and real threads. But that requires building your own scheduler that can move a user thread to a different process at need. Which is non-trivial, and is unlikely to cooperate well with any third party libraries.

That said, whether M:N threading is a real optimization depends on a ton of things. A big one is the underlying performance of the scheduler and the language. Which is improving for Perl 6..but has a long way to go.

Whether optimizing at this level matters is another good question. Most companies shouldn't care. Those that should, should be cautious about deploying Perl 6 into a mature infrastructure.

So that's why people can think that M:N is important, and why you might legitimately disagree.

Re:Why I’m Learning Perl 6 by tomcam on Wednesday July 26, 2017 @08:04PM (#14861525)
Very useful explanation. Thank you.
Re:Why I’m Learning Perl 6 by dragonwriter on Wednesday July 26, 2017 @11:50AM (#14857240)
> That's kind of the problem when the author uses meaningless words like "WEB SCALE", we don't really know what it means exactly.

When the author (as is the case here with M:N threading) identifies the specific feature of interest, we do know exactly what it means, independently of whether they also reference something less precise.

> If it's about serving a great number of concurrent connections, I'm not sure why M:N is the only way to go.

If you want to posit an argument challenging the idea that M:N threading is an advantage, that's a different thing that disagreeing with the authors contention that choices are fairly limited if you want a language with language-level integration of M:N threading.

Re:Why I’m Learning Perl 6 by hashkb on Wednesday July 26, 2017 @11:13AM (#14856853)
Author demonstrates exactly what is meant with code exam in the article
Re:Why I’m Learning Perl 6 by weberc2 on Wednesday July 26, 2017 @10:04AM (#14856244)
This is incorrect. Coroutines and fibers are not m:n threads. There are important implementation and usability distinctions. Besides, there are a lot of additional usability issues with bolted-on concurrency vs language-level concurrency.
Re:Why I’m Learning Perl 6 by Tloewald on Wednesday July 26, 2017 @03:40PM (#14859484)
Ah yes, let's use Perl 6 to avoid usability issues.
Re:Why I’m Learning Perl 6 by weberc2 on Wednesday July 26, 2017 @04:54PM (#14860247)
I wasn't advocating for Perl; I've never used it.
Re:Why I’m Learning Perl 6 by RX14 on Wednesday July 26, 2017 @09:51AM (#14856160)
Unfortunately, unless the IO in the stdlib uses or is modified to schedule these fibers around evented IO, they're essentially useless as soon as you use an external library. Go and Erlang do this, Crystal does this but with only N:1 multiplexing (but that will change before 1.0), but I don't know about other languages.
Re:Why I’m Learning Perl 6 by anonacct37 on Wednesday July 26, 2017 @02:29PM (#14858821)
Very true. Node does this too, in the sense that all node libraries are async/callback aware or have async/callback versions of those.

I much prefer the go/erlang version of this. Playing the "what color is your function" game or "will it block" is no fun.

Re:Why I’m Learning Perl 6 by dmix on Wednesday July 26, 2017 @04:14PM (#14859836)
> Playing the "what color is your function" game or "will it block" is no fun.

This was quite painful with.js Node 1-2yrs back when Promises and coroutines started picking up steam [such as https://github.com/tj/co]. It was always a shot in the dark.

Has this situation changed now? (basically is the stdlib all async friendly now?)

Re:Why I’m Learning Perl 6 by bmn__ on Thursday July 27, 2017 @05:20AM (#14863715)
Yes, insofar it's been possible for a few years to mechanically upgrade stdlib functions marked with the Async suffix into promises with a helper library such as Bluebird. Any promise based function complying with the common promise spec is automatically async friendly, that is to say the async/await keywords do what you expect and the programmer does not need to chain then-ables anymore. Node v8.0.0 adds a promisifier into core for convenience: https://nodejs.org/dist/latest-v8.x/docs/api/util.html#util_...
Re:Why I’m Learning Perl 6 by vishbar on Wednesday July 26, 2017 @06:21PM (#14860907)
.NET has both nonblocking evented IO and standard blocking IO in the stdlib. Most modern libraries use asynchronous IO, however lots of legacy applications still use blocking calls, making them unsuitable to run in the thread pool.
Re:Why I’m Learning Perl 6 by jimbokun on Wednesday July 26, 2017 @02:58PM (#14859082)
Pretty sure the ALL CAPS on WEB SCALE CODE indicates the author is in on the joke.
Re:Why I’m Learning Perl 6 by Rafert on Wednesday July 26, 2017 @01:40PM (#14858318)
> Ruby has [...] likewise community gems with more robust functionality like Quasar.

https://github.com/ruby-concurrency/concurrent-ruby for those wondering. Rails uses it as of version 5.

Re:Why I’m Learning Perl 6 by dullgiulio on Wednesday July 26, 2017 @04:31PM (#14860012)
Concurency doesn't mean multithreading. The "yield" keyword in Python enables a for of concurrency and Python is infamous for not being multi-threaded (when interpreting Python.)
Re:Why I’m Learning Perl 6 by chrisseaton on Wednesday July 26, 2017 @02:24PM (#14858777)
You must be thinking of something else. Concurrent Ruby doesn't include any functionality that is anything like fibres, lightweight threads, or M:N threading (I work on the project).
Re:Why I’m Learning Perl 6 by wolco on Wednesday July 26, 2017 @08:26AM (#14855582)
PHP even has it.
Re:Why I’m Learning Perl 6 by ashayh on Wednesday July 26, 2017 @01:47PM (#14858390)
Ruby also has JRuby, which has true parallelism & concurrency and does not suffer from the global interpreter lock like vanilla ruby/python.
Re:Why I’m Learning Perl 6 by darkr on Wednesday July 26, 2017 @04:38PM (#14860096)
Unfortunately most ruby gems are written with no concept of thread safety. I say this from the perspective of a long time ruby fan, terrified when I hear of critical systems that are running on jruby
Re:Why I’m Learning Perl 6 by czeidler on Wednesday July 26, 2017 @08:36PM (#14861690)
Also see Kotlin's concurrency
Re:Why I’m Learning Perl 6 by carapace on Wednesday July 26, 2017 @06:20PM (#14860904)
The intersection of "needs M:N threading" and "doesn't want to use Go"...
Re:Why I’m Learning Perl 6 by jjtheblunt on Wednesday July 26, 2017 @01:46PM (#14858378)
Nonsense: Solaris has had that thread multiplexing for over 15 years, for one counterexample.
Re:Why I’m Learning Perl 6 by gpderetta on Thursday July 27, 2017 @03:17AM (#14863265)
Didn't solaris end up ripping out M:N threading and switching to pure kernel threading as it was found inadequate as a general pthread implementation?
Re:Why I’m Learning Perl 6 by rootlocus on Wednesday July 26, 2017 @08:19AM (#14855531)
I'd like to know why people are downvoting this.
Re:Why I’m Learning Perl 6 by dragonwriter on Wednesday July 26, 2017 @09:16AM (#14855934)
Probably because it treats a high-level feature (M:N threading) as equivalent to a lower-level feature (coroutines) and then compounds that by treating as equivalent languages where the former is built into the core with ones where the latter is available as a third-party library, all as a way of dismissing the value of languages where the former feature is built in to the core, and layered with scorn and derision on top of being doubly wrong.

Now, I preferred correction to downvoting, but I can easily see why others would choose downvoting.

Re:Why I’m Learning Perl 6 by StevePerkins on Wednesday July 26, 2017 @08:50AM (#14855735)
Ehh... I don't really understand the impulse to "call out" downvotes, or ask people to explain why they're downvoting something that you wrote or agree with.

It's probably just my age, as I started out online when most discussion forums lacked explicit voting mechanisms. Before social media came along, and "gamified" human conversation. Maybe it helps police trolls, but at the cost of encouraging group-think and excessive meme-referencing, and just making a lot of people more neurotic and insecure in general. Overall, I think that online discussion is worse than it was 15 years ago, due to the psychological effects of all the imaginary Internet points.

But anyhoo, at the time I saw this (45 minutes after my original comment), my comment had 25 votes and was the top-most thing on this page. More often than not, if you wait an hour, the crowd straightens out the impact of any early outliers.

Re:Why I’m Learning Perl 6 by rootlocus on Wednesday July 26, 2017 @09:10AM (#14855882)
There's a difference between downvoting because you don't like the tone or don't agree on something subjective and downvoting because something is incorrect or inaccurate. If people downvote because it's incorrect I'd like to know what's incorrect about it.

> But anyhoo, at the time I saw this (45 minutes after my original comment), my comment had 25 votes and was the top-most thing on this page.

It was grayed out at the bottom of the page when I saw it.

Re:Why I’m Learning Perl 6 by reitanqild on Wednesday July 26, 2017 @09:43AM (#14856107)
Ehh... I don't really understand the impulse to "call out" downvotes, or ask people to explain why they're downvoting something that you wrote or agree with.

Sometimes there is some really weird voting dynamic here and it would be interesting to know what goes on.

For a lack of a better word you can call it intellectual curiosity or something.

Re:Why I’m Learning Perl 6 by sanderjd on Wednesday July 26, 2017 @11:49AM (#14857234)
I can't remember the last time I saw a reasonable comment more than a half hour or so old greyed out. But I often see highly-voted comment with an old greyed-out reply saying "why is this being downvoted". All that's happening is that the sample size of votes is too small early on to be meaningful.
Re:Why I’m Learning Perl 6 by flukus on Thursday July 27, 2017 @12:22AM (#14862698)
> But I often see highly-voted comment with an old greyed-out reply saying "why is this being downvoted".

Many times it's because the user was just asking a question, or made what they thought was a perfectly reasonable comment.

HN is pretty horrible with voting etiquette.

Re:Why I’m Learning Perl 6 by reitanqild on Wednesday July 26, 2017 @03:00PM (#14859103)
But I still often wonder about the motivation for those early downvoters.

And I'll still hold that this is acceptable curiosity.

Re:Why I’m Learning Perl 6 by oblib on Thursday July 27, 2017 @12:16AM (#14862671)
I agree with that. I too was curious about why someone downvoted a comment I made the other day. I can honestly say I don't worry about losing a point over that since I've not looked into how points work much but I thought it was odd that someone would do that without explaining what motivated them.
Re:Why I’m Learning Perl 6 by lackbeard on Wednesday July 26, 2017 @10:01AM (#14856219)
Because it not a helpful comment. It's wrong in a lot of ways and misleading in others.

Like dragonwriter said, having coroutines in your language or a fiber library is not close to the same thing as the language having a scheduler that maps those fibers onto native threads for you. Some of the languages mentioned can't even execute multiple threads on multiple cores in parallel.

Re:Why I’m Learning Perl 6 by supremesaboteur on Wednesday July 26, 2017 @08:33AM (#14855628)
For Python there is no native support for threads because of GIL. You can get around this through multiprocessing etc but these are hacks
Re:Why I’m Learning Perl 6 by anonacct37 on Wednesday July 26, 2017 @02:33PM (#14858862)
You might have chosen bad words, but people reading this should know this statement it false. Python has threads. You can create them and (if you're using linux) see them in the /proc filesystem.

A more accurate statement is: "python bytecode can't execute concurrently w/o something like multiprocessing". The key difference? You can do multithreaded I/O all day long, which is a pretty important use case of threading. You can also hand off work to C and release the GIL, although this isn't as common. Example: XML parsing, regex, numeric work.

FWIW all these gotchas are one of the reasons I love just having complete threading support built in to my language.

Re:Why I’m Learning Perl 6 by chrisseaton on Wednesday July 26, 2017 @02:42PM (#14858948)
> A more accurate statement is: "python bytecode can't execute concurrently

No this isn't any more accurate. Python bytecode does execute concurrently if you have multiple threads. It just doesn't execute in parallel.

Re:Why I’m Learning Perl 6 by ensiferum on Wednesday July 26, 2017 @04:46PM (#14860171)
In fact it is quite common. If you look into the c modules you see a pattern where they release the gil before performing some operation and then reacquire it
Re:Why I’m Learning Perl 6 by askvictor on Wednesday July 26, 2017 @08:45AM (#14855697)
There is native support for threads, it's just that they don't run concurrently.
Re:Why I’m Learning Perl 6 by nothrabannosir on Wednesday July 26, 2017 @09:53AM (#14856171)
At this point the word is so far removed from the meaning in this thread you might as well say a haberdasher supports threads.
Re:Why I’m Learning Perl 6 by askvictor on Wednesday July 26, 2017 @04:53PM (#14860241)
There are valid reasons for programming with threads with only a single CPU (indeed when I learnedb thread programming in the 90s pretty much only single core cpus were available). For instance, IO blocking.
Re:Why I’m Learning Perl 6 by dragonwriter on Wednesday July 26, 2017 @12:46PM (#14857836)
They do run in parallel, so long as all but one are running low-level library code which has released the GIL, but you can't call back into the runtime without grabbing the GIL.
Re:Why I’m Learning Perl 6 by weberc2 on Wednesday July 26, 2017 @10:19AM (#14856345)
While we're being pedantic, they do run concurrently (for example, dispatching a dozen concurrent HTTP requests), they just the GIL prevents most CPU work from executing in parallel.
Re:Why I’m Learning Perl 6 by e12e on Wednesday July 26, 2017 @09:14AM (#14855921)
Afaik erlang has a lock as well - in fact I think most green thread implementations do: what you want/need at a minimum is one os process or thread per cpu core that works as a scheduler and then some form of (co-operative) scheduling with only user-mode/low-overhead context switching?
Re:Why I’m Learning Perl 6 by brightball on Wednesday July 26, 2017 @09:47AM (#14856134)
Scheduling and locking aren't the same thing. Here's a brief excerpt from Joe Armstrong that I found on SO while looking for a good explanation.

> [Erlang] is a concurrent language – by that I mean that threads are part of the programming language, they do not belong to the operating system. That's really what's wrong with programming languages like Java and C++. It's threads aren't in the programming language, threads are something in the operating system – and they inherit all the problems that they have in the operating system. One of the problems is granularity of the memory management system. The memory management in the operating system protects whole pages of memory, so the smallest size that a thread can be is the smallest size of a page. That's actually too big.

> If you add more memory to your machine – you have the same number of bits that protects the memory so the granularity of the page tables goes up – you end up using say 64kB for a process you know running in a few hundred bytes.

With Erlang, there are threads running concurrently across multiple CPU's and a preemptive scheduler within each of those threads. This is all transparent through use of Erlang processes. Essentially, many concurrent schedulers managing hundreds of thousands of processes that start out at 0.5kb each. A JVM thread is 1024kb by comparison and a goroutine is 2kb.

You're getting legitimate concurrency with Erlang, as well as isolation. There are only so many things that a single CPU can do at the same time and that's where the scheduler becomes necessary. The scheduler also ensure that if you fire off 100,000 processes and one of them is CPU intensive that it can't hog the processor and preventing the others allocated to that processor from executing.

Re:Why I’m Learning Perl 6 by yellowapple on Wednesday July 26, 2017 @09:34AM (#14856046)
That doesn't sound right re: Erlang; its everything-is-immutable model is supposed to avoid the need for a GIL. Erlang processes are also preemptive, so there's no possibility for a process to lock up the whole VM (unless it calls into a long-running NIF, but native code is always dangerous in a "crash BEAM if you don't get it right" kind of way).
Re:Why I’m Learning Perl 6 by dragonwriter on Wednesday July 26, 2017 @12:48PM (#14857858)
IIRC, Erlang uses shared mutable state internally, though its not exposed, so it needs to lock around accesses to that, but this isn't equivalent to the Python GIL.
Re:Why I’m Learning Perl 6 by pjmlp on Wednesday July 26, 2017 @08:39AM (#14855661)
Only relevant on CPython, other implementations don't have a GIL.
Re:Why I’m Learning Perl 6 by gebruikersnaam on Wednesday July 26, 2017 @08:47AM (#14855706)
Re:Why I’m Learning Perl 6 by pjmlp on Wednesday July 26, 2017 @09:47AM (#14856127)
Ok, but ZipPy, Jython and IronPython do not.
Re:Why I’m Learning Perl 6 by weberc2 on Wednesday July 26, 2017 @10:23AM (#14856371)
Have Jython and IronPython reached compatibility with Python 3 yet? Last I checked they hadn't, or it was still in alpha.
Re:Why I’m Learning Perl 6 by throwaway7645 on Wednesday July 26, 2017 @01:10PM (#14858069)
Yes, but ain't nobody really using Jython or IronPython. Yes some people are, but I'd bet a very small group of the greater Python whole. You also introduce incompatibilities that way. Perl6 has this in the base implementation, so I don't have to switch to a lesser maintained distribution.
Re:Why I’m Learning Perl 6 by pjmlp on Wednesday July 26, 2017 @10:34AM (#14856483)
I don't know, but Python 3 still isn't relevant to many people.

For example, I have to explicitly install 2.7 to be able to use Cocos2d-x build scripts.

Re:Why I’m Learning Perl 6 by weberc2 on Wednesday July 26, 2017 @11:10AM (#14856814)
Maybe, but it seems like a bad idea to invest in Python 2 (which is teaching is ended of life) just to get parallelism when there are other programming languages that have great concurrency stories and a bright future. That said, if PyPy3 ditches the GIL, let me know!
Re:Why I’m Learning Perl 6 by throwaway7645 on Wednesday July 26, 2017 @01:13PM (#14858103)
Someone posted an article awhile back...people are moving over by a significant amount now.
Re:Why I’m Learning Perl 6 by sli on Wednesday July 26, 2017 @06:02PM (#14860772)
Yeah, "not relevant to many people" is a wildly false claim. If it were anywhere near true, we wouldn't see the level of Python 3 support that we have in the major packages (95.8%).

http://py3readiness.org/

Re:Why I’m Learning Perl 6 by dragonwriter on Wednesday July 26, 2017 @12:57PM (#14857942)
Jython project's latest release and news post was mid-2015; the project would seem to be pining for the fjords, so to speak.

Jython and IronPython are both limited to Python 2.x, ZipPy is WIP without a stable release.

Re:Why I’m Learning Perl 6 by reubano on Thursday July 27, 2017 @02:26AM (#14863074)
But not PyPySTM
Re:Why I’m Learning Perl 6 by kafkaesq on Wednesday July 26, 2017 @03:20PM (#14859283)
I'd like to know why people are downvoting this.

Because for some people, it's not enough to think of others as misinformed, or not so well-spoken.

They need to be thought of as blundering idiots, and castigated accordingly.

Re:Why I’m Learning Perl 6 by thehardsphere on Wednesday July 26, 2017 @08:31AM (#14855610)
People often downvote anything that's remotely negative, even if it is true or merely a critical question.