News for hackers. Stuff that matters.
Re:Erlang/OTP 20.0 by rdtsc on Wednesday June 21, 2017 @12:47PM (#14605451)
Some things I like from the new release:

* Dirty schedulers: This allows easy integration of blocking C-based libraries. So for example can wrap something like RocksDb and make it available to the rest of the VM easier. Or libcurl and others.

* DTLS : This lets it talk to WebRTC clients

* Erlang literals are no longer copied when sending messages : This is kinda of a sneaky one. By default (with exception of large binaries) Erlang VM usually copies data when it sends messages. However, in this case module literals (constants, strings, etc) will be another thing that's not copied. There is a hack to dynamically compile configuration values or other tables of constants as a module at runtime. So if you use that hack, you'd get a nice performance boost.

* code_change, terminate and handle_info callbacks optional in the OTP behaviors. This is very nice. I always wondered why I had to write all that boiler plate code.

Also here is a detailed list of changes:

http://erlang.org/download/otp_src_20.0-rc2.readme

Re:Erlang/OTP 20.0 by equalunique on Wednesday June 21, 2017 @02:11PM (#14606400)
> * Dirty schedulers: This allows easy integration of blocking C-based libraries. So for example can wrap something like RocksDb and make it available to the rest of the VM easier. Or libcurl and others.

Amazing. I'm new to erlang, but within the last hour just read comments by a seasoned programmer on how erlang's shiny scheduler has an achilles heel when it comes to blocking C-based libraries. I suppose then that this truly is a significant advancement.

Mention of the scheduler was in a comparison between erlang and node.js. Article here: https://notamonadtutorial.com/interview-with-jesper-louis-an...

Re:Erlang/OTP 20.0 by rdtsc on Wednesday June 21, 2017 @02:49PM (#14606800)
> how erlang's shiny scheduler has an achilles heel when it comes to blocking C-based libraries.

It's an improvement but it was already possible to write C-based libraries. There are drivers, ports, NIFs were there before but for long running computation had to do your own thread and queue setup. This just avoids that bit because VM writers did it for the user so to speak (and probably did the right way).

Also it is really something advanced and not what most Erlang programmers would end up doing anyway. Once you start writing C code and directly loading it into the VM, the same caveats as before apply - that is a some fault tolerance and safety guarantees go out of the window.

Re:Erlang/OTP 20.0 by sausman on Wednesday June 21, 2017 @07:39PM (#14608982)
Yeah, the only thing I've used it for thus far is hashing passwords e.g. https://github.com/riverrun/argon2_elixir
Re:Erlang/OTP 20.0 by brightball on Wednesday June 21, 2017 @04:38PM (#14607698)
Exactly. Half the point of the beam is not having that type of code impact the scheduler.

Case in point, you can write a faster JSON parser in C and use it (jiffy)...but it's not desirable to pollute your BEAM for the minor performance gain.

Re:Erlang/OTP 20.0 by fredliu on Wednesday June 21, 2017 @01:08PM (#14605716)
* DTLS : This lets it talk to WebRTC clients

Could you elaborate more on this? I don't know much about erlang/OTP's support for WebRTC. Does adding DTLS makes erlang/OTP possible to talk to WebRTC clients because WebRTC requires DTLS? or this means erlang/OTP now has a complete stack and API for WebRTC communication out of the box?

Re:Erlang/OTP 20.0 by rdtsc on Wednesday June 21, 2017 @01:26PM (#14605941)
DTLS is TLS (SSL) but for datagrams (UDP) instead of the traditional streams (TCP).

WebRTC uses DTLS as one of its base protocols. The other one is SRTP but even its setup requires DTLS.

I am not sure how complete the rest of the support it but having DTLS is a major step ahead.

Re:Erlang/OTP 20.0 by fredliu on Wednesday June 21, 2017 @01:55PM (#14606248)
Thanks for the clarification. I was aware of the DTLS requirement for WebRTC, but not sure the whole WebRTC stack support on erlang/OTP. Now thinking about it, maybe an erlang/elixir/whatever-on-BEAM implementation of STUN/TURN server would make sense...
Re:Erlang/OTP 20.0 by scandox on Wednesday June 21, 2017 @05:40PM (#14608192)
I thought that the STUN/TURN stuff was just for discovery and used fairly conventional protocols...Isn't the DTLS for actual media transmission - i.e. directly between clients (or via some kind of Gateway?)
Re:Erlang/OTP 20.0 by fredliu on Wednesday June 21, 2017 @05:54PM (#14608288)
STUN is for signaling, but the hole-punching business is not 100% guaranteed to work, so you need TURN as the back up relay server. DTLS is needed for both p2p traffic and the replaying traffic through TURN, so your TURN server does need DTLS for its job.
Re:Erlang/OTP 20.0 by zurn on Thursday June 22, 2017 @01:33AM (#14610284)
STUN/TURN are just NAT traversal, they are building blocks in WebRTC signaling.
Re:Erlang/OTP 20.0 by _asummers on Wednesday June 21, 2017 @02:33PM (#14606642)
ETS CAS is a huge one, too!
Re:Erlang/OTP 20.0 by bostonvaulter2 on Wednesday June 21, 2017 @08:18PM (#14609163)
Can you explain what that is for the uninitiated?
Re:Erlang/OTP 20.0 by greenleafjacob on Wednesday June 21, 2017 @10:16PM (#14609656)
ETS is a fast [1] in-memory K/V store including a set type with constant time put/get in the standard library of Erlang. New in OTP 20 is an atomic CAS (compare-and-swap) operation.

[1] http://www.erlang-factory.com/static/upload/media/1459269312...

Re:Erlang/OTP 20.0 by wcummings on Wednesday June 21, 2017 @02:31PM (#14606618)
I thought dirty NIFs were available since like 17 or 18. I know I wrote some code that used them a couple of years ago.
Re:Erlang/OTP 20.0 by marianoguerra on Wednesday June 21, 2017 @02:43PM (#14606742)
They were marked as experimental. Now they are marked as stable
Re:Erlang/OTP 20.0 by mtanski on Wednesday June 21, 2017 @01:09PM (#14605726)
> * Dirty schedulers: This allows easy integration of blocking C-based libraries. So for example can wrap something like RocksDb and make it available to the rest of the VM easier. Or libcurl and others.

Silly question, but shouldn't elang have some good & highly concurrent http libraries?

Re:Erlang/OTP 20.0 by micmus on Wednesday June 21, 2017 @01:27PM (#14605947)
Dirty schedules are about functions implemented in C - called NIFs (Native Implemented Functions) - a sort of FFI you'd find in most languages.

Because of preemptive nature of Erlang, doing lengthy work in those functions can destabilise the system (the "magic value" is said to be around 1ms). That is because C functions can't be preempted in the middle of execution like Erlang ones can.

Using dirty schedulers lifts this time limitation, but gives a higher constant overhead when calling a function on a dirty scheduler, since it means switching OS thread. This tradeoff, however, is perfectly acceptable for a lot of cases.

And yes, Erlang has good & highly concurrent HTTP libraries implemented in Erlang.

Re:Erlang/OTP 20.0 by eugene_pirogov on Wednesday June 21, 2017 @01:34PM (#14606019)
A question regarding dirty schedulers: does it still mean that when called C code crashes during execution, entire Erlang VM will crash as well?
Re:Erlang/OTP 20.0 by micmus on Wednesday June 21, 2017 @01:45PM (#14606140)
Yes. That's not changed. All NIFs (regular or dirty ones) are executed directly in the context of the VM. A safe option would be a port - a regular program where you communicate through stdin/stdout. Ports allow representing such program (running in a separate OS process) as something equivalent to a native Erlang process.

There's also work on supporting writing NIFs in Rust, which gives some degree of additional safety. The relevant project would be: https://github.com/hansihe/rustler

Re:Erlang/OTP 20.0 by mononcqc on Wednesday June 21, 2017 @01:45PM (#14606145)
Yes. For such cases, you'll want to instead use a C-Node or a Port program to interface with the C code.
Re:Erlang/OTP 20.0 by rdtsc on Wednesday June 21, 2017 @01:28PM (#14605960)
Oh it does there are already a good number of decent ones. I just used libcurl as an example. And as someone suggested one reason to wrap libcurl could be is because it supports a lot of the corner cases and protocols.
Re:Erlang/OTP 20.0 by puzza007 on Wednesday June 21, 2017 @11:52PM (#14610012)
I wrapped libcurl as a port https://github.com/puzza007/katipo. It would be interesting to see how much faster it could be made as a NIF.
Re:Erlang/OTP 20.0 by lpgauth on Wednesday June 21, 2017 @03:32PM (#14607106)
Yes, you can see some HTTP client benchmarks here.

Disclaimer: I wrote the Buoy (https://github.com/lpgauth/buoy)

Re:Erlang/OTP 20.0 by tomjakubowski on Wednesday June 21, 2017 @01:23PM (#14605896)
libcurl supports some other protocols, too. Maybe Erlang lacks good and highly concurrent Gopher libraries. :-)