News for hackers. Stuff that matters.
Re:A Lua 5.3 VM and compiler written in Go by bpizzi on Tuesday October 03, 2017 @02:54AM (#15390675)
Slightly related: I'm in the process of choosing the best way to give our Go binaries some scripting capabilities (it's entreprise stuff, the idea is "let's the customer script this behaviour").

I'm aware of Lua and JS interpreters, some fully native, some made of binding against existing interpreters. Native JS interpreters have my preference right now: easier to build/maintain, I can bear the loss of perfs, and JS seems easier to sell than Lua (remember it's entreprise stuff).

Does anyone have some insight or experience in that area and care to give feedback to a fellow HNer? And did I miss other scripting languages?

Re:A Lua 5.3 VM and compiler written in Go by Derbasti on Tuesday October 03, 2017 @05:40AM (#15391246)
Lua is incredibly easy to embed. The whole Lua interpreter including the standard library is about 150kb. That's Kilo, not mega! You can swap out the memory allocator or the number type on one simple header file. Lua was built specifically for embedding! What is more, it is a joyously well designed language that is well suited for building DSLs. And it is pretty fast and memory efficient for a dynamic language.

Other options are TKL, but the syntax is a bit unorthodox. Or Python, but it is much bigger, and harder to embed. JS is not a great fit, I would say, mostly because it is comparatively HUGE and not particularly optimized for embedding.

Re:A Lua 5.3 VM and compiler written in Go by hnlmorg on Tuesday October 03, 2017 @06:18AM (#15391356)
Perl 5 is another option. There's a few embedded Perl libraries out there. But Perl has a bad reputation (undeservingly in my personal opinion) so potentially another hard sell for enterprise.
Re:A Lua 5.3 VM and compiler written in Go by hnlmorg on Wednesday October 04, 2017 @03:44AM (#15398898)
I guess it's too much to ask for whoever downvoted my comment to explain why they did? Otherwise it's going to look little more than a knee-jerk reaction from some Perl hater.
Re:A Lua 5.3 VM and compiler written in Go by wahern on Thursday October 05, 2017 @03:58AM (#15407348)
I didn't downvote you, but IME Perl is a poor language for embedding. Perl XS is itself pretty bad, whether embedding a C application with Perl, extending a Perl application with C, and especially if you're doing C->Perl->C->Perl mixes--the Perl XS interfaces quickly become leaky abstractions. And don't even get me started on the build process....

I've written far more than my fair share of Perl XS code over the past nearly 15 years. I still use Perl and the last time I wrote Perl XS module was just last month. I could go on for pages about the negative aspects, without having much to say about the positive aspects.

But here's a very concrete reason why nobody should contemplate embedding Perl: I filed this bug report (Thread race in dist/IO/IO.xs) over two years ago:

  https://rt.perl.org/Public/Bug/Display.html?id=125685
It's still not fixed. The fix isn't easy, either. It was introduced in an attempt to fix some other stuff. But maybe the biggest reason it hasn't yet been fixed is because the bug is only likely to be encountered when creating multiple Perl interpreter instances in separate threads--a very uncommon usage for Perl.[1] Uncommon usages will not be (and are not) well supported.

Non-threaded Perl embedding may be more common, but ultimately you don't want to be embedding any interpreter that has shared state, especially if the fact of this shared state leaks into interfaces or breaks the interface contracts. Python is also still problematic in terms of not being able to completely erase vestigial assumptions of one interpreter per process. Neither Perl nor Python were originally designed for embedding, and even their extension APIs were bolted on. While they've both been improved over the years and work well for straight-forward C bindings, the fact that neither the languages nor implementations were originally designed for embedding eventually comes through and cause headaches when you want to do anything sophisticated--sometimes even when you're not doing anything sophisticated.

[1] The only significant use of Perl in the open source world that I'm familiar with that embeds Perl this way (multiple instances, multiple threads) is Apache mod_perl. But for other reasons mod_perl creates and initializes all Perl instances from a single thread. This is why the bug went undiscovered for so long after it was introduced, and why there's little impetus to fix it.

Re:A Lua 5.3 VM and compiler written in Go by hnlmorg on Saturday October 07, 2017 @03:35AM (#15422755)
I don't think it is fair to compare XS with other embeddable languages because you wouldn't expect to use C with them under those circumstances either. Granted performance would be subpar without XS but if that is your primary concern then you'd write your pluggins I'm C and do away with the embedded scripting interface entirely.

Threading is an interesting point. It's worth noting this issue exists with quite a few scripting languages though and the way Apache gets around it (eg with mod_perl, which is garbage by the way) is by forking rather than threading (I mean you can obviously run Apache with threading if you can guarantee all your XS code is thread safe, but it's safer to just prefork)

Re:A Lua 5.3 VM and compiler written in Go by Yaggo on Wednesday October 04, 2017 @06:58AM (#15399569)
I didn't downvote you, but I'd like to know what makes Perl a good option for embedded language and what implementations are available.
Re:A Lua 5.3 VM and compiler written in Go by hnlmorg on Wednesday October 04, 2017 @09:10AM (#15400184)
Like with Python, Javascript, etc Perl has got a huge and collaborative eco-system (CPAN is the godfather (in terms of age) of npm / pip / et al). It's also terse enough that writing scripts doesn't become a bigger job than it needs to be. The language also offers better type safety than Javascript and most sysadmins are already proficient in Perl because, until recently, it used to be the goto scripting language for servers (these days it seems to be more Python) so Perl makes some sense for enterprise systems.

There are big drawbacks for using Perl though. It's appallingly bad for writing OO code (objects in Perl are really just hashes / maps) so larger code bases or complex code can quickly become a pain to maintain. While it has one of the better approaches for comparing data of different types (imo), Perl still has more than it's fair share of hidden traps.

Ultimately though, it's a better language than many give it credit for even if it is considered old fashioned these days. So if languages like Python and Javascript are to be considered then I think Perl deserves a mention as well.

As for implementations, the standard Perl interpreter can be linked to which makes it very easy to embed into other programs.

Re:A Lua 5.3 VM and compiler written in Go by cheaprentalyeti on Tuesday October 03, 2017 @07:44PM (#15397190)
TKL? Is that something new?
Re:A Lua 5.3 VM and compiler written in Go by nicpottier on Tuesday October 03, 2017 @07:54AM (#15391750)
Would love if there was another option between those two as well. I know lots of people love Lua but I find it different enough that using it as an embedded scripting language is annoying. IE, I'm not programming in it all the time, so things like 1-indexing of arrays never becomes feels natural and I make indexing mistakes. Other differences like arrays and dictionaries being the same type cause other context switches which I dislike. (and can cause super subtle bugs in JSON marshaling and unmarshaling if you aren't careful)

I would really love a fast, lightweight stripped down "C Flavored Syntax" dynamic language to use instead of JS or Lua but I haven't found either yet.

Re:A Lua 5.3 VM and compiler written in Go by tangent128 on Tuesday October 03, 2017 @11:06AM (#15392892)
Have you encountered Squirrel yet? It seems to be targeting a "C-flavored Lua" niche: http://www.squirrel-lang.org/
Re:A Lua 5.3 VM and compiler written in Go by acqq on Tuesday October 03, 2017 @09:49AM (#15392370)
I have almost the same feeling as you: I'd love most of the Lua more with zero indexing and more of C like syntax (without these "end" literals, I really like C math-formula-like minimalism). I don't care however if arrays and dicts are the same behind the scenes.
Re:A Lua 5.3 VM and compiler written in Go by nicpottier on Tuesday October 03, 2017 @10:01AM (#15392449)
Ya the weird case we ran into with the dict / list was when unmarshaling some JSON which had an empty list `[]` then marshaling it back to JSON you end up with `{}`. Which is pretty broken and given the prevalence of JSON as an on the wire format is seriously disappointing.

Not sure if there's an easy fix to that apart from using `null` instead of empty lists.