News for hackers. Stuff that matters.
Re:Hello Lua by tomcam on Friday April 29, 2016 @12:37AM (#11593786)
Haxe is one of the best kept secrets of the programming world. Created by none other than the author of curl, it's A C-ish, javascripty, but strictly typed language with almost infinite portability. It "compiles" to many different target languages, from PHP to Java to Python to C++ and more. It's fast, has a beefy standard library, well-documented system interfaces, and just feels fun. A remarkable accomplishment.
Re:Hello Lua by jdonaldson on Friday April 29, 2016 @12:45AM (#11593812)
I agree completely! ... Well, almost agree completely.

Daniel Stenberg is the author of curl: https://daniel.haxx.se/

Nicolas Cannasse is the (main) author of haxe: https://medium.com/@ncannasse

Both are awesome in their own right.

I wrote the Lua target announced here, and am happy to answer any questions.

Re:Hello Lua by tomcam on Friday April 29, 2016 @11:46AM (#11596457)
Thanks for the authorship correction! Sorry, Daniel Stenberg, you complete stud!
Re:Hello Lua by veli_joza on Friday April 29, 2016 @04:57AM (#11594505)
Can you explain who is the target audience for Lua target? What can I do with it?
Re:Hello Lua by i336_ on Friday April 29, 2016 @08:37AM (#11595282)
Well, with LuaJIT you get speed.

How much speed?

CloudFlare's firewall system uses LuaJIT to analyse approximately 4 million GET requests per second.

(Small pause, reread)

That much speed. That was the figure in June 2015, at least.

Some of the autogenerated code looks like this: http://i.imgur.com/F5aT0NF.png (snagged via pdfimages from http://www.slideshare.net/jgrahamc/lua-the-worlds-most-infur...)

This video was where I had my major TIL moment: http://www.thedotpost.com/2015/06/john-graham-cumming-i-got-...

I also got the impression from the one other page of autogenerated code briefly visible in this video that the firewall logic is also full of regular expressions. And yet, it completes its analysis in roughly less than 1 millisecond.

It's almost a crime I haven't learned LuaJIT yet. :P

That said, looking at the slideshare link, you can see that Lua as a language has some pretty annoying syntax, so I've sorta been shying away from it.

So maybe I'll learn HaXe instead. :D

Re:Hello Lua by delroth on Friday April 29, 2016 @09:22AM (#11595507)
> CloudFlare's firewall system uses LuaJIT to analyse approximately 4 million GET requests per second.

While I agree with your premise (LuaJIT is a pretty awesome project), this example means nothing in terms of performance.

— "analyse" doesn't give an indication of the workload -- it could vary from "matching against a precompiled regexp" to running advanced statistical models.

— 4 millions requests per second does not mean anything without knowing the size of the deployment. You can always throw more hardware at a "trivially" parallelizable problem.

— Throughput is only one dimension on the "speed" continuum, latency (and especially tail latency) is also very important. Some languages have very good throughput and fail hard at tail latency (Java, Go until recently). Some others have lower throughput but more predictable latency.

Re:Hello Lua by i336_ on Friday April 29, 2016 @09:49AM (#11595678)
You make incredibly fair points and I don't have the data to answer what you've raised. This could be lack of research or the fact that CF hasn't released it. I expect a 20/80 non-researched/non-available mix.

My understanding is that the WAF detects trivial stuff like SQL injection and XSS, and the video also mentions it detected ShellShock and so forth (yeah, this was a while ago).

"Thousands of regular expressions" is mentioned verbally in the video around ~11:25. That's where I got that from.

There are some bits mentioned about the hardware around 19:30:

- E5-2630v3 (16 cores, 32 w/ HT which is turned off), 128GB RAM, 10Gbps

- Custom built by Quanta: 4 nodes per 2U (each node is like half-width and 1U high), redundant 2xPSU in the middle

- 40 machines run Kafka storing 400TB of log data (ingest @ 15Gbps), 50TB disk per node

- 5 machines run CitusDB (sharded PostgreSQL) to power realtime graph generation, 12TB SSD per node

- 100+ machines for realtime analytics running Go (and presumably LuaJIT) - LuaJIT seems to be the realtime "hmm, interesting" flagging system, while Go is used for in-depth intelligent analysis that's (I assume) slightly behind realtime.

Also, the video is actually over here - https://www.youtube.com/watch?v=LA-gNoxSLCE

---

I'm curious to understand what tail latency is, and very interested to learn more about language latency and performance in general. What subjects would you recommend I learn about / read? (I would describe myself as interested from a hobbyist-intermediate standpont, but from a non-academic POV; I unfortunately get dismally lost with formulaic presentations, math is not a strong subject for me.)

Re:Hello Lua by signa11 on Friday April 29, 2016 @12:53PM (#11596994)
> While I agree with your premise (LuaJIT is a pretty awesome project), this example means nothing in terms of performance.

Yup, without knowing deployment details etc this number doesn't really mean much.

perhaps you might have heard of the snabb-switch project ? which does line rate 10gbps forwarding of 64b packets on a single x86 core. entire thing is written in lua including the user land 'drivers' for Intel 10g cards :)

Edit: s/shabby/snabb (damn you autocorrect)

Re:Hello Lua by i336_ on Saturday April 30, 2016 @06:45AM (#11601203)
Wait, wait. Lua or LuaJIT? That's amazing!
Re:Hello Lua by signa11 on Monday May 02, 2016 @02:39AM (#11609538)
> Wait, wait. Lua or LuaJIT? That's amazing!

from 'user' perspective you are still writing lua code. lua-jit would be an optimizing compiler doing it's magic no ? get the code from here : https://github.com/snabbco/snabb

Re:Hello Lua by i336_ on Monday May 02, 2016 @06:27AM (#11610347)
The README says it uses both Lua and LuaJIT, and I can see clear references to LuaJIT in the codebase.

I'd heard of snabb a little while ago (I think in the context of Erlang), but this only makes it even more interesting. Thanks for the pointer!

Re:Hello Lua by mateuszf on Friday April 29, 2016 @05:48AM (#11594662)
My guess would be for current lua embedding scenarios - for example for use as a scripting language in games.
Re:Hello Lua by knotz on Friday April 29, 2016 @09:23AM (#11595513)
You can build a pretty damn good simulation of a Boeing 737-300 with it. (http://ixeg.net/)
Re:Hello Lua by vidarh on Friday April 29, 2016 @02:09AM (#11594031)
Meanwhile, Daniel also did a C-like language back in the day: FPL. It's never seen widespread use, but was the macro-language for his editor, FrexxEd, for the Amiga.
Re:Hello Lua by trevex on Friday April 29, 2016 @06:09AM (#11594723)
Lua is still my goto embedded scripting language. I don't have any experience with haxe per se, but is the transpiler embeddable as well?
Re:Hello Lua by yoklov on Friday April 29, 2016 @09:44AM (#11595645)
The compiler is written in ocaml, so I'd guess not.
Re:Hello Lua by PeCaN on Friday April 29, 2016 @01:20AM (#11593890)
Haxe is what ActionScript should have been. However (imo), it's hard to find a niche for it now. What does it get me that Perl and OCaml don't? I've been drawn to Haxe several times over the years, but have never really thought of a project, "huh, Haxe would be a great fit here". Maybe if I wrote a mobile game or something.
Re:Hello Lua by jdonaldson on Friday April 29, 2016 @01:59AM (#11593989)
Haxe is really a formidable little virus. It does a lot of small things nicely, but doesn't have a single knock-out feature that you can trot out during language comparisons.

However, all those little features add up over time and become addictive. You start really missing them when you go back to other languages. In the back of your mind you think of how you might solve something simply with abstracts or macros, or how you might easily reduce the file size or execution speed with dead code elimination or inlining.

I would choose Haxe over Perl because of Haxe's expressive type extension capabilities. I could add useful methods to string or array types, and make the resulting code short and sweet (although perhaps less readable for others).

I would choose Haxe over OCaml and I would gain Haxe's superior compiler error messages. I would also get to move seamlessly between functional and imperative programming styles, and still get to use ADTs.

Re:Hello Lua by catnaroek on Friday April 29, 2016 @03:06AM (#11594197)
OCaml accommodates imperative programming just fine: `if` without `else`, `for` loops, single semicolons as statement delimiters (without enclosing statement blocks in parentheses), records and objects containing mutable data, etc. OCaml isn't like SML and Haskell, which <strike>do get in your way when you want to program imperatively</strike> push you a little bit more forcefully towards programming in a functional style.

On the other hand, Haxe lacks features like higher order modules and signature ascription, which give the programmer very fine-grained control over the scope of abstractions. Haxe's abstract types are more like what ML had in the 70's, before MacQueen invented the ML module system.

Re:Hello Lua by jdonaldson on Friday April 29, 2016 @03:25AM (#11594261)
OCaml provides some nice syntactic sugar for imperative programming patterns like "if" statements. But, I wouldn't really call it an imperative language, as it has its roots firmly in ML (which is not such a bad thing after all).

Arguing that ML had a precursor to Haxe abstract types is well within reason, but I'm not so sure they're the same. I'll have to do some more reading.

Re:Hello Lua by daurnimator on Friday April 29, 2016 @01:55AM (#11593977)
The reason we started using it at work was we could target both Javascript and Flash from the one codebase.

However, we finally don't need to support flash anymore... so now there's no reason to not just write in javascript.

Re:Hello Lua by throwaway6845 on Friday April 29, 2016 @04:53AM (#11594496)
I find Javascript is quite a good reason not to write in Javascript, but YMMV...
Re:Hello Lua by sitkack on Friday April 29, 2016 @09:16AM (#11595479)
Haxe is still an excellent hedge. Js doesn't offer the same breadth of deployment targets.
Re:Hello Lua by zem on Friday April 29, 2016 @06:25PM (#11599154)
from what little i've seen of haxe (used it for a few toy projects), it seems a lot more pleasant to program in and to maintain than javascript.
Re:Hello Lua by jestar_jokin on Friday April 29, 2016 @12:46AM (#11593816)
cURL was created by Daniel Stenberg, Haxe was created by Nicolas Cannasse. (EDIT: beaten)

My favourite Haxe feature is compile-time macros. This leads to massive meta-programming and language extension capabilities. My least favourite aspect is that it does not strictly unify or abstract every target language & runtime, so you end up writing in that language's semantics but using Haxe as syntax sugar. (Good for C++, less useful for other targets.)

Re:Hello Lua by sspiff on Friday April 29, 2016 @03:35AM (#11594287)
Can you explain what you mean by "not strictly unify or abstract every language & runtime"?

In day-to-day use, how would you notice this problem? Can you give an example of what problems or hurdles this can cause?

I've only used Haxe briefly for a few small game demos, and loved it, but I've only used the JS backend.

Re:Hello Lua by yoklov on Friday April 29, 2016 @09:49AM (#11595675)
Presumably they mean that porting your js examples to C++ isn't likely to be as easy as just flipping a switch.

Still orders of magnitude less difficult than rewriting the whole thing in another language, and given that a big target is games, abstracting over the platform semantics would be too costly IMO.

Re:Hello Lua by jestar_jokin on Saturday April 30, 2016 @09:39PM (#11604496)
Some targets will have different behaviours regarding null[1], mostly for static language targets; e.g. basic types that are represented by primitives cannot be null when targeting C#, Java, Flash, or C++. Dynamic language targets don't have this issue. (Can be avoided by wrapping your values in Null<T>).

Overflow is normally not handled consistently[2]; it's left up to the target platform to define the overflow behaviour. E.g, JS only has one numeric type (a 64-bit floating point), and can't represent integers over 52-bits. Can be avoided by using Int32 and Int64 classes, but this reduces the code interop if you want to export a library (consumers would need to make use of the Haxe representation of a 64-bit integer).

Arrays aren't fixed sized; if you want this, you need to use Vectors[3], which aren't interface-compatible with Arrays, so you can't use them as a drop-in replacement. (Compare this to Java's Collections API.)

File system access isn't available on every platform, e.g. JS, Flash. If you target Node.js you can use the file system, you just can't use the normal file API, so you need to write stuff specific to each target.

Dates are, as always, a pain to deal with. Timezones aren't supported by the Date class[4]. DateTools.makeUtc() is not available on some platforms[5].

The built-in database API is only accessible on C++, Neko (Haxe-specific bytecode/runtime), and PHP[6]. AND it only seems to support MySQL. This has stymied my ideas about a cross-language multi-DB ORM.

Basically, you have to intimately know the gotchas of every language you target, and if you already know them you may as well write in that language.

Having said all that, Haxe seems to be greatly improving these abstractions with every release; I think given enough time, it will become an amazing tool. I'm starting to use it in place of Python as my "go to" language. For most purposes you can just use a single target; Neko works well for most functionality, but you can always target C++ for "true" cross-platform compatibility and performance, and JS for the browser.

[1] http://haxe.org/manual/types-nullability.html

[2] http://haxe.org/manual/types-overflow.html

[3] http://api.haxe.org/haxe/ds/Vector.html

[4] http://api.haxe.org/Date.html

[5] http://api.haxe.org/DateTools.html

[6] http://api.haxe.org/sys/db/Mysql.html

Re:Hello Lua by sspiff on Sunday May 01, 2016 @11:13AM (#11606468)
Thanks for an excellent list of examples!
Re:Hello Lua by anifow on Friday April 29, 2016 @07:52AM (#11595062)
Also, HaXe is the only viable technology which enables a high-quality canvas-based applications which mirror native applications (while also permitting compilation to those native applications).
Re:Hello Lua by pierrec on Friday April 29, 2016 @02:28PM (#11597719)
I suppose you're not counting the likes of NW.js and Electron because the native version isn't really native - though from the user's perspective, it can feel as native as a Haxe application, assuming they don't look at the ridiculous distributable size and RAM usage of the NW version.

Even then, this still leaves C/emscripten which can also fulfill those requirements, and with similar output sizes (I've tested both and compared the sizes of the resulting web versions, seems that both have an overhead of 1-2 MB for a small graphical application). It depends on what you're planning to do, but one problem I found with Haxe is that it's lacking a good general-purpose UI library that also works well on the web target. Only lower-level graphics libraries.

Re:Hello Lua by agentgt on Friday April 29, 2016 @07:45AM (#11595030)
Wait which curl are we talking about because I'm not sure Daniel worked on Haxe. There is also Curl the programming [1] language (which reminds me of Haxe's goals) but I don't think the creator of that worked on Haxe either.

[1]: https://en.wikipedia.org/wiki/Curl_(programming_language)

Re:Hello Lua by alehander42 on Friday April 29, 2016 @05:43AM (#11594637)
You might be also interested in https://github.com/alehander42/pseudo-python . It's still in very early stages, but some of it goals overlap with Haxe-s
Re:Hello Lua by choosername on Friday April 29, 2016 @12:47PM (#11596948)
> "compiles"

* transpiles

Re:Hello Lua by Rusky on Friday April 29, 2016 @01:46PM (#11597370)
"A compiler is a computer program (or a set of programs) that transforms source code written in a programming language (the source language) into another computer language (the target language)" [0]

Haxe does everything a typical compiler does, right up until the very last step. Not much point in correcting people about the use of the term.

[0] https://en.wikipedia.org/wiki/Compiler