News for hackers. Stuff that matters.
Re:Lua/APR - proper standard library for Lua by davidhollander on Friday July 08, 2011 @06:43PM (#2744399)
Only downside with LuaAPR is you're back to writing Apache style servers, with an OS thread for every connection instead of using non-blocking sockets where possible (Didn't find any support for nonblocking sockets or a select\poll variant in the linked docs)

For my Lua server [1] I currently use the nixio library from the LuCI project and polling instead[2]

Although the downside of both nixio and Lua/APR library is the use of compiled C bindings to manually interact with the virtual machine.

The exciting development with LuaJIT is the high performance FFI that allows you to make system calls and create structs directly from in Lua[3], so I am currently considering switching from nixio to ljsyscall when I do a new version [4]

[1]https://github.com/davidhollander/ox

[2]http://neopallium.github.com/nixio/

[3]http://luajit.org/ext_ffi_tutorial.html

[4]https://github.com/justincormack/ljsyscall

Re:Lua/APR - proper standard library for Lua by drudru11 on Friday July 08, 2011 @09:11PM (#2744708)
There is a good reason for the thread per connection architecture and there is a good reason for the polling style arch. The latter being better for static content.
Re:Lua/APR - proper standard library for Lua by davidhollander on Friday July 08, 2011 @09:45PM (#2744767)
I'd contend the opposite, polling is better for dynamic content.

Let's say you have 1000 requests for a landing page, and the landing page has a data request that scrapes the entire database to find the popular-maximum-geo-similarity-reccomendation-zone-geo-user-thing list for the entire site, but only updates once every 10 seconds.

Ideally, you want to only send at most 1 request to your database every 10 seconds, without having to open a new connection. If you are doing a polling server and have all 1000 clients waiting for the data, you can trigger a callback to all of them when the database reports back without having to recopy data, use shared memory, do more ipc, or context switches. And for the 1000 clients to pause to wait for that single combined data callback, no extra blocking reads\connections to anything were necessary.

If by static content you are referring to file transfers, polling does not actually work on regular files in Linux. I believe they will always return ready to read even if reading would block. So usually a thread or forked process is actually required when handling files unless you dont mind disk seeks pausing the entire event loop on your server.

Re:Lua/APR - proper standard library for Lua by rcoder on Saturday July 09, 2011 @02:14AM (#2745146)
At the risk of feeding the trolls, I couldn't help weighing in here...

First of all: you're just straight-up wrong about file transfers and polling. The select(), epoll(), et. al. syscalls are exactly what you want for large network file transfers. This is not the same as naive "polling", but is what most implementers mean when they talk about an event/polling based server implementations.

Even "green" threading runtimes like MRI and CPython can still multiplex threads in the presence of blocking I/O calls, because the standard C syscalls can easily indicate when the main thread would block while trying to read or write.

Furthermore, a "threaded" server in no way has to allocate a full hardware thread for every client. Single-threaded, callback-based systems may fare nearly as well for I/O-bound workloads, but threads still have real value for many if not most real-world concurrent systems. A pool of worker threads can keep both the CPU and I/O subsystems maximally active without forcing a "shared nothing" architecture onto a system located on a single physical host.

In a world of multicore CPUs, limited L1/L2 cache, and fast kernel schedulers, threaded code can stand toe-to-toe with pure event loops for non-trivial workloads.

Re:Lua/APR - proper standard library for Lua by davidhollander on Saturday July 09, 2011 @01:22PM (#2745907)
>you're just straight-up wrong about file transfers and polling. The select(), epoll(), et. al. syscalls are exactly what you want for large network file transfers.

I think there's some confusion on your end. My assertion was not that you shouldn't use polling for file transfers, but that you also needs to use at least one additional fork\ thread to do the polling in a second event loop to isolate disk-dependent connections, because polling does not work on the disk part of a file transfer, only the network part.

> standard C syscalls can easily indicate when the main thread would block while trying to read or write

Only on the socket end. Not the regular-file end. This has been the case for a long time on linux with O_NONBLOCK and regular files (aka not pipes\sockets\processes). Which OS\kernel version are you using?

Re:Lua/APR - proper standard library for Lua by asb on Saturday July 09, 2011 @04:20AM (#2745311)
Non-blocking sockets and poll are supported in the underlying APR library. There just doesn't seem to be support for the apr_poll stuff in lua-apr yet:

http://apr.apache.org/docs/apr/1.4/group__apr__poll.html

Re:Lua/APR - proper standard library for Lua by xolox on Saturday July 09, 2011 @02:06PM (#2746010)
FYI: I'm considering adding a binding for the pollset module.
Re:Lua/APR - proper standard library for Lua by asb on Saturday July 09, 2011 @05:43PM (#2746399)
Awesome. I'm a happy user of lua-apr and would recommend it to anyone.
Re:Lua/APR - proper standard library for Lua by xolox on Friday July 08, 2011 @03:01PM (#2743652)
Hi all, author of the Lua/APR binding here. If you have any questions or comments, feel free to post them here. I haven't gotten a lot of feedback on the binding yet so would love to hear what people think about it!
Re:Lua/APR - proper standard library for Lua by jolan on Friday July 08, 2011 @05:07PM (#2744106)
Nice job. This fills a big gap in Lua.

I think the stewards of Lua are afraid to grow the standard library because most of Lua's early popularity stemmed from how small and embeddable it is.

Re:Lua/APR - proper standard library for Lua by tung on Friday July 08, 2011 @10:49PM (#2744871)
The correct title for this is "Lua/APR: Apache Portable Runtime binding for Lua". I'm pointing this out because I'm tired of seeing submission titles used to make a point instead of describing what's behind the link. This mightn't be a big deal for other people, but personally, I don't like having opinions[#] forced on me like this.

[#] The opinion here is that Lua needs a standard library. Maybe it does, maybe it doesn't, but I want to come to that conclusion myself, that's all.

Re:Lua/APR - proper standard library for Lua by xolox on Saturday July 09, 2011 @02:04PM (#2746006)
This is my fault really, the first paragraph of the homepage contains the following remark:

> APR powers software such as the Apache webserver and Subversion and Lua/APR makes the APR operating system interfaces available to Lua, serving as an extended standard library.

I wrote that description because I like Lua so much that I want to use it not only for embedding but also for writing standalone scripts. However I found that writing standalone Lua scripts that run on both Windows and UNIX is hard because Lua's standard library contains almost no operating system interfaces (e.g. no directory handling and minimal process handling). Also lots of third party libraries either work on Windows or POSIX systems but not both.

After reading your comments and those by compay about Penlight I realized that "standard library" may not be the best way to describe the Lua/APR binding because its focus is really on being an extended operating system interface.

Re:Lua/APR - proper standard library for Lua by wildmXranat on Friday July 08, 2011 @02:47PM (#2743615)
This is very nice. It's good to see examples weaved into the site on how to use it. If anything, main libapr site could use something a bit better than the Doxygen output manual as reading the unit tests from the actual source is more approachable to me than their documentation.
Re:Lua/APR - proper standard library for Lua by xolox on Friday July 08, 2011 @03:08PM (#2743680)
I totally agree on the Doxygen documentation and indeed, while writing the Lua/APR binding the APR unit tests were more useful to me than the public headers and documentation.

I must admit I also looked through the implementation here and there, when the documentation didn't explain the 'grand scheme' and there were no tests to read :-(

Re:Lua/APR - proper standard library for Lua by compay on Friday July 08, 2011 @04:58PM (#2744073)
That's a very useful library, and also great example of how easy it is to hook into C code from Lua.

Another one which I prefer personally because it's more comprehensive, is Steve Donovan's Penlight. If you have a background in Python you might enjoy using it:

https://github.com/stevedonovan/Penlight

Re:Lua/APR - proper standard library for Lua by philips on Friday July 08, 2011 @05:40PM (#2744224)
If you are interested in hooking C from Lua or vice versa: I gave a recent talk on the ways of hooking C code from Lua at Open Source Bridge. The slides and code are here: https://github.com/philips/beaming-up-alien

Sometimes really simple examples can be useful.

Re:Lua/APR - proper standard library for Lua by BasDirks on Friday July 08, 2011 @03:03PM (#2743657)
Don't mean to hijack, but could I be referred to a good introduction to Lua? I keep hearing about it, but it's one of the few languages I know NOTHING about, and I have a free weekend!
Re:Lua/APR - proper standard library for Lua by huyegn on Friday July 08, 2011 @03:11PM (#2743694)
If you liked Kernighan and Ritchie's "The C Programming Language", then you'll love "Programming In Lua". The first edition is available online here:

http://www.lua.org/pil/

Like K&R, it's style is colloquially terse, yet each sentence is chock full of information. It's a great book for someone who's already coming from another language like Python.

If you're into graphics programming, then http://love2d.org/ would be the perfect vehicle for you to get into Lua while you read PIL.

Re:Lua/APR - proper standard library for Lua by reginaldo on Friday July 08, 2011 @03:11PM (#2743693)
When I was getting started I bought the "Programming in Lua"[1] ebook, by the author of Lua himself.

There's an older edition available for free online[2].

Then there's of course the lua users wiki and its "Learning Lua" section[3].

I find the wiki a little hard to navigate so I just went ahead and bought the book. IMHO the gems in Chapter 10 (Complete Examples) and the clear explanation of coroutines are alone worth the price. The rest is a very nice bonus too.

[1] http://store.feistyduck.com/products/programming-in-lua [2] http://www.lua.org/pil/ [3] http://lua-users.org/wiki/LearningLua

Re:Lua/APR - proper standard library for Lua by BasDirks on Friday July 08, 2011 @03:22PM (#2743738)
Thank you (and all other commenters) kindly. I purchased the book.
Re:Lua/APR - proper standard library for Lua by xolox on Friday July 08, 2011 @03:09PM (#2743686)
The first edition of "Programming in Lua" is available for free online and might make for more interesting reading than the reference manual: http://www.lua.org/pil/
Re:Lua/APR - proper standard library for Lua by justincormack on Friday July 08, 2011 @03:05PM (#2743670)
The reference manual is very well written http://www.lua.org/manual/
Re:Lua/APR - proper standard library for Lua by randrews on Friday July 08, 2011 @04:22PM (#2743929)
This isn't a language introduction, but after you learn it, I've written a bunch of posts at http://playwithlua.com, all about weekend-sized hacks in Lua.
Re:Lua/APR - proper standard library for Lua by meric on Saturday July 09, 2011 @02:11AM (#2745141)
Re:Lua/APR - proper standard library for Lua by swah on Friday July 08, 2011 @03:28PM (#2743769)
Re:Lua/APR - proper standard library for Lua by ay on Saturday July 09, 2011 @12:48AM (#2745044)
Another interesting combination would be:

https://github.com/justincormack/ljsyscall

this would work with LuaJIT: http://luajit.org/

The FFI in LuaJIT is kind of brilliant. Check yourself.

EDIT: with FFI in LuaJIT you can jump very comfortably between the high-level Lua code and low-level native calls. It's almost addictive. And the code is running X times faster than the Lua baseline (X between single-digit and triple-digit values).

Re:Lua/APR - proper standard library for Lua by malkia on Saturday July 09, 2011 @12:55AM (#2745051)
I've made LuaJIT FFI bindings to some "C" precompiled libraries for these platforms: OSX, iOS, Windows, Linux (x86+arm)

There libraries are: GLFW, SDL, ZeroMQ, AntTweakBar, and planning on few more. There is also OpenCL and OpenGL and glu.

Please visit: http://github.com/malkia/ufo

You should not need to recompile, as I've packaged the binaries.

Re:Lua/APR - proper standard library for Lua by shubber on Friday July 08, 2011 @02:48PM (#2743618)
I have to admit, that's kind of brilliant.