News for hackers. Stuff that matters.
Re:Lua support in the NetBSD kernel by lifeisstillgood on Wednesday October 16, 2013 @08:08PM (#6562875)
Holy crapola - they are putting a Lua interpreter in the kernel, and it has access to some limited calls.

This is partly driven by "we need device drivers and no one understands C anymore", partly because it's cool, but mostly because Moores law is still alive and well. They are putting a scripting language in a kernel ! Forget write your app in python / perl / Ruby then optimise in C - the raw power argument is going to overwhelm us all.

I heard a stat the other day - that a greetings card, the kind that plays a silly tune, has more processing power than all computers on the planet in 1960. And we should expect a similar growth in the next 50 years. Even if that's out by two orders of magnitude just let it sink in.

I think we are at a Cambrian explosion period - where the goal is to try out every possible new body shape, as fast as possible and see which ones get the Darwin seal of approval.

New organisation forms are possible, some for the first time in human history, new ways of thinking and communicating - it's stuff like this that makes one realise the water around the frog is getting hotter.

(Still it's worth remembering that most train companies in 19C England failed and the average return for stockholders was 10% - just because the world will change beyond recognition does not mean industry stocks is a great return)

Re:Lua support in the NetBSD kernel by ketralnis on Wednesday October 16, 2013 @09:00PM (#6563065)
Kernel programming is an eternal struggle between people trying to pull more into the kernel for performance, and people trying to pull stuff back out for safety and modularity. We've all seen the arguments about the in-kernel HTTP daemons and whatnot. I tend to be on the side that says that more things should be pulled out.

> This is partly driven by "we need device drivers and no one understands C anymore"

I don't buy this argument. Who are the people

* Using NetBSD

* having cutting-edge hardware for which drivers don't exist

* deciding to write device drivers

* that have device specs/docs needed to write them

* that don't know/want to use C

* that would do it if only they could write it in a scripting language?

Do these people exist? I assert that the far more common case is that the device specs, documentation, BIOS, etc. aren't accessible. If that's true, they couldn't write the driver regardless of the language, C is hardly the limiting factor here

Re:Lua support in the NetBSD kernel by tedunangst on Wednesday October 16, 2013 @09:41PM (#6563188)
A far better rationale is that it makes fast prototyping easier. You write three lines of Lua, stuff into the kernel, and run. Did the blinkenlights blink? Yes, good, now translate that code into the C driver.
Re:Lua support in the NetBSD kernel by voltagex_ on Wednesday October 16, 2013 @10:33PM (#6563376)
This would be great for something like the Raspberry Pi.
Re:Lua support in the NetBSD kernel by koralatov on Thursday October 17, 2013 @06:37AM (#6564498)
My worry in situations like that is that the `fast prototype' ends up also being the final driver, because it works well enough and nobody feels the urgent need to rewrite in C.
Re:Lua support in the NetBSD kernel by drdaeman on Thursday October 17, 2013 @09:18AM (#6565113)
If it works and is sufficiently performant - is there anything wrong with the fact it's written in X, not Y?

And if it doesn't, then there's certainly a reason to optimize.

Re:Lua support in the NetBSD kernel by agumonkey on Thursday October 17, 2013 @12:27AM (#6563664)
Re:Lua support in the NetBSD kernel by gwu78 on Wednesday October 16, 2013 @09:31PM (#6563153)
"Do these people exist?"

This is spot on. When you hear NetBSD developers arguing about proposals (or commits, ex post facto, like here), there is often mention of "users". But these are mostly fictional.

I think it's safe to say most NetBSD users are comfortable working with C.

I reckon it's not much different with OpenBSD: most changes they make are for the benefit of the developers themselves, to suit personal preferences, than for the benefit of other "users". (And that's perfectly acceptable, I think.) Of course, these changes can benefit users beyond simply those who make them and those who approve them, but those users are not the ones driving the changes.

Re:Lua support in the NetBSD kernel by groovy2shoes on Wednesday October 16, 2013 @09:58PM (#6563253)
This is spot on. When you hear NetBSD developers arguing about proposals (or commits, ex post facto, like here), there is often mention of "users". But these are mostly fictional.

I think this is true of much open source software. Just for the sake of example, a few years ago the Gnome devs removed some printer config options because they decided the "users" didn't need so many options. It's not that Gnome doesn't have users, but the developers seemed to have some notion of a type of user that didn't really exist.

I think more often than not, fictional users are used as justification for the addition of some nice-to-have feature, but of course I'm just speculating.

Re:Lua support in the NetBSD kernel by plextoria on Thursday October 17, 2013 @02:29AM (#6563942)
I want to get involved and experiment with NetBSD, but I don't feel comfortable in C, nor C++ for that matter.
Re:Lua support in the NetBSD kernel by thelukester on Thursday October 17, 2013 @05:28AM (#6564337)
Novice developers should not be writing drivers in any language. If you don't have a C background, you're not going to be able to contribute much code to any OS. Besides config files and scripts, NetBSD is nearly 100% C.

If you really want to get into OS programming, I recommend you start with "Computer Architecture" by Hennessy & Patterson along with a course in C programming.

[1] http://www.ohloh.net/p/netbsd

Re:Lua support in the NetBSD kernel by protomyth on Thursday October 17, 2013 @12:43PM (#6566503)
"Novice developers should not be writing drivers in any language."

Well, it might not be such a bad thing to write your stuff in Lua and get the lights blinking as a way to work your way up to C and mainline device driver development. If the device drivers interest the novice programmer, let's encourage that. Its not like there aren't a ton of safeguards between the novice and getting their driver in the mainline.

Re:Lua support in the NetBSD kernel by chubot on Wednesday October 16, 2013 @10:26PM (#6563355)
> We've all seen the arguments about the in-kernel HTTP daemons and whatnot. I tend to be on the side that says that more things should be pulled out.

I always wondered if a lot of components like this could be either sped up, or moved out of the kernel, using a zero-copy messaging system based on splice() / vmsplice() in the Linux kernel (which works by page table remapping as far as I understand). I guess the prime candidate would be device drivers.

Re:Lua support in the NetBSD kernel by ketralnis on Wednesday October 16, 2013 @11:44PM (#6563557)
That's one approach. Another one attempted sometimes is to go the other way and move the networking stack into userland as well. Depends on your bottlenecks etc
Re:Lua support in the NetBSD kernel by TheCondor on Wednesday October 16, 2013 @11:24PM (#6563512)
I've used lua in kernel space elsewhere. It links easily, it can invoke your C functions easily, and it's great for testing stuff. I could see some clever hooks being out in so you could custom tune some stuff. I bet typically they use it for testing though.
Re:Lua support in the NetBSD kernel by deutronium on Thursday October 17, 2013 @06:49AM (#6564533)
I'm curious, was this in Linux per chance.

I was looking into embedding lua in a kernel module in Linux, just to hack around with stuff.

How did you strip out the stuff that means it won't compile for kernel space?

Re:Lua support in the NetBSD kernel by fafner on Wednesday October 16, 2013 @09:52PM (#6563231)
Are they using regular Lua? Because LuaJIT can actually be quite fast. Snabb Switch is written mostly in Lua and it features Linux userland ethernet device drivers. http://blog.lukego.com/blog/2013/01/03/snabb-switchs-luajit-... (edit: link to the code https://github.com/SnabbCo/snabbswitch/tree/master/src/apps/...)

btw. there once was a project which added a Scheme interpreter to the Linux kernel. Sadly it's dead now: http://www.abstractnonsense.com/schemix/

Re:Lua support in the NetBSD kernel by maximilianburke on Wednesday October 16, 2013 @11:01PM (#6563452)
I'm not sure they would be well suited together, NetBSD has a huge list of old hardware they support; I don't think Mike Pall will be porting LuaJIT to m68k anytime soon.
Re:Lua support in the NetBSD kernel by baldfat on Thursday October 17, 2013 @07:44AM (#6564716)
Moore's law will be over by 2025 (We have 12 years or so). Semi-conductor will allow a separation of just 3 - 5 atoms to the conducting part. The belief is that quantum physics will allow the electrons to go through "walls" at that point and the switch will always be closed.

I think we will go back to the days of 640k where there is a ceiling and we will just have to refine for "large" projects. Software and not hardware will be Moore's law BUT only for a extreme circumstances. The Golden Age is coming to an end in our life time.

Re:Lua support in the NetBSD kernel by recuter on Wednesday October 16, 2013 @08:13PM (#6562893)
Lua can be sandboxed quite effectively, can you elaborate on your concerns?
Re:Lua support in the NetBSD kernel by vor_ on Wednesday October 16, 2013 @08:22PM (#6562922)
What concerns?
Re:Lua support in the NetBSD kernel by robert-wallis on Wednesday October 16, 2013 @09:32PM (#6563156)
sprintf buffer overflow in the string library https://github.com/LuaDist/lua/commit/52eff16f51750cf47afaa5...

This library is not included in NetBSD as far as I can tell: http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/modules/lua/?onl...

But it's an indication that it shouldn't be included in a Kernel.

Re:Lua support in the NetBSD kernel by Sanddancer on Thursday October 17, 2013 @04:13AM (#6564166)
There were buffer overflows in the Linux kernel in 2012 ( http://www.securityfocus.com/bid/53401 ). Should we not have filesystems in the kernel either?
Re:Lua support in the NetBSD kernel by robert-wallis on Thursday October 17, 2013 @01:08PM (#6566657)
Certainly the string library, of all places, should be using secure code.

I'm sure there are sooo many people running Linux on pre-OSX Mac harddrives formatted with HFS that could be hacked with a buffer overflow.

Are you seriously arguing that buffer overflows in the kernel are not a big deal?

Re:Lua support in the NetBSD kernel by pdpi on Thursday October 17, 2013 @06:59AM (#6564561)
Well... No, if you're a proponent of the micro-kernel design.
Re:Lua support in the NetBSD kernel by tedunangst on Wednesday October 16, 2013 @09:38PM (#6563181)
You linked to a 24,000 line diff to point out what exactly?
Re:Lua support in the NetBSD kernel by robert-wallis on Wednesday October 16, 2013 @09:43PM (#6563197)
wait for it... the exact section of the code that contains the "sprintf" I mentioned will pop into your screen. Just be patient.
Re:Lua support in the NetBSD kernel by scott_karana on Wednesday October 16, 2013 @09:05PM (#6563080)
GP was probably in response to ketralnis' comment. :)
Re:Lua support in the NetBSD kernel by ketralnis on Wednesday October 16, 2013 @09:48PM (#6563216)
recuter's comment already existed when I wrote mine, so it couldn't be in response to me
Re:Lua support in the NetBSD kernel by tonyplee on Wednesday October 16, 2013 @09:41PM (#6563192)
Haven't they been keep saying that about JAVA - Just Another Vulnerability Announcement?

Mac just turn off JAVA completely in user space and NetBSD is putting Lua in kernel?

NSA: "I have a dream! Someday, a script language will be in BSD kernel, so I can ping an script into the kernel"

:-)