News for hackers. Stuff that matters.
Re:Boot a linux kernel right inside your browser. by nupark2 on Tuesday May 17, 2011 @09:25AM (#2556260)
This is demonstrative of the advantages of the new low-level APIs being added to JavaScript to work efficiently with binary data.

Fabrice uses this to implement an x86 interpreter -- it could not be done efficiently without typed arrays. However, it is still slow -- imagine what kind of advances could be made if a common bytecode was established that would be JIT'd by the JavaScript VM, and could be output directly by the emulator.

This is why so many people want to see the browser execution environment offer more complete, low-level APIs instead of high-level APIs locked to HTML/CSS and legacy browser technology. Efficiently supporting high-performance, high-complexity systems such as an x86 emulator (or a video game, or custom font rendering, or even an application framework) absolutely require efficient low-level APIs.

Re:Boot a linux kernel right inside your browser. by dkersten on Tuesday May 17, 2011 @03:53PM (#2557604)
I've always wished that, instead of a "browser language", browsers exposed a LOW LEVEL bytecode environment (which browsers are then free to interpret, JIT compile or on-the-fly AOT compile at page load time). It should be reasonably low level, in that it should provide an abstraction from the hardware, but still allow for very efficient execution of lower level code. Then high level languages and frameworks could be layered on top. This way, browser languages would be 1) more efficient if they need to be and 2) more diverse.

Its a pity that javascript is so in-grained in client-side web now, as its really a terrible bytecode... Hell, failing the above, I would have liked to see Lua as the default client-side language, rather than Javascript (for reasons I will mention shortly), but of course that will never happen now that Javascript has been the default for fifteen years.

The reasons I would like Lua as the default language, besides the fact that LuaJIT is very fast (certainly for number crunching code, especially if you use the FFI to operate on low level structs (I mean, without calling into C) - the FFI is also one of, if not the, simplest FFI I've seen in any language[1]), it is, at least under my impression, a much more consistent language, while being just as flexible, easy to learn and dynamic. The thing I dislike most about Javascript is that it has lots of little gotchas and inconsistencies. Having said that, though, perhaps Lua does too and I just haven't used it enough yet to get bitten by them. Then again, it didn't take long for this to happen in Javascript, so Lua does at least seem to be a little friendlier.

[1] Actually, I think Factor's FFI is similarly simple, if my memory serves me correctly

Re:Boot a linux kernel right inside your browser. by copper on Wednesday May 18, 2011 @12:20AM (#2559042)
Factor, Racket, and Lua all have ffi's that are very easy to use. I've never quite liked Lua's metaprogramming, though (macros are very useful when you're writing bindings.) and my least favourite Lua quirk would be the lack of integer types - both other languages have fairly sane (and performant!) numeric towers.

That said, I'd give a higher priority for the language in my browser to be safe: some kind of correctness guarantee would be nice. Or even, while we're at it, some kind of way to analyze the script and say "this script GETS from urls x,y, and posts data d to z"

Re:Boot a linux kernel right inside your browser. by dkersten on Wednesday May 18, 2011 @06:32AM (#2559669)
Standard Lua may not have integer types, but as far as I know, LuaJIT does actually support integer types under the hood. At the very least, you can force it by declaring an integer using the FFI, eg:

    local ffi = require("ffi")
    int = ffi.new("int")
    int = 10
    print(int)
I also don't find the lack of macros in Lua as limiting as I do in other languages. For example, if I wanted to create an EDSL in Lua, I would have it parse a string at startup. For example, imagine a state machine DSL:

    fsm = my_dsl[[
        start: foo
        stop:  quux
        foo -> bar
        foo -> baz
        bar -> foo
        baz -> quux
    ]]{ foo = function()
            code for foo here
        end,
        bar = function()
            code for bar here
        end,
        baz = function()
            code for baz here
        end,
        quux = function()
            code for quux here
        end
    }
    fsm.run()
As a real-world example, here is some Lua code I'm using in a hobby game project:

    DEFINE(Traits.Position)[[
        float x;
        float y;
        float z;
    ]]
    DEFINE(Traits.Movement)[[
        float x;
        float y;
        float z;
    ]]
    DEFINE(Behaviors.do_movement){
        input={Traits.Position, Traits.Movement},
        output=Traits.Position,
        frequency=0.01,
	
        -- Update entities position
        fn = function (pos, mov)
            local delta = dark.timedelta
            new_pos = Traits.Position.new()
            new_pos.x = pos.x + (mov.x * delta)
            new_pos.y = pos.y + (mov.y * delta)
            new_pos.z = pos.z + (mov.z * delta)
            return new_pos
        end}

    -- Create a sample entity
    local entity = Entities.new()
    entity.add(Traits.Position, {1.0, 0.0, 0.0})
    entity.add(Traits.Movement, {0.0, -1.0, 0.0})

"That said, I'd give a higher priority for the language in my browser to be safe: some kind of correctness guarantee would be nice. Or even, while we're at it, some kind of way to analyze the script and say "this script GETS from urls x,y, and posts data d to z""

I definitely agree with the above.

Re:Boot a linux kernel right inside your browser. by leoc on Wednesday May 18, 2011 @01:12AM (#2559128)
> Its a pity that javascript is so in-grained in client-side web now, as its really a terrible bytecode...

Gilad Bracha complains about this at http://gbracha.blogspot.com/2011/03/truthiness-is-out-there....

Re:Boot a linux kernel right inside your browser. by tlrobinson on Tuesday May 17, 2011 @02:48PM (#2557387)
Imagine what kind of advances could be made if a common bytecode was established

I'm beginning to believe NativeClient and PNaCl (LLVM on NaCl: http://nativeclient.googlecode.com/svn/data/site/pnacl.pdf) has the best shot at becoming this.

Google is already including NaCl in Chrome developer builds, so I wouldn't be surprised if they enabled it in releases by the end of the year, and started releasing Native Client applications soon after.

http://blog.chromium.org/2011/02/native-client-getting-ready...

Re:Boot a linux kernel right inside your browser. by equark on Tuesday May 17, 2011 @04:29PM (#2557768)
I would agree, but watching the NativeClient talk at Google IO this year made me much less certain about the commitment. The presenter's answers to questions just seemed to suggest nobody was using it and Google/the developer didn't really care or expect anybody to actually adopt it.

Hopefully this was an inaccurate impression, because NaCl could have a huge impact on both the client and server

The best path to adoption would be for Google to very aggressively push NaCL as a Flash-like plugin. If they could get adoption -- through Unity3d on Facebook or something like that -- perhaps they'd win in the long run. If they just keep it part of Chrome it seems rather hopeless.

Re:Boot a linux kernel right inside your browser. by Lerc on Tuesday May 17, 2011 @06:25PM (#2558195)
I think it may be a bit of a problem being a project from Google. Making the system totally secure is a difficult task. That is true for all sandboxes. People find vulnerabilities, people fix the holes.

The problem is not that NaCl is insufficiently secure, but any vulnerability will be spun as "Google making your system insecure". I think an independent group doing the same thing would be no more or less secure, but wouldn't have the baggage that would come from being seen as a component of a much bigger entity.

Re:Boot a linux kernel right inside your browser. by nupark2 on Tuesday May 17, 2011 @02:56PM (#2557413)
> I'm beginning to believe NativeClient and PNaCl has the best shot at becoming this.

Me too, although the biggest hurdles seem as much political as they are technical.

My hope is that Google can strong-arm NativeClient into active use, forcing otherwise recalcitrant browser makers (Mozilla, Apple, Microsoft) into adopting it.

Re:Boot a linux kernel right inside your browser. by jonknee on Tuesday May 17, 2011 @02:56PM (#2557415)
NativeClient is being distributed in the official public build of Chrome, just in a disabled state. You can turn it on in about:flags.
Re:Boot a linux kernel right inside your browser. by tectonic on Tuesday May 17, 2011 @03:23PM (#2557494)
I don't know a lot about NaCl but my gut reaction is that it will be insecure and allow my computer to be remotely exploited. Why am I wrong?
Re:Boot a linux kernel right inside your browser. by tlrobinson on Tuesday May 17, 2011 @03:38PM (#2557544)
The entire point of NaCl is to run native code securely.

I won't get into the details because there's tons of information out there (http://www.chromium.org/nativeclient/reference/research-pape...), but code must be compiled using a special NaCl compiler, then before it's executed the client runs it through the NaCl verifier to ensure it's safe to execute.

Of course it's possible there are bugs in NaCl, but Google won't enable it in Chrome by default until they're very confident it's secure.

Re:Boot a linux kernel right inside your browser. by nupark2 on Tuesday May 17, 2011 @03:30PM (#2557517)
http://www.chromium.org/nativeclient/reference/research-pape...

It's possible that the design could be flawed, but the same is true for browser implementations and mobile application sandboxing.

Re:Boot a linux kernel right inside your browser. by guygurari on Tuesday May 17, 2011 @03:06PM (#2557448)
Exactly. I never understood why the web community decided to re-invent this wheel. We started with a model for text publishing (HTML), and are trying to evolve it to allow full blown apps. If browsers simply supported the JVM as the browser execution environment, instead of implementing their own javascript engines, we'd have:

1. More language independence

2. Great performance for games, video (think implementing codecs without the need for browser support), and for crazy things like this.

3. Most importantly, the ability to run actual desktop-like apps in the browser at near-native speeds, instead of the mess that is currently required to get web apps to behave like desktop apps. (Nowadays Rails and friends are hiding most of this mess, but it's still there.)

Re:Boot a linux kernel right inside your browser. by neopanz on Tuesday May 17, 2011 @03:57PM (#2557619)
Agreed. And for those who pull the Java applet as the perfect counter-example, they need to ask themselves: does it take this long to start an Android app? Of course not! Because you can use the JVM in many different ways to eliminate the load time. In a way, one could imagine Android has a better example of how to mix OS/Web/Multi-Architecture(CPU)
Re:Boot a linux kernel right inside your browser. by mythz on Tuesday May 17, 2011 @03:20PM (#2557484)
Have you never used Java Applets? (i.e. JVM in the browser) the load time alone makes that a terrible idea. Not too mention the 'breaking the web/page model' and poor UX that ensued.

Strange that people actually think Java would be the saviour. Feel free to keep continue to use Java applets if you think they're superior tech, they should still be well supported.

Re:Boot a linux kernel right inside your browser. by guygurari on Tuesday May 17, 2011 @03:42PM (#2557561)
I have used and written applets, and they are horrible, but it's not what I'm suggesting. Applets run in their own area on the page and can't interact with the rest of the page like javascript does. I'm suggesting to replace the javascript runtime with a JVM runtime, while supporting everything javascript does through an API. Here's the use case I'm imagining:

1. When a user starts loading a page (read 'web app') the browser spawns a new JVM for that page. How long does this take? You can bring it down to the price of a fork if you keep a JVM running at all times.

2. Instead of running javascript that's embedded in the page, you compile and run Java/JRuby/Jython. Better yet -- have the server precompile the embedded code and send you the bytecode. Load it dynamically into the JVM and run it in the page's thread. With a modern JVM you get JIT for free and the code runs at near-native speeds.

3. Want long-running background tasks? No problem -- the JVM code can start its own threads. Suddenly AJAX, and all the other stuff we use to make web apps behave like desktop apps becomes trivial.

4. User leaves the page / web app? Kill the JVM.

Re:Boot a linux kernel right inside your browser. by Jeema3000 on Tuesday May 17, 2011 @09:27PM (#2558711)
"Applets run in their own area on the page and can't interact with the rest of the page like javascript does."

You can actually call methods in a Java applet from Javascript and vice-versa. It just doesn't seem to be used very much:

http://download.oracle.com/javase/tutorial/deployment/applet...

Re:Boot a linux kernel right inside your browser. by rbranson on Tuesday May 17, 2011 @07:25PM (#2558407)
You seem highly optimistic about the JVM's appropriateness here. In reality what the browser needs is something more like the Dalvik VM, which is designed for compactness, efficiency, and multi-process models.
Re:Boot a linux kernel right inside your browser. by rbranson on Tuesday May 17, 2011 @03:25PM (#2557503)
While I don't think the JVM is the savior, applets are a far cry from a good example. Applets usually cause the browser to lazy load the entire JVM as a plug-in, from scratch. Very little effort has been put forth to make this process fast. It's likely if the JVM was directly integrated, it would remain resident.
Re:Boot a linux kernel right inside your browser. by rbranson on Tuesday May 17, 2011 @03:22PM (#2557492)
Seriously? the JVM? You listed the pros, but what about all the cons? Modern JVMs are fast, but highly specialized for long-running applications, which web pages are decidedly not. That's just the tip of the iceberg.
Re:Boot a linux kernel right inside your browser. by guygurari on Tuesday May 17, 2011 @03:52PM (#2557603)
Yes, seriously. In what sense are JVMs "specialized for long-running applications"? If you mean they start slowly, that's hardly the case even if you start from scratch:

   > time java HelloWorld
   Hello, World!
   java HelloWorld  0.28s user 0.06s system 135% cpu 0.248 total
In any case, the JVM is just an example (although I still argue it can serve this purpose well). My main point is that the "correct" model for the web is a bytecode-running VM inside every browser that lets you do everything javascript does through APIs.
Re:Boot a linux kernel right inside your browser. by delackner on Wednesday May 18, 2011 @01:16AM (#2559134)
Seriously? .28s is 280 milliseconds, for a result with zero content. That is already at the outside acceptable threshold for being perceived as slow response, and that's before we spend any time at all rendering the page. Certainly a VM of some kind could boot faster, but milliseconds count.
Re:Boot a linux kernel right inside your browser. by rbranson on Tuesday May 17, 2011 @07:13PM (#2558363)
If you're just suggesting that we need to ship a common bytecode to browsers, then I agree.
Re:Boot a linux kernel right inside your browser. by sehugg on Tuesday May 17, 2011 @03:34PM (#2557531)
The applet model had/has major issues:

* Takes way too long to load the JVM. Deal-breaker for many sites.

* Horrible (AWT) and complicated (Swing) UI toolkits that ignored any look-and-feel parity with the rest of the web site or even the web browser itself.

* The environment is controlled by Oracle. I don't think they're looking to compete with JS, Flash and Silverlight. Thus web applets (except for niche internal apps) are dead, RIP.

Re:Boot a linux kernel right inside your browser. by guygurari on Tuesday May 17, 2011 @03:44PM (#2557567)
Applets are terrible but it's not what I'm suggesting. See my reply to mythz.
Re:Boot a linux kernel right inside your browser. by powertower on Tuesday May 17, 2011 @02:00PM (#2557191)
>This is why so many people want to see the browser execution environment offer more complete, low-level APIs instead of high-level APIs locked to HTML/CSS and legacy browser technology.

It's called the desktop and offers all the native and low-level stuff you can possibly imagine.

And the only people that want to see it, are the ones who have forgotten about it.

Maybe there is a middle ground between desktop software and browsers, but personally I'd like to keep them apart.

Re:Boot a linux kernel right inside your browser. by nupark2 on Tuesday May 17, 2011 @02:16PM (#2557260)
> It's called the desktop and offers all the native and low-level stuff you can possibly imagine.

Actually, I'd say it's called 'mobile'; The desktop failed to solve the sandboxing problem.

> And the only people that want to see it, are the ones who have forgotten about it.

As a desktop-turned-mobile developer, I haven't forgotten anything.

Simply put, I want a future in which we aren't forced to re-implement our applications multiple times for disparate, proprietary mobile platforms.

I already lived through that on the desktop. The only reason I haven't embraced the web for application development is that the technology stack is, by comparison, hobbled.

> Maybe there is a middle ground between desktop software and browsers, but personally I'd like to keep them apart.

Treating the browser as the sacrosanct temple of HTML, CSS, and JavaScript is throwing away the limited opportunity browsers have to pre-empt Android and iOS as a first-class, standardized application platform.

Re:Boot a linux kernel right inside your browser. by adrianN on Tuesday May 17, 2011 @03:03PM (#2557440)
It might just be me, but it seems profoundly silly to implement what basically amounts to a complete operating system in a browser. Why not use something reasonable like the Java or .Net VM to implement your cross platform applications? I think browsers should stay programs to browse webpages and therefore keep their focus on scripting that enables fancier HTML and CSS manipulation. While running Linux in a browser is an impressive feat, to me it is similar in spirit to simulating a Turing machine in the Game of Life.
Re:Boot a linux kernel right inside your browser. by powertower on Tuesday May 17, 2011 @02:35PM (#2557341)
I am not able to apprehend the kind of confusion of facts and ideas that could provoke your statements.

1. The function of the desktop, the browser, and the mobile is not the same thing.

2. Desktops have not failed, as you claim, in sandboxing/security/virtualization; (especially when compared to browsers).

3. Turning the browser into the desktop (low level APIs), is in turn re-implementation of the desktop.

Re:Boot a linux kernel right inside your browser. by rbranson on Tuesday May 17, 2011 @03:27PM (#2557510)
You're always going to feel hobbled by a standards-based, compatibility-driven platform like the web where you must maintain compatibility with fairly ancient software. This is just the way things are. If it wasn't the web, it'd be whatever layer was used to allow your applications to run across multiple devices.
Re:Boot a linux kernel right inside your browser. by podperson on Tuesday May 17, 2011 @02:35PM (#2557340)
I'd love to see something replace HTML, CSS, and JavaScript, but how would this be different from Java? We already have Java.
Re:Boot a linux kernel right inside your browser. by mythz on Tuesday May 17, 2011 @03:22PM (#2557488)
>> but how would this be different from Java?

It wouldn't suck.

Re:Boot a linux kernel right inside your browser. by podperson on Wednesday May 18, 2011 @03:57PM (#2561510)
Well, that would be the plan :-)
Re:Boot a linux kernel right inside your browser. by dkersten on Tuesday May 17, 2011 @05:34PM (#2558015)
Qt Quick with its QML environment (UI markup + Javascript + native extensions if you need them) is a great middle ground.

It seems that the plan for Qt 5 is to move the enitre Qt toolkit over to the "QML first" model - that is, QML becomes the default way to create interfaces, for both mobile and desktop apps, and native widgets are only used when you require them. They also want to better integrate webkit and web content. In a recent Qt desktop app of mine, I was already doing this: QML + webkit + native backend for the heavy lifting and it works very well. Be awesome to have it as the default in all environments.

FWIW, I think QML/JS would have made a greate HTML/CSS/JS alternative.

Re:Boot a linux kernel right inside your browser. by mythz on Tuesday May 17, 2011 @08:59PM (#2558644)
We've used QML in a recent QT/C++ desktop app and it's performance is comparatively poor compared with WebKit rendering and its CSS subset support is woefully inadequate.

The HTML/CSS/JS dev model is fine as it is, there's no need to hope for nirvana grass-is-greener frameworks that don't exist. Learn HTML5 like the rest of us.

Re:Boot a linux kernel right inside your browser. by guygurari on Tuesday May 17, 2011 @04:00PM (#2557634)
The desktop has several limitations. First, you have to code separately for each platform because there's not even a common runtime, let alone API. Second, installation and updates are not seamless. Third, it is less secure since native code can have many long-lasting side effects, even without security exploits.

Web apps more or less solve all these problems (code can't be completely browser-agnostic, but it's close). Why not add also the benefits of desktop apps?

Re:Boot a linux kernel right inside your browser. by qbg on Tuesday May 17, 2011 @01:20PM (#2557027)
In theory, someone could create a common bytecode as a library. Such a library would probably use the Relooper algorithm from Emscripten (https://github.com/kripken/emscripten) and JIT the bytecode to Javascript that can then be eval'd for near Javascript-level speed.
Re:Boot a linux kernel right inside your browser. by chc on Tuesday May 17, 2011 @01:44PM (#2557119)
The whole idea is to get better performance than current JavaScript interpreters allow. Making something slower than JavaScript is the opposite of why people want a common bytecode.
Re:Boot a linux kernel right inside your browser. by memvov on Wednesday May 25, 2011 @01:30AM (#2583023)
It can be implemented first on Emscripten to see if people like it; later it can be standardized.
Re:Boot a linux kernel right inside your browser. by azakai on Tuesday May 17, 2011 @02:38PM (#2557348)
I agree that for CPU emulation, something low-level like typed arrays is absolutely necessary.

However, with more high-level code, in general there shouldn't be such a need. For example, RPython is very high-level, but is compiled into very efficient C. The same could be done for JavaScript (and to some degree already is).

Re:Boot a linux kernel right inside your browser. by euroclydon on Tuesday May 17, 2011 @09:36AM (#2556290)
Can you explain the steps necessary to combine a JS x86 interpreter with a binary Linux disk image into a running VM?

The part I don't understand is, what environment does the Kernel need to think is there, and how is that done in JS?

Re:Boot a linux kernel right inside your browser. by cakeface on Tuesday May 17, 2011 @10:12AM (#2556408)
This is explained well in the Technical Notes section of the site.

http://bellard.org/jslinux/tech.html

Re:Boot a linux kernel right inside your browser. by euroclydon on Tuesday May 17, 2011 @10:25AM (#2556448)
I saw that, but I don't understand how it's booted.
Re:Boot a linux kernel right inside your browser. by nupark2 on Tuesday May 17, 2011 @11:16AM (#2556589)
From what I can see from the minimized JS, it:

- Loads the linux binary and the root memory disk image into 'memory' at the expected addresses (0x100000 and 0x400000, respectively)

- Sets the interpreted x86 CPU's EIP to the kernel's entry point

- Initializes additional requisite register state

- Starts execution.

Take a look at load_binary() and start() functions.

[edit] rickard already discussed this here: http://news.ycombinator.com/item?id=2555552