News for hackers. Stuff that matters.
Re:PhantomJS - minimalistic headless WebKit by vitovito on Tuesday March 22, 2011 @03:55PM (#2356145)
Note: not genuinely headless. It still requires you run an X server, such as Xvfb.

As far as I know, there's no genuine headless Webkit.

For a genuinely headless system, consider Chris Lord's "offscreen" branch of Gecko, discussed here: http://blog.mozilla.com/ted/2010/07/29/moz-headless-screensh...

I'd be interested in sponsoring someone who wanted to bring the offscreen branch up to mainline, and Chris would be willing to mentor you in such an effort.

Re:PhantomJS - minimalistic headless WebKit by newhouseb on Tuesday March 22, 2011 @04:48PM (#2356413)
Yeah, I wanted something like this a couple years ago and was disappointed to find no true headless library. I spent a lot of time looking at WebCore, but eventually gave up after a while.

I wonder if there's a way to extricate some of Chrome's rendering logic to do genuinely headless stuff. I know that things are rendered in separate processes and then piped to the browser window process and I have to wonder if there's a way to launch one without the other, and thus, without any requirements for a display server like X.

Re:PhantomJS - minimalistic headless WebKit by vitovito on Tuesday March 22, 2011 @05:18PM (#2356551)
If you're okay with using Gecko instead of Webkit, the headless branch does compile into a standalone executable you can fork out to, it's what we do.

The issue with Gecko was less about rendering offscreen and more about rendering widgets (dropdowns, input boxes, etc.) which are normally handled by the windowing system.

Re:PhantomJS - minimalistic headless WebKit by maximilianburke on Wednesday March 23, 2011 @02:14AM (#2358278)
EA's branches of WebKit, available at http://gpl.ea.com, are designed for integration into console video games and don't require any sort of X server or OS interface. It's not the most up-to-date but it's sufficiently modularized to adhere to the LGPL without violating the Sony/Microsoft developer agreements.
Re:PhantomJS - minimalistic headless WebKit by vitovito on Wednesday March 23, 2011 @02:09PM (#2360539)
This might be interesting. Remember to check all of the products listed, as we only release our latest EA WebKit version when a game ships with it.

"EA WebKit supports Win32, Win64, Playstation 3, and Xbox 360 platforms" according to the documentation, but perhaps the modularization and the EA Raster primitives gets you further along than some of the other forks.

If anyone has any questions about EA WebKit, let me know and I can pass them along.

EDIT: ...Or you can ask the parent poster, who actually works on them. :)

Re:PhantomJS - minimalistic headless WebKit by wslh on Tuesday March 22, 2011 @05:36PM (#2356631)
I use htmlunit for scraping, it's a headless browser although using WebKit will be far better.

It's sad that WebKit lacks some easier integration (good COM/.NET object in Windows).

Re:PhantomJS - minimalistic headless WebKit by vitovito on Tuesday March 22, 2011 @06:55PM (#2356923)
Crowbar is a Gecko-based scraper, if you're interested in using a real browser: http://simile.mit.edu/wiki/Crowbar
Re:PhantomJS - minimalistic headless WebKit by wslh on Wednesday March 23, 2011 @07:55AM (#2358972)
The issue is using Crowbar in latest Gecko versions.
Re:PhantomJS - minimalistic headless WebKit by ojbyrne on Tuesday March 22, 2011 @11:57PM (#2357929)
I've used htmlunit as well, basically because it seems to be the most mature. I find it chokes on some google maps js.
Re:PhantomJS - minimalistic headless WebKit by adamkhrona on Wednesday March 23, 2011 @09:36AM (#2359284)
We created a genuinely windowless branch of Chromium two years ago named Awesomium; it's free for non-commercial use and is used in several major games and commercial projects: http://www.awesomium.com
Re:PhantomJS - minimalistic headless WebKit by vitovito on Wednesday March 23, 2011 @12:47PM (#2360167)
"Runs on Windows and Mac OSX Intel platforms."

If you only run on Windows or OS X, it's windowless, but it's not headless.

Headless means something running on a command-line console, and you can't run either of those operating systems that way.

Re:PhantomJS - minimalistic headless WebKit by adamkhrona on Wednesday March 23, 2011 @02:50PM (#2360726)
We're working on a Linux port for use server-side but that's not to say our current framework is not headless. We developed this branch to be completely independent of any windowing systems (and have even coaxed Flash and Silverlight to render correctly in such an environment)-- check out our HelloAwesomium sample in the SDK, it runs straight from the command-line and outputs a JPEG.