News for hackers. Stuff that matters.
Re:Etherpad Shows Google Docs How It’s Done (By AppJet) by smoody on Wednesday November 19, 2008 @09:44AM (#369489)
"Changes are made in absolute real time, something even Google hasn’t been able to do (Google docs update every fifteen seconds)."

To imply that Google 'hasn't been able to do it' is kind of silly. It's more likely that they chose not to do it and/or their scaling requirements made it impractical.

Re:Etherpad Shows Google Docs How It’s Done (By AppJet) by pg on Wednesday November 19, 2008 @11:19AM (#369577)
It's not Google's scaling requirements that limit their simultaneity (or, obviously, a deliberate choice). Google Docs uses a different architecture, where a master copy of the document is kept on the server and broadcast every n seconds to all the clients. This is easier to implement, but can't be made fast enough to feel realtime.

Etherpad uses a different approach: they ship all the edits to all the users as soon as they happen, and each client processes them to produce an updated document. The hard part is making sure each user is seeing the same thing. This is easy with the Google Docs model where a single master copy is regularly rebroadcast. To do it the Etherpad way, without a master copy, is extremely hard. From what I've heard about what they had to do, it sounds like one of the more impressive hacking achievements by a YC-funded co to date.

Re:Etherpad Shows Google Docs How It’s Done (By AppJet) by celoyd on Wednesday November 19, 2008 @10:14PM (#370270)
A master model is not necessarily slower. Everything goes through the server either way, and it comes down to how much data you have to transmit and how much the sender, server, and receiver have to process it.

At http://draftastic.com/ we keep a master but only broadcast changed portions -- a compromise between re-sending the whole document and handling a hail of tiny diffs. This uses automatic locking, which we thought was reasonable given that people rarely want to type directly on top of each other, but it means a much simpler implementation of undo and so on. Basically, we've made a heartbeat-based system with heartbeats so cheap that they can be fast, and we never need a merge algorithm.

In any case, I'll be watching the just-raw-edits approach with interest. Best of luck to Etherpad -- there's plenty of room in the market.

Re:Etherpad Shows Google Docs How It’s Done (By AppJet) by zhyder on Wednesday November 19, 2008 @12:35PM (#369670)
Hopefully the mechanism they chose is amenable to cross-user undo. (Though it's currently missing, I'm assuming single-user undo can be added easily enough.)
Re:Etherpad Shows Google Docs How It’s Done (By AppJet) by aaroniba on Wednesday November 19, 2008 @02:09PM (#369786)
Undo works, just type control-Z (on windows) or command-Z (on a mac). It will undo only your own edits.

We haven't figured out a way to let people undo other users' edits. (We can perform the manipulation on the document, we just haven't figured out a sensible UI for it).

Re:Etherpad Shows Google Docs How It’s Done (By AppJet) by boucher on Wednesday November 19, 2008 @03:29PM (#369879)
I don't think there is a sensible UI for it. I'd leave it out completely.
Re:Etherpad Shows Google Docs How It’s Done (By AppJet) by Retric on Thursday November 20, 2008 @10:28AM (#370647)
Look at MMO's or FPS games to see how fast they can make that model.
Re:Etherpad Shows Google Docs How It’s Done (By AppJet) by ashu on Wednesday November 19, 2008 @08:39PM (#370203)
Paxos-style distributed consensus?
Re:Etherpad Shows Google Docs How It’s Done (By AppJet) by axod on Wednesday November 19, 2008 @10:34AM (#369526)
Agreed. Etherpad is sending a new POST request pretty much every keypress. Considering the HTTP headers and cookies, that's around 500-1k of data per letter typed. At google docs volume that would be a lot.

(Total data sent when I press 'j' was 1073 bytes of HTTP headers, and 123 bytes of program data).

The other reason may be that whilst it looks really cool to see each other typing, is it really needed?

Does look quite slick though :)

Re:Etherpad Shows Google Docs How It’s Done (By AppJet) by drm237 on Wednesday November 19, 2008 @10:48AM (#369538)
I think they're using comet for the requests so while the activity may be high when people are typing, during breaks it should be very low as opposed to ajax polling.

I do think that real-time is really needed for many applications. The few times I've tried collaborating on a google doc, I've given up because the lag introduced the possibility of overwriting each others edits. That and just a general annoyance with saying "Do you see it yet?" over and over.

Re:Etherpad Shows Google Docs How It’s Done (By AppJet) by marcus on Wednesday November 19, 2008 @10:49AM (#369539)
Well that's a stupid way to handle it... keep the socket open... read about Comet. http://en.wikipedia.org/wiki/Comet_(programming)
Re:Etherpad Shows Google Docs How It’s Done (By AppJet) by axod on Wednesday November 19, 2008 @11:02AM (#369551)
Are you being sarcastic? Does anyone know what comet is? Please explain exactly how you can SEND multiple packets over time from browser to server without sending more HTTP requests, and without using any java applet/flash.

It's a shame people seem to have started saying "use comet" without understanding what it means. Did you read that link?

Re:Etherpad Shows Google Docs How It’s Done (By AppJet) by abacad on Wednesday November 19, 2008 @11:18AM (#369576)
Connection: Keep-Alive
Re:Etherpad Shows Google Docs How It’s Done (By AppJet) by axod on Wednesday November 19, 2008 @11:24AM (#369582)
Yes, you can specify keep-alive (Although some browsers refuse to do it in an XHR - Opera for one). However, you still send the full glut of HTTP headers every single request.
Re:Etherpad Shows Google Docs How It’s Done (By AppJet) by marcus on Wednesday November 19, 2008 @12:15PM (#369650)
True, you still need the headers, what you don't need is a new connection every key stroke. Establishing a new TCP connection is a relatively high cost network item (between all the Acks and Syns you need 3 messages have to go back and forth for it to establish), keep the connection alive, the stuff will run a lot faster.
Re:Etherpad Shows Google Docs How It’s Done (By AppJet) by blasdel on Thursday November 20, 2008 @07:13AM (#370534)
You can null out the default headers on a XHR so they don't get sent.
Re:Etherpad Shows Google Docs How It’s Done (By AppJet) by lacker on Wednesday November 19, 2008 @11:20AM (#369579)
Without any java applets or flash?
Re:Etherpad Shows Google Docs How It’s Done (By AppJet) by marcus on Wednesday November 19, 2008 @10:46AM (#369536)
You're forgetting Google didn't develop Google Docs in-house they purchased Writely back in 2006 and that's the codebase Google Docs is built on. Depending on how it was built originally the change might require a massive rewrite...
Re:Etherpad Shows Google Docs How It’s Done (By AppJet) by sant0sk1 on Wednesday November 19, 2008 @10:55AM (#369542)
Was Google Spreadsheets built on top of the Writely codebase? Because their Spreadsheet app is WAY better than their Document app.
Re:Etherpad Shows Google Docs How It’s Done (By AppJet) by marcus on Wednesday November 19, 2008 @11:49AM (#369617)
No that's based on a different acquisition, 2Web Technologies
Re:Etherpad Shows Google Docs How It’s Done (By AppJet) by drm237 on Wednesday November 19, 2008 @10:39AM (#369529)
What's the difference between "something even Google hasn’t been able to do" and "their scaling requirements made it impractical"? When you get past the semantics, it's essentially the same thing. Either way, they're not doing it.
Re:Etherpad Shows Google Docs How It’s Done (By AppJet) by mseebach on Wednesday November 19, 2008 @11:50AM (#369618)
Lack of action doesn't imply inability. That's what's different.

A parked car is not a car that's not able to drive. It's a car that's not driving.

Re:Etherpad Shows Google Docs How It’s Done (By AppJet) by aneesh on Wednesday November 19, 2008 @12:50PM (#369692)
The end user only cares whether its driving. If it's not moving, the user doesn't really care why.