News for hackers. Stuff that matters.
Re:Breaking the web with hash-bangs by raganwald on Wednesday February 09, 2011 @12:16AM (#2196460)
I accept the brittleness of a content-oriented site switching to a javascript-driven SPI implementation, however I'm confused by the assertion that the strongest reason to switch is because it's cool. I can't speak for Gawker, but some sites may switch because they want the browser to do more of the rendering work, such as rendering templates. Others may want to avoid the overhead with a full page refresh as users navigate the site.

The author clearly dislikes what's going on, and the post would be stronger if he simply stated the disadvantages and let others speak to the benefits rather than putting up a strawman and claiming that people implement this sort of site because it's cool.

Re:Breaking the web with hash-bangs by bad_user on Wednesday February 09, 2011 @04:11AM (#2196895)

       Others may want to avoid the overhead with a full page 
       refresh as users navigate the site.
Avoiding a full-page REFRESH only makes sense when the current page does NOT change (e.g. doing login).

Speaking about overhead, to avoid bandwidth problems related to page-switching 90% of websites out there would be better off if they just added the Cache-Control on their static assets or GZip the content sent (yes, it's kind of ironic that so many developers ignore common-sense techniques already available in the HTTP protocol).

Also, if the overhead is in the server-side processing done, then offload that processing making it async and cache the results. Why is that so hard?

For me Twitter's new interface is slower and more cluttered than the old one, without providing the benefits of the iPhone client I'm using. It's a disaster IMHO, so speculating they only did it because it's cool is not unreasonable.

Re:Breaking the web with hash-bangs by raganwald on Wednesday February 09, 2011 @06:40AM (#2197131)
Good, healthy observations, thanks.

Avoiding a full-page REFRESH only makes sense when the current page does NOT change (e.g. doing login

User navigates from "A" to "B" Some or even most content on the page changes. Scenario 1: New page is fetched from the server. Scenario 2: Content is fetched from the server via AJAX and rendered with a template in the browser, then injected into the current DOM.

I assert there are technical benefits to the second scenario. Whether they outweigh the disadvantages is not my argument. My argument is that there are technical benefits and that "because it's cool" is a straw man argument.

if the overhead is in the server-side processing done, then offload that processing making it async and cache the results. Why is that so hard?

Yes, and one such strategy is to offload it to the user's browser. Again, I am not arguing it is the best strategy, as I am not responsible for Gawker Media or your application.

Re:Breaking the web with hash-bangs by mcantor on Wednesday February 09, 2011 @10:07AM (#2197600)
Didn't the article author assert that you can use click handlers on your links to achieve AJAX loading without a full page refresh?
Re:Breaking the web with hash-bangs by Nagyman on Wednesday February 09, 2011 @12:29PM (#2198311)
Indeed, but then the URL doesn't change, so the page can't be emailed, tweeted, etc. The hash-bang allows you to change the url, use AJAX to change the content, and retain browser history.

The history API in HTML5 allows you to accomplish the same thing without the hash-bang, but browser support is limited. I favour using the history API for progressive enhancement and defaulting to traditional page requests.

Re:Breaking the web with hash-bangs by nailer on Thursday February 10, 2011 @08:39AM (#2201884)
+1 for history API.

It's unbelievably neat to transition between pages without ever redrawing the full screen. Pity there won't be any IE9 support.

Re:Breaking the web with hash-bangs by raganwald on Wednesday February 09, 2011 @12:27PM (#2198305)
Yes of course he did, however every design decision has trade-offs. Using click handlers has some costs and some benefits. Using URL fragments has costs and benefits.

I'm not disputing that the author has proposed a way forward or that the author's suggestions about the costs of using fragments are mistaken, I am only disputing the argument that there is no reason to use url fragments besides "coolness."

Re:Breaking the web with hash-bangs by rimantas on Wednesday February 09, 2011 @07:22AM (#2197198)
It depends (of course). First of all, markup wise page content can take only a small part of the whole page HTML. Navigation, widgets, etc. can be substantial.

Also, you'd want Expires, not Cache control. Otherwise you may end sending dozens of request just to get 304 back.

Keeping that in mind, sending single AJAX request which delivers _only_ content seems quite beneficial compared to full refresh scenario.

Re:Breaking the web with hash-bangs by bluesmoon on Wednesday February 09, 2011 @02:27AM (#2196738)
some sites may switch because they want the browser to do more of the rendering work, such as rendering templates. Others may want to avoid the overhead with a full page refresh as users navigate the site

Both of these can be achieved using progressive enhancement. There is no technical reason to include it in the page itself.

Re:Breaking the web with hash-bangs by daleharvey on Wednesday February 09, 2011 @03:07AM (#2196800)
there are technical problems with progressive enhancement, the latency between page load and progressive enhancement being applied being a fairly obvious one, the other is the providing full html fallbacks combined with progressive enhancement can be very complicated and time consuming, its very easy to see time when time to live wins out over semantic correctness

hopefully html5 pushstate will alleviate a lot of the complications with providing full fallbacks

Re:Breaking the web with hash-bangs by kls on Wednesday February 09, 2011 @12:26PM (#2198297)
As well the elephant in the room that gets missed is that many rail about the technical correctness without taking into account the target market. Personally I build web applications that replace desktop applications. Progressive enhancement would kill the simplicity of our programming model and literally triple the work effort because we would be in the hybrid world of half server side and half client side.

As it sits, we use a full client side UI library and push all UI concerns into the UI, they communicate with the server via REST to get data and the client is responsible for rendering that data. We research our numbers extensively and the cost analysis of changing our development model to support clients that cannot support full JavaScript clients supports the conclusion "at least for us" that a progressive enhancement development model is not cost effective.

The money recouped with the shortened development cycle of Ajax clients outweighs the amount we would receive chasing some fraction of < 2% of potential clients, If we get to the point that we need to chase that 2% for growth we are a) doing great and b) would be better served segmenting that traffic to a completely independent server framework based UI that it built for those clients.

It is my belief that mixing the two development models gives you the worst of both worlds when it comes to features and maintainability.

Re:Breaking the web with hash-bangs by bluesmoon on Wednesday February 09, 2011 @05:41AM (#2197031)
the latency between page load and progressive enhancement being applied shows up primarily if you load your JavaScript at the bottom of the page. Now I know that this is what Yahoo!'s performance rules suggest, I was on the team that made those recommendations. They made sense between 2006-2009, but browsers have improved significantly since then and that rule isn't necessarily true any more. You can load scripts in the head of your document without any of the blocking issues we saw in the past. You can even do multiple scripts in parallel to reduce download latency. See my <a href="http://calendar.perfplanet.com/2010/thoughts-on-performance/... on performance</a> on the 2010 performance advent calendar.
Re:Breaking the web with hash-bangs by iron_ball on Wednesday February 09, 2011 @08:09AM (#2197287)
Interesting, thanks. Does "browsers have improved significantly since [2006]" mean that we still need bottom scripts for IE?
Re:Breaking the web with hash-bangs by bluesmoon on Wednesday February 09, 2011 @11:22AM (#2197968)
IE6 is turning into a C grade browser this quarter (http://yuiblog.com/blog/2010/11/03/gbs-update-2010q4/) so you can probably just not serve JavaScript to it (server side detection). IE7 still doesn't do parallel script downloads, so you might need to do bottom loading for it, but it's best to first find out what your users use before optimizing for it. You can get really complicated and change your page structure based on useragent and user's network characteristics, but that requires far more development effort.
Re:Breaking the web with hash-bangs by eli on Wednesday February 09, 2011 @01:55PM (#2198756)
Gawker redirects IE6 to their (crappy) mobile site, for what it's worth.
Re:Breaking the web with hash-bangs by strager on Wednesday February 09, 2011 @10:56AM (#2197832)
Nice remark; thanks!

Full (unbroken) link: http://calendar.perfplanet.com/2010/thoughts-on-performance/

Re:Breaking the web with hash-bangs by ZoFreX on Wednesday February 09, 2011 @04:31AM (#2196917)
The author does his point a disservice by being under-informed, imo. I agree that Gawker's redesign is horrible, and share his hate of needless javascript (why re-implement half the browser stack?), but his suggestions for improvement would actually be a reduction in functionality.

He suggests using real URLs - instead of "/#!5753509/hello-world-this-is-the-new-lifehacker" we would have "/5753509/hello-world-this-is-the-new-lifehacker". Sounds great, but what happens when you click to a new article, which is loaded with AJAX?

> Engineers will mutter something about preserving state within an Ajax application. And frankly, that’s a ridiculous reason for breaking URLs like that.

Is it? Let's try things all possible ways (with the assumption that content is loaded via AJAX) and see what happens. Firstly, the current state:

Address bar: "/#!5753509/hello-world-this-is-the-new-lifehacker", click on a link, now it's "/#!downloads/5753887". I can bookmark this, send the link to a friend, etc etc.

Option two is to not track this at all, after all preserving state isn't that important, right?

Address bar: "/#!5753509/hello-world-this-is-the-new-lifehacker", click a link, it doesn't change. I can't bookmark the new page, I can't send the link to a friend, my back button won't work properly.

Third option is to use real URLs as he suggested... we need to track state (otherwise we hit the same problems in option 2) so let's try that: Address bar: "/5753509/hello-world-this-is-the-new-lifehacker", click a link, it now contains "/5753509/hello-world-this-is-the-new-lifehacker#!downloads/5753887". Oops. That's even less clean.

Conclusion: If you must load content with AJAX, using URL fragments to track state is the most functional and cleanest-URL option available.

Re:Breaking the web with hash-bangs by Isofarro on Wednesday February 09, 2011 @05:02AM (#2196977)
"The author does his point a disservice by being under-informed, imo."

I admit my Google searching turned up no useful results about hash-banging URLs for new websites, only Google's attempt to index sites that ignored progressive enhancement and hid all content behind Ajax calls.

"Sounds great, but what happens when you click to a new article, which is loaded with AJAX?"

If you want to load the content in via Ajax, you start with a valid and direct URL to a piece of content. You can then use JavaScript on the click handler to mangle the URL into a hash-bang, and then proceed as you wish. You get all the benefits of hash-bang without the downside of uncrawlable URLs.

"now it's "/#!downloads/5753887". I can bookmark this, send the link to a friend, etc etc."

* Bookmark it to a third party service - yes (assuming they don't truncate fragment identifiers!) * That service then retrieves the content of that article (for summarising, entity recognition, extracting microformatted data). no.

The hash-bang URL is not a direct reference to the piece of content. The indirection shenanigans isn't part of HTTP/1.1 or RFC-2396. So it's not Restful for starters.

"after all preserving state isn't that important, right"

That can be done without making all links on the page non-traversable with a crawler.

"Conclusion: If you must load content with AJAX, using URL fragments to track state is the most functional and cleanest-URL option available."

a clean URL would not be an indirect reference that would need to be reformatted into a direct reference before being curled.

State can be done without mangling URLs in this way.

Re:Breaking the web with hash-bangs by ZoFreX on Wednesday February 09, 2011 @08:24AM (#2197310)
> You can then use JavaScript on the click handler to mangle the URL into a hash-bang, and then proceed as you wish

I wish, unfortunately this is only supported in the very latest browsers (I've literally seen one website do this [1]). You can only change the portion of the address bar following the "#" with javascript, which is why you see URLs like "/content/30485#!content/9384" in some places.

The short of it is, if you insist on using AJAX to load your primary content, and to power your primary navigation, then the way they do it is the best method at our disposal at this time.

Personally, I don't think the best method is good enough, but right now there is no better solution - imo, the only way to avoid this nastiness is not to load your content using AJAX in the first place! I don't see the benefit in throwing away half of HTTP, all of REST, breaking inter-operability and SEO, requiring javascript, and having ugly URLs, in order for... what? It's hard to see the plus side to be honest, this is categorically the Wrong Way to make a website and I wish people would stop doing it!

[1] http://www.20thingsilearned.com/

Re:Breaking the web with hash-bangs by Isofarro on Wednesday February 09, 2011 @09:57AM (#2197564)
"I wish, unfortunately this is only supported in the very latest browsers (I've literally seen one website do this [1]). You can only change the portion of the address bar following the "#" with javascript, which is why you see URLs like "/content/30485#!content/9384" in some places."

Okay, you are clearly missing the solution here.

This is what the html looks like: <a href="path/to/page">Page</a>

Then add a click handler to the link (using your preferred JavaScript library): $('a').click(function(e) { this.href = "#!" + $(this).get(0).href; });

So this code mangles your URL into a hashbang with JavaScript when the link is clicked. Now the rest of your code remains exactly the same, and updates the window.location.hash the same way as before - it gets the same value as before.

Except the benefit is that with no JavaScript - like a crawler - sees a working URL. And the JavaScript then mangles the URL appropriately, leaving your framework blissfully unaware of the difference.

This is progressive enhancement - using JavaScript to mangle links that are only meant to work when JavaScript is available (and in Gawker's case, make it to the browser). If you're quite confident in your JavaScript skills, you can mangle all the links on the page in one go right at DOMReady.

The key is not expecting JavaScript to be running. And when JavaScript is indeed running, then mangle links to your heart's content.

Re:Breaking the web with hash-bangs by ZoFreX on Wednesday February 09, 2011 @11:07AM (#2197892)
Ah yes, I did misunderstand what the OP and you meant by that. Yes, progressive enhancement would be preferable, I (almost) always try to use the PE approach myself.

The problem with having this mix though is that someone could visit "/45382/some-content-url" with JS turned on and then click a mangled link - taking them to "/45382/some-content-url#!94812/some-other-content" which is absolutely horrible.

As far as I'm concerned, the only real solution is to not use javascript for loading pages rather than, you know, invoking page loads. I really don't see why this method is beneficial at all, there is nothing of merit in the design or functionality of Gawker in my opinion.

Re:Breaking the web with hash-bangs by miniatureape on Wednesday February 09, 2011 @10:34AM (#2197709)
Along these lines, one pattern is to build a widget or page that renders in straight html and has links to take you to pages to perform actions. If javascript exists, scoop up the html and introspect it to build your rich widget. If the javascript never loads, you have a working page.

This, I believe is a good use of custom data attributes. When your JS introspects the html, it can use data here for configuration and setting values. It's messy and often impossible to use the same values you output to html in your javascript models. (For instance you might have a Number with arbitrary percision on your model, but only spit it onto the page with decimal points.)

But, yeah sure. It's a some extra work and planning we don't always have time for.

Re:Breaking the web with hash-bangs by sjs on Wednesday February 09, 2011 @10:18AM (#2197641)
> He suggests using real URLs - instead of "/#!5753509/hello-world-this-is-the-new-lifehacker" we would have "/5753509/hello-world-this-is-the-new-lifehacker". Sounds great, but what happens when you click to a new article, which is loaded with AJAX?

I would deliver the page with plain old HTML links to other pages. On page load I'd use JavaScript to see if the new pushState[1] API is available, and if so fetch the JS required to update the page dynamically and then update the link elements to do the AJAXy thing. Seems like it would degrade gracefully in old browsers and keep the URL current in new browsers.

[1] https://developer.mozilla.org/en/DOM/Manipulating_the_browse...

edit: is sjs382 my doppelganger?! (or vice versa)

Re:Breaking the web with hash-bangs by ZoFreX on Wednesday February 09, 2011 @11:13AM (#2197916)
Yes, if I were forced to use JS for loading the primary content I would do that too. There are a couple of options for doing so if you don't want to roll your own (and you don't - it is actually a very complex thing to get working cross browser), JQuery History [1] which is nice and simple, and JQuery BBQ if you need a bit more control.

Personally I think it's a little early for pushState, and degrading to #! is really ugly, so for now I would say just use normal page loads like everyone else unless you have a really good reason (good reason: GMail. bad reason: glorified blog.)

[1] http://tkyk.github.com/jquery-history-plugin/ [2] http://benalman.com/projects/jquery-bbq-plugin/

Re:Breaking the web with hash-bangs by sjs on Wednesday February 09, 2011 @04:45PM (#2199489)
Agreed. I meant degrade to regular URLs, not the #! thing. IMO you don't need the #! for something as simple as switching articles.
Re:Breaking the web with hash-bangs by sjs382 on Wednesday February 09, 2011 @10:32AM (#2197695)
edit: is sjs382 my doppelganger?! (or vice versa)

Haha, not at all. Just my old university ID since "sjs" was taken here ;)

Re:Breaking the web with hash-bangs by sjs on Wednesday February 09, 2011 @12:34PM (#2198343)
sjs can be a hard handle to get. had to resort to _sjs on twitter, and that was in 2007 :)
Re:Breaking the web with hash-bangs by nailer on Thursday February 10, 2011 @08:54AM (#2201929)
Well now you know who to blame :^).
Re:Breaking the web with hash-bangs by sjs382 on Thursday February 10, 2011 @08:25PM (#2204735)
Not me. :p I'm sjstrutt on Twitter
Re:Breaking the web with hash-bangs by sjs382 on Wednesday February 09, 2011 @10:09AM (#2197609)
He suggests using real URLs - instead of "/#!5753509/hello-world-this-is-the-new-lifehacker" we would have "/5753509/hello-world-this-is-the-new-lifehacker". Sounds great, but what happens when you click to a new article, which is loaded with AJAX?

Well you've already confirmed that the client is running Javascript so:

1. Use JavaScript to rewrite the existing clean URLs into JS function calls (or however you want to handle that)

2. Use window.history.pushState to push the clean (canonical) URL for the page you're viewing to the browser.

I'm not sure why this isn't obvious...

Re:Breaking the web with hash-bangs by gnubardt on Wednesday February 09, 2011 @10:18AM (#2197643)
history.pushState() and history.replaceState() are introduced in the html 5 spec, so not all browsers support it yet.
Re:Breaking the web with hash-bangs by sjs382 on Wednesday February 09, 2011 @10:33AM (#2197699)
Test for the API and fall back to the hashbangs if it isn't available. Check out this example in an old browser and a modern one:

http://www.asual.com/jquery/address/samples/express/portfoli...

Re:Breaking the web with hash-bangs by ZoFreX on Wednesday February 09, 2011 @11:15AM (#2197926)
My point was the degradation to hashbang (which is what most users are going to see) is absolutely horrid, to the point that I would actively avoid it.
Re:Breaking the web with hash-bangs by sjs382 on Wednesday February 09, 2011 @12:25PM (#2198284)
You're concerned about /content/123/#!/content/456/, right?

Well, if the client isn't modern, 301 them to #!/content/456/ and proceed as you would.

Either way though, all this is terrible for a blog, or even Twitter.

Re:Breaking the web with hash-bangs by sunchild on Wednesday February 09, 2011 @04:22AM (#2196908)
The author also completely misses the business case: putting a single gateway through which Google can index makes sense for huge content generators. Sites like Twitter actually have some leverage with Google, as the Twitter firehose is more "real time" than Google's index. Using the #! allows Twitter tight control over Google's access to content. As for Gawker, I suspect they overestimate their leverage with Google, and have no similar business case.
Re:Breaking the web with hash-bangs by Isofarro on Wednesday February 09, 2011 @04:42AM (#2196937)
"The author also completely misses the business case: putting a single gateway through which Google can index makes sense for huge content generators."

That's called a domain name. www.lifehacker.com is a single gateway to all of lifehacker's blog posts. Then you use the path part of the URL to specify which post or collection of posts you want.

Twitter hasn't leveraged anything. The tweet URLs without the hash-bangs were more accessible to Google.

You want tight-control as to which user-agents can see which set of tweets - that's what the Controller is for on the server.

Re:Breaking the web with hash-bangs by ceejayoz on Wednesday February 09, 2011 @10:36AM (#2197720)
sitemap.xml and robots.txt offer much, much more control over Google's crawling, and require essentially no dev work.
Re:Breaking the web with hash-bangs by sunchild on Friday February 11, 2011 @09:14AM (#2206353)
And can be ignored...
Re:Breaking the web with hash-bangs by ceejayoz on Friday February 11, 2011 @08:06PM (#2208902)
The hashbang technique could be easily worked around by a company with the resources of Google, and the PR ramifications of GoogleBot violating robots.txt would be pretty big.