If your database and computations are requiring multiple seconds on a normal web page, you have serious user experience problems. When you're under 140ms, it feels like the response is happening at the same time as the request (Dabrowski and Munson weren't able to reproduce the old 50- or 100-millisecond rule of thumb in what sounds to me like a poorly-controlled experiment; http://books.google.com/books?id=aU0MR-MA-BMC&pg=PA292&#...). Increasing Google search page render time from 400ms to 900ms dropped traffic by 20%, according to Marissa Mayer (http://glinden.blogspot.com/2006/11/marissa-mayer-at-web-20....). Traditional OLTP systems tried to keep response times under one second; beyond a second, people start to get frustrated and wonder if something is broken.
So, for a normal application, the milliseconds you might save by optimizing your database and computations are peanuts in comparison to the second or more that TCP-level optimizations could save you.
It sounds rather poorly-controlled to me too.
They mention that they didn't account for the time from mouse-down to mouse-release. Seriously? Here is a program that can measure that difference: http://www.daimi.au.dk/~sandmann/click.py. It uses GTK+ so you'll probably need Linux to run it. For me, the delay seems to be around 30 ms. They also don't mention the framerate of the screen or whether they controlled for that. On a 60Hz monitor there is a delay of 17ms between frames.
Both 17 and 30 ms are huge numbers if you are measuring intervals on the order of 100ms.
Then there is the question of what you consciously perceive vs. what you subconsciously perceive. It would surprise me if you couldn't measure a difference even in cases where the subjects didn't notice anything themselves.
Finally, we can definitely reject the idea that latencies below 100 ms never matter: there is an obvious difference between a 10 fps animation and a 60 fps one.
This is almost always the case when I think "this website is slow", though. When HN is slow, it's not because of some added network latency, but because something is making HN take 3 seconds to serve up my "threads" page, or 2 seconds to successfully post my comment. Same with "reddit is slow", or "this Wordpress blog is taking forever to load" or "Twitter spins the 'working' icon for 2 seconds when I click on that 'retweets' thing in the sidebar before returning anything". Those things are really common in my experience, and those rather than network round-trip times are by far the biggest and most annoying slowdowns, at least in my browsing.
If IW9 or IW10 gets widely adopted, you'll find yourself thinking "this website is slow" when you visit the rare backwater that doesn't use it.
Unless you are serving static content only (in which case you are hardly creating an "app"), the milliseconds you might save with TCP-level optimizations are peanuts in comparison to the multiple seconds your database and computations will be requiring.