HN has scaled, but not because pg hasn't worried about scaling (although to be fair, it doesn't seem to have worried much):
"I already have problems enough with that. Remember, the original motivation for HN was to test a new programming language, and moreover one that's focused on experimenting with language design, not performance. Every time the site gets slow, I fortify myself by recalling McIlroy and Bentley's famous quote
The key to performance is elegance, not battalions of special cases.
and look for the bottleneck I can remove with least code. So far I've been able to keep up, in the sense that performance has remained consistently mediocre despite 14x growth. I don't know what I'll do next, but I'll probably think of something."
Re:Ask HN: HN web stack?
by silentbicycle on Saturday December 25, 2010 @08:39AM
(#2038912)
Sure. Profiling and incremental tuning has been sufficient so far. It will probably be hard to scale indefinitely, but HN already handles quite a bit more traffic than many sites.
Re:Ask HN: HN web stack?
by kaiser_pelagic on Friday December 24, 2010 @10:34PM
(#2038460)
using flat files just seems crazy to me, but HN seems to be doing it well.
Why? A database is just an optimized flat file with a query engine.
Because a database is an optimized flat file with a query engine. Its commonly the case that you wish to optimize your flat files' performance and query them. I believe that pg has mentioned a few times that he's 'notorious' for using flat files for this kind of thing. I'd love to see the code for Viaweb and the design decisions in there.
I think about that when people worry about scalability. HN seems to do okay.