News for hackers. Stuff that matters.

Ask HN: HN web stack?

I'm curious about the HN web stack. I know they use arc as the icing but I have no idea what is underneath.

server: Apache(mod_scheme) or plt(racket) web server are my two guesses db: MySQL, PostgreSQL - I read somewhere that they use MySQL

( 19 comments | 31 points | on HN )
Re:Ask HN: HN web stack? by jjoe on Friday December 24, 2010 @10:36PM (#2038463)
It's running the Arc web server, which has its own implementation of the HTTP protocol. I don't think there's anything behind or in front of it. Probably not even Memcached. Think of the app serving HN as a framework. Here's how a "URL" entry would be implemented in Arc:

(defop hi req (prn "Hi there"))

http://news.ycombinator.com/whoami will return your login and IP. It's probably implemented in this fashion:

(defop whoami req (prn (req 'ip)) (prn (req 'id)))

Regards

Joe

Re:Ask HN: HN web stack? by silentbicycle on Friday December 24, 2010 @10:27PM (#2038454)
No database. Just flat files cached in memory on demand, IIRC.

I think about that when people worry about scalability. HN seems to do okay.

Re:Ask HN: HN web stack? by j_baker on Friday December 24, 2010 @11:30PM (#2038518)
HN has scaled, but not because pg hasn't worried about scaling (although to be fair, it doesn't seem to have worried much):

"I already have problems enough with that. Remember, the original motivation for HN was to test a new programming language, and moreover one that's focused on experimenting with language design, not performance. Every time the site gets slow, I fortify myself by recalling McIlroy and Bentley's famous quote

The key to performance is elegance, not battalions of special cases.

and look for the bottleneck I can remove with least code. So far I've been able to keep up, in the sense that performance has remained consistently mediocre despite 14x growth. I don't know what I'll do next, but I'll probably think of something."

http://paulgraham.com/hackernews.html

Re:Ask HN: HN web stack? by silentbicycle on Saturday December 25, 2010 @08:39AM (#2038912)
Sure. Profiling and incremental tuning has been sufficient so far. It will probably be hard to scale indefinitely, but HN already handles quite a bit more traffic than many sites.
Re:Ask HN: HN web stack? by kaiser_pelagic on Friday December 24, 2010 @10:34PM (#2038460)
using flat files just seems crazy to me, but HN seems to be doing it well.
Re:Ask HN: HN web stack? by cloudhead on Saturday December 25, 2010 @04:41AM (#2038737)
Why? A database is just an optimized flat file with a query engine.
Re:Ask HN: HN web stack? by rlmw on Saturday December 25, 2010 @07:03AM (#2038855)
Because a database is an optimized flat file with a query engine. Its commonly the case that you wish to optimize your flat files' performance and query them. I believe that pg has mentioned a few times that he's 'notorious' for using flat files for this kind of thing. I'd love to see the code for Viaweb and the design decisions in there.
Re:Ask HN: HN web stack? by Darmani on Friday December 24, 2010 @10:17PM (#2038442)
http://www.arclanguage.com/install

You can read the source code yourself. I don't remember the database, but it's running on an Arc server.

Re:Ask HN: HN web stack? by jackolas on Saturday December 25, 2010 @12:02AM (#2038544)
Being proxied to Server:Apache/2.2.8 (Ubuntu) fyi

edit: I don't think the page itself is actually proxied it seems only the static content is.

Re:Ask HN: HN web stack? by ryan-allen on Saturday December 25, 2010 @04:27AM (#2038724)
If the app is behind apache and writing static files for serving things up, and only hitting arc on operations like votes and cache misses then of course it's gonna scale! Especially if the app is read heavy :)

Is the HN source code actually open source itself?

Re:Ask HN: HN web stack? by jackolas on Monday December 27, 2010 @02:46AM (#2042030)
yes its in the arc source
Re:Ask HN: HN web stack? by kaiser_pelagic on Friday December 24, 2010 @10:38PM (#2038465)
I have taken a look at the source, but wasn't sure if they were using all of it in production. If they are then quite impressive.
Re:Ask HN: HN web stack? by epochwolf on Friday December 24, 2010 @10:20PM (#2038445)
I thought the database was flat files.
Re:Ask HN: HN web stack? by known on Saturday December 25, 2010 @12:50AM (#2038591)
Yes, everything is a file in UNIX
Re:Ask HN: HN web stack? by omnix on Friday December 24, 2010 @10:27PM (#2038453)
or OP could be trying to hack hackernews
Re:Ask HN: HN web stack? by jrockway on Friday December 24, 2010 @10:33PM (#2038457)
And what a crisis that would be. All of our publicly-available comments would become available publicly!
Re:Ask HN: HN web stack? by dholowiski on Saturday December 25, 2010 @12:37AM (#2038582)
How about all of our hashed passwords- that would be a treasure trove
Re:Ask HN: HN web stack? by cypherpunks01 on Saturday December 25, 2010 @01:10AM (#2038599)
Re:Ask HN: HN web stack? by RossM on Saturday December 25, 2010 @08:32AM (#2038906)
I don't see why you're being downvoted, your comment is obviously sarcastic.