News for hackers. Stuff that matters.
Re:Ask HN: How do I handle a million hits a minute? by igorhvr on Wednesday November 12, 2008 @12:40PM (#362041)
"new record is appended, a search is performed and the post field is sorted, then the relative position of the visitor's new database record is identified in the results (example 49,501 records from the top)."

It might be a good idea to try to twist things so that no sort is necessary - instead of appending and then sorting you should try to insert the data right at the spot it should be. If your constraints allow you to not use a database, you could consider a data structure that inserts data in such a way that it is always sorted.

Also, as an improvement to your point 1, you should put most of the JavaScript itself as well as images, etc somewhere else - with this your bandwidth will be used only for the dynamic data (which will be more than enough).

Finally, the best tool I know for scalability are queues. Try to reorganize what you are doing to arrive at an architecture where you put things on queues that multiple consumers will process.

Re:Ask HN: How do I handle a million hits a minute? by owkaye on Wednesday November 12, 2008 @12:50PM (#362050)
Yes, eliminating the sorting is a better way to approach the problem I believe. I've always just 'sorted' to determine a new record's position in the list, but it seems there may be other (more efficient) ways to determine the record's position without sorting ...
Re:Ask HN: How do I handle a million hits a minute? by mleonhard on Thursday November 13, 2008 @01:33AM (#362784)
Do all of the rankings need the same precision? Maybe you can get away with perfect accuracy for the top 1,000 rankings and then provide an approximation for anything >1,000?

You could use a list for the first 1,000 and then store the rest in a tree. Then have another process that walks the tree in-order and updates the ranking of each node.