Well you can't get around that considering that `freq` is the result of aggregation. Therefore you do have to "take all available results". Or you could keep a tally for all users and refresh it periodically.

At first glance, it seems to me like the query would be much faster if joined activity_types and activities were filtered subqueries instead.

Also, activities.user_id = #{id} might be better handled with an SQL CASE statement as opposed to using the .to_i conversion in Ruby.

I have similar feelings about subqueries. This is a rather elementary standard join query, and I always felt subqueries were only tacked onto SQL because people had trouble understanding joins.

The case statement would only be there to set it to a numeric 0 when it's an unusual value like an apostrophe - which is what the OP said it did in a comment somewhere else in this discussion (https://news.ycombinator.com/item?id=14499589).

>> I have similar feelings about subqueries. This is a rather elementary standard join query, and I always felt subqueries were only tacked onto SQL because people had trouble understanding joins.

Not exactly sure what you meant by similar feelings (did you mean you also don't get the idea about it?). In the absence of knowing MySQL's execution plan for the OP's query, I tend to think using subqueries that reduce the rows of activity_types and activities would make the join, aggregation and ordering faster (assuming the tables are already properly indexed). I've seen this reduce query times on other database engines for similar types of queries.

Indices are based on the same logic, i. e. "99% are reads, let's try to do more work on writes".

Redis is great. But to be honest: so is MySQL. If you give MySQL enough RAM and create the right indices, you can easily get to 6-digit numbers of queries per second.