There are some sites that generate decent Amazon Associates revenue without millions of visitors — it's a question of whether one of the main interactions users expect to make with your site is buying something from Amazon. Considering what you're building, it's worth at least experimenting with. Other booksellers (like Powells) have their own affiliate programs too.