Maybe direct democracy works in a small scale, and by "small" I mean a few dozen people in a town.

Referendum is no panacea. A legislative body can tweak legislation and debate it and compromise. When voting on a referendum there is no such compromise. It's just a binary choice.