Agreed.
The word "religion" comes from two Latin words: "res" + "ligare". "res" is loosely "thing" or can be used to refer to a concept (ie, "republic" - "res" + "pubblica" ['thing/item/abstract concept' of the people] and "ligare" is a 1st declension (if I recall correctly) verb meaning "to bind".
To be religious - fundamentally speaking - is simply to follow a set of rules or an ideology; to be bound to something.
While a recurring feature in practice, I don't think believe in a deity should be a defining feature of religion. (It is probably a sufficient feature, though.)