Using a password with a per-filesystem salt and a modern key derivation function will be just fine here - the key derivation is expensive enough that brute forcing the key isn't practical, even assuming relatively weak passwords.