Homebrew has build servers that compile pre-built binaries.[1] Most of the common software that people install with it (not considering Casks) comes in this form.

It’s not the case that anyone can upload a malicious formula, either. They do review requests to update formulas.

[1] https://docs.brew.sh/Bottles