Rate limiting is not useful meaningfully. For a service we ran we regularly had botnets with 100k+ IP addresses making one request an hour to endpoints, which absolutely decimated the backend but hit no limits at all that a real user wouldn't also trigger. Even with a couple of requests an hour you could enumerate the entire phone number space in a very short period with that botnet.

Out of curiosity, how does someone possibly get 100k+ IP addresses? I had enough trouble getting 1 public IP address.

People build botnets by enticing people to install trojans on their computers, e.g. a free utility app or game.

They can then earn money from people who want to rent access to these botnets.

There are also free VPN services who, in their fine print, say that users grant them permission to route other traffic via their connections.

There are "residential proxy services" offering exactly this and you only ever pay for bandwidth. Using 100,000 unique non-datacenter IPs will only cost you few thousand dollars as long as you only sending tiny API requests.

And this is service offered by registered Israeli company that get formal agreement from "bots" to route traffic through them. Very shady, but totally legal service that used by a lot of data collection agencies for price tracking on Amazon or getting data from Linkedin, etc.

botnets. With all of the crappy IoT devices out there it is even easier to get inside of consumers networks.

How do you defend against such an attack? Putting a service behind something like Cloudflare won't bring it down but it will still leak the phone numbers existence, no?

Don't leak whether or not the phone number belongs to an account. All failed login attempts should be some form of "Invalid login" regardless of whether or not it was an attempt against an actual account or not.

Also worth noting that time response deviance when user exists or not can also be a leak of info

Usually you'd try to make the effort/cost no longer worth the data with minimal user impact. For instance, text/email the inputted address with the result instead of displaying it to the requestor through the browser

Or if this functionality needs to return the value, require an authenticated user and impose rate limits based on reputation (which could just be account age)

For instance, Facebook and Twitter used to tell you which profile a phone number belonged to when you put it in the search box (maybe it was this issue). You could restrict that to authenticated users that were 30 days+ old and impose rate limits per day on top of that. A regular user could still look up a few numbers per day but someone enumerating phone numbers would need lots of 1 month old accounts (more effort/cost)

I guess I was thinking more like "limiting the number of attempts" than "limiting the number of attempts over time" -- take time out of the equation (but then NAT causes trouble). But even so, you're right: as the threat landscape approaches the size of the result set, it breaks down no matter what.

That has some problems. If you limit the total number of attempts globally then the feature is effectively disabled, every botnet and script will blow through the attempt budget and real users can't use it. Global limits and IP address limits are not useful, and because we're assuming the user is unauthenticated (using the password reset), we have no other way of distinguishing good traffic.

Captcha comes to mind, but that's a cat-and-mouse game in the age of machine learning (not to mention actual humans working for a bad actor). Cloudflare seems to be on the cutting edge with their newest challenge mechanism, but good vs bad is somewhat distinct from human vs script.

My wife was in charge of security at MySpace back when MySpace was still a thing and there was one occasion that the MySpace team was manually feeding images to a suspected human acting as a bot. As I recall it became clear to both sides that there were humans on the other end and it ended with a picture of a scantily-clad woman and a response of “very funny.”

Solving a few thousand captchas an hour costs you like 200$ per day. Forget about it if someone is dedicated.