In every implementation of agile that I've ever seen or encountered, something like this occurs:

"Okay, this story for Feature X, how many points should we give it?"

"Well, it'll probably take N days, so, let's say, M points."

Eventually, for the sake of consistency, a rough translation between points and time intervals is established.