The brain is definitely not doing gradient descent. That is the biggest issue with using artificial neural networks as a neuroscience modeling tool. The fact other learning algorithms have not come close to gradient descent performance in most cases has therefore largely kept neuro theory as a niche in modern academic research -- despite its entanglement with the early development of artificial neural networks.

Not taking a stance on whether "intelligence" can emerge out of gradient descent but it's certainly not biological.