^ Do not use cloudsql-proxy ever. GCP docs are wrong. DO NOT proxy all your db requests through a single VM.
Switching to private ip definitely had the largest impact by far on performance.
Interesting. I'm looking Cloud SQL right now and the advice seems to lean in the opposite direction: use public IPs for ease of connecting. Can you quantify the decrease in latency? All I can find is bits about reduced network hops.
I didn't have to worry about the 'ease of connecting' when using CloudSQL as all my GCP services are in the same VPC.
Also, private ip is one less security problem to worry about.
We abandoned cloudsqlproxy partially because the sidecar in Kubernetes caused random disconnects, but also because it just had bizarre network errors sometimes. (That was mostly when connecting from outside GCP though.)
MySQL for our DB.