Ah, I totally misread this line: "Aurora Serverless can scale from a minimum of 2 ACUs to a maximum of 256 ACUs. You can specify the minimum and maximum ACUs your database can consume." I thought it meant you had to always run 2 ACUs.
My bad.
My bad.
Cold-to-warm takes 25 seconds, but for dev/test this shouldn't be a problem.