It would be better to store all the Client Secrets in such a Secrets Manager right? Or should I create an encryption key per user and store that in the Secrets Manager?

Best practice is to do envelope encryption where you encrypt the data with one key, then you encrypt that key with another (hence envelope). That allows you to routinely rotate the outer key without having to laboriously re-encrypt the actual data.

Whether you use one envelope key or one data key per client, one data key for all, or one envelope key for all is really a judgement call and depends on how paranoid you want to be vs. how much you want to worry about juggling keys.

Personally I don’t go per-user, but if you have the concept of a company/tenant/etc. I might do one for each of those. If you’re storing each tenant in a different database or region it lends itself more towards having a separate key for each as well.

Wouldn’t you have to re-encrypt if the envelope key changes?

Specifically I would think the decrypted data key is derived from the encryption key.