Unfortunately no. See e.g. boulanger, cahier, etc. (Although I do believe you're correct about verbs, I wasn't able to think of any verbs that pronounce "er" irregularly).

I don't know of any regular, even semantic, rule off the top of my head that covers when it should be pronounced which way for non-verbs. I wonder if someone more versed in French etymology could find one.