There are civic organizations that meet regularly, do work within a community, and either minimize or are avoid religious affiliation.
if you're in the US, see if there's a UU congregation in your area; it might be just what you're looking for. https://en.wikipedia.org/wiki/Unitarian_Universalism
I remember it fondly from being a child even though I no longer believe in it.