That water turns out fine, often for weeks at a time.
Store the cold brew out of direct sunlight, don't leave the grounds in for a super long time, and you'll be fine. It's not like brewing alcohol, you're not adding a bunch of sugar to it or anything like that. The coffee grounds you're adding are shelf-stable, the water you're adding it to is shelf-stable, you're not introducing any real food for bacteria/fungus. Water will eventually go off if you leave it in direct sunlight (mostly due to algae), especially after introducing extra organic matter, but the actual risks of following established methods for brewing cold-brew at room temperature are very low in my opinion. Especially chlorinated water like you probably get out of your pipes.
Wet coffee grounds will grow mold quicker than any other food product I've seen.
Even a pot of coffee will get mold on the top within like a week.
More here - https://kitchenappliancehq.com/can-cold-brew-coffee-mold/
Not saying that mold is impossible, but I don't think the solutions is to buy their amazon affiliate coffee maker and special coffee butter.
If you have mold issues after 24 hours of cold brewing I think you have some deeper issues at play. Consider getting mold tests for your house, sterilizing the tools/containers you use, and switching out your beans.
Use the fridge - trust my experience. It may take a bit longer to brew at colder temperatures, due to decreased convection of the liquid, but it will taste much better, as it prevents bacterial and fungal growth.