https://automatetheboringstuff.com/
Chapter 12-14 look relevant to you (Excel, Word, and CSV).
The documentation may look daunting, but VBA isn't that difficult to do useful things with. You'll be mostly copying/pasting from google like most professional coders :)
You can start with a macro and turn it into VBA, which is sometimes useful.
I've been using PowerShell to automate various tasks with differing levels of complexity, and I've had a lot of success so far.
It's given me a lot of opportunities to flex my creativity on what was previously a fairly mundane Excel-centric role.
Having a supportive manager is really valuable for this stuff, though.
There are also a lot of options in Azure , but they require a little bit more of coding (They are incredible powerful nonetheless).
I'm not a coder but I would like to automate some of the more mechanical/repetitive aspects of my work, e.g. generating reports from csv files, pulling comments from Word files into a separate document, dashboard creation of live data in Power BI etc.