At least in the US, the cost to start a business is chickenfeed. If you do business under your own name, e.g., as an artist or a consultant you can probably start today at no charge. The IRS only cares that you fill out a Schedule C at tax time. If you want to have a business name, it can cost a bit more, but typically less than $100. Change your legal structure? In my state it cost me, IIRC, about $150 to file the paperwork for an LLC and my accountant charges an extra $100/year to do my taxes.

The IRS has a lot of online information on starting a business, but it's buried in multiple documents. The best overall guide I know of is Publication 334: https://www.irs.gov/publications/p334

I think it's a matter of personality. Some people want everything analyzed before they start, but I'm not much of a planner. I just jump in and see how the water feels.