The difficulty with not going into a formal program for an area you already have exposure to (say, physics, computer science, finance) is that for most people, at 17, you don't know what you don't know. The benefit of a curriculum is that the larger community you wish to enter has already figured out: this is the minimum knowledge people need to work with us.
Personally, I did not know what I did not know, and I sincerely doubt I would have learned on my own what I did learn, without the guidance of course curricula. Even though I work as a computer scientists, and most of my education is in computer science, I am enormously happy that I also minored in physics in college. I learned just enough to appreciate it (hopefully) better than the layman. On my own, I doubt that would have happened.
Its like in a discussion about fuel efficient cars, pointing out the death rate is slightly lower in a land barge and its real nice for carrying my skis. Yes, but that'll also "destroy the planet" if all 6 billion of us acted that way. Nobody wants to live in the high crime city so we'll "ALL" move out, leading to guess what in the burbs?
Evolution doesn't create perfect solutions to problems, college is by no means perfect, but they do create effective ones, whereas designed solutions often aren't effective because they didn't take into advantage some important premise or edge case. Take this into account when you design the path you're going to take through your early twenties.
It not be a positive argument, but it does set a floor for considering alternatives. Whatever you do end up choosing to do with your post-high school years, if it's not college, has to beat what college offers you.
There's a lot to consider. Are you good at motivating yourself to do stuff? Can you think usefully at a high level? Can you chart your own course, using only resources that you, yourself can acquire and integrate, and get somewhere in the four years you'd be giving up by going to college?
Then there's there's this: Are the questions you're asking yourself regarding the decision going to remain relevant after the new reality sets in? Can you trust your 17-19 year old self to make decisions?
I chose not to go to school, and that decision threw me into a purgatory that lasted until the end of my twenties. I lost my goals and ambitions and had to find them later. Now I've found personal traction and am working on personal product-market fit.
I don't know how my life would have been different if I'd have gone to school. I like my life now but I can't use that to say that the college decision I made was good or bad. Maybe it would have been better. Maybe I'd have found the traction I enjoy now earlier had I gone to school. Maybe I would have never found it.
Maybe it's a wash. In that case, it would have been better to go to school, because at least I would have had better jobs and situations while sorting through all my personal stuff to find traction.
School is a good default decision to make in a world where there are very few good default decisions available to make. Think long and hard before you decide to not take advantage of that.