Would it be possible for you to treat the vast amounts written on the internet regarding career choices as your "large dataset" and use your knowledge and tools to explore and analyze this dataset, and visualize it to us?
Presumably you have some for one. Why are you still uncertain?
Trying to Cure Cancer: $25k/yr, bump to $40k after 6 years
Engineering Medical Devices, Airplanes, etc: $60k/yr
Trying to Build the Next Twitter: $100k/yr, $150k/yr after 6 years
Helping Rich People Game the System to Get Richer: $150k/yr, $300k/yr after a few years if successful
The desire to stay in academia comes from having different priorities than the market. Many/most people do. The people who don't usually happen to specialize in a field that the market is currently smiling upon. It's great if your dream can be gently tweaked to be compatible with market considerations, but please understand that for most people this is not the case. There are many Big Problems lurking on the horizon where intermediate progress can't be monetized. Academia lets you work on them. The market doesn't.
This sounds reasonable to me. Most people never have the experience of tripling or quadrupling their salary in a single career pivot, but that's what happens when you decide to bail on your sci/tech graduate research program and start working in the software industry. It takes a pretty crazy level of passion for your research domain to accept such insane opportunity costs. When I realized that by staying in science/academia long term I might never be able to afford to buy a house in a decent neighborhood or have kids it made it really easy for me to quit.
Would warmly recommend it. Visualization is a very large field, so you'll have to carve out some niche. I am in information visualization, which is the branch most generally applicable if you want to do data mining-related things. But there are many other variants: Visualization of scientific/simulation data, e.g. flow rendering, combustion processes, climate simulation, as well as medical fields: CT/MRI volume rendering, real-time 3D ultrasound and quite a few others. Central themes at the moment are using GPUs to implement more advanced 3D volume rendering techniques, or even using GPUs to draw data which is not 3D but where there are performance issues when using CPU alone. For instance, drawing dynamic (25FPS, interactive) scatterplots of large (>1 million records) datasets.
I guess the definition of a "large" dataset varies by context, in visualization you hit this limit earlier than in statistics and non-visual data mining if you use "discrete" methods where every item is drawn on screen.
I have found myself thinking many times that a position in the industry, where I can use my teaching, data processing and analysis skills to further some business goal, seems like a much more preferable option than sitting around writing research papers and applying for grants all day. Not to mention that academia pays less and has worse overtime conditions than any industry job I could concievably get.
This article really nails the key issues for why I am feeling this reluctance towards an acedemic career.