The reason energy and AI are more attractive than healthcare has little to do with "identity", "culture" or "positionality". It has much more to do with the fact that the field is chock full of incidental complexity that makes doing any kind of good work an extremely unrewarding slog. Look at the 834 EDI format [1], for example. Does anyone want to work with a file format as nasty as this? Does anyone want to work in an industry where data is exchanged by uploading files to FTP servers, and waiting a day to see if it was successfully processed? And finally, does anyone want to work in an industry where they have absolutely zero power to change any of the above?

As engineers and humans, we like to do good work, in a field where we have the autonomy to choose best practices. Health care makes it difficult to do that. I don't think it's productive to call people out for choosing not to work in a field where they have to deal with arbitrary and capricious rules, ignorant bureaucracies, and massive amounts of incidental complexity which has nothing to do with the primary objective: keeping people healthy.

[1]: https://en.wikipedia.org/wiki/ANSI_834_Enrollment_Implementa...

Wait a second... what’s stopping you from changing —this— situation then? Solve the data format problem. Make it easier for everyone to exchange data in meaningful, efficient ways.

Sounds like a terrible pain that you understand well enough to be able to solve. When you have, it might remove a major bottleneck and open the floodgates for more smart people to join, ultimately benefitting healthcare in a massive way.

Make people want to work in that industry :)

I used to work with a healthcare company and dealt the 837 file format and FTP batches and wrote a pretty neat-o parser for it, the reason is simple- inertia. And it's not like people aren't trying because there's been a huge push to get REST-like APIs in place, but when you have close to 40 years of software infrastructure in place it's non-trivial to explain to the boss why you need to spend money on something that gets everyone paid either way.

I think it would be truly revolutionary if we could change the format, but what I think would be almost a better step is to write open source parsers that can accurately translate X12 into something sane and keep the interchange the same so that everything can be balanced out like a checkbook. Check out the 835 format for this part as it's supposed to be a data stream of transactions. Too bad the dozens of flavors of X12 are locked behind licensing and thousands of dollars of books for the rules :/

Additional reading on this god-awful data format: https://docs.oracle.com/cd/E19398-01/820-1275/6ncv5s178/inde...

oh man I'm having flashbacks to one of my first jobs after school. I spent a year writing a parser for that abomination of a format. Before I left I'd build a front-end and processing queue that did X12 -> XML and several modules to output the same data in various formats and flavors to suit the taste of various insurance companies... Parsing X12 was definitely among the least enjoyable things I've ever had to do. At the time I did it for way too little pay.

Short answer: hospitals, insurance companies, governments

Long answer: It's not like people don't understand that the current situation is suboptimal. It's that any kind of change implemented unilaterally will leave you unable to communicate with your counterparties. The severe consequences of this (not getting paid, being out of compliance with HIPAA) means that any kind of change to health-care IT at a system-wide level requires an absolutely massive amount of politicking and often fails to improve the situation.

The problem is that health care institutions are so interlinked that driving any kind of global change is a process best measured in decades. Combine this with the fact that the culture in health care is very conservative, and people tend to be hostile towards change by default and you have a situation where it's often easier to leave the industry and save your sanity than to remain and continue bashing your head into a brick wall over and over again.

The more interconnected the greater the energy required to change from the groundup but trivial to change at the top (submit in this format or no payment)

Except that health care has no "top". If an insurance company tried to unilaterally impose a new standard, they'd quickly lose support among hospitals, and then they'd lose their subscribers shortly thereafter.

With a [mostly-]single-payer system (i.e. pretty much everywhere in the world except the US) there is a clearly defined "top" that can unilaterally require particular standards for e.g. electronic health records.

Also, for USA, legal requirements (such as HIPAA) are universal and can impose certain standards on the whole industry, as long as there's political will to do so.

> Solve the data format problem. Make it easier for everyone to exchange data in meaningful, efficient ways.

https://xkcd.com/927/

The whole field feels like such a mess to deal with what basically boils down to text/numerical data. You could probably spend the rest of your life just writing parsers to consolidate different data formats. I truly don't believe healthcare will be solved by technology until there is a single ubiquitous format. It's really a problem of policy, and I won't purport to know the answer or who should be solving the issue, but I don't think it's ever going to be solved by one person at a keyboard.

There are standards such as HL7/FHIR, but in my experience, most companies incorrectly or partially implement the spec, and then it might as well be something else.

this is one of the main reasons I drifted away from healthcare and towards engineering. It's hard to deal with limitations that seem purposefully aimed at preventing you from getting the job done, and done right. My time on earth is limited and so is my patience.

> You're not well-positioned to understand the core nature of problems, and which are most important, from the outside

Great point. What startups that are trying to infiltrate "old/traditional" industries like hospitals or healthcare are realizing. Need to shadow doctors to see their workflow, not just propose a solution based on your observation or reading articles about hospital shortcomings.

More than that, you need to figure out how administrators view your offering, i.e. How much money does it make them. Some things doctors like cause the hospital to lose money -- like actually spending time getting to know patients and understand their condition, or keeping them out of the hospital. Doctors aren't always the people running things

Ive also noticed that even though the industry has done things X way for a long time, sometimes those are done because "well We've always done it that way". It's cargo culting, and nobody has a good reason why its done.

Being an outside can sometimes mean that you call attention to that, and show better ways. And with the internet and publications, can be an effective way to do that.

Most of the times however, there are reasons and you should at least understand why. But for those cases where "why" isn't defined, poke at them.... You're likely to find something interesting.

Agreed. It is very easy to try to disrutpt a process without understanding the reasons behind the current process. A lot more difficult to understand the current process then propose changes that address that processes shortfall.

There is always a reason why things are done a particular way "It's always been done that way" is shorthand for "I don't know why it's done that way". Like history, you need to understand the reasons behind a process before you can effectively change it, otherwise you are doomed to make the same mistakes and very soon your shiny new process will start to look as cobbled together as the previous one.

You mean... make software for the people who use it, not the people who make it?

Following doctors will only provide you with better tools for doctors. You need to shadow the admin side and understand the painpoints they experience because ultimately they pay bill.

The way the system works in whatever country you live in has shortcomings. Finding ways to fix those issues higher up the chain will provide the most value.

I like this. The only worry is that sometimes it can take a long time to get to the center and in the process of making that journey you may realise that you chose the wrong center to go towards.