There is also a rule in (system) operations: don't page unless immediate action (from humans) is required. This doesn't stop us from collecting data. If there is an anomaly, then we can look back at the data if necessary.
Alternatively, there ARE things you can alert on. My brother has type 1 diabetes. He got a free urinalysis in the form of ants crawling in the toilet. I expect – but since I don't have time series data from this toilet, I can't prove – that sensors would detect and increasing amount of glucose in the urine over time.
If anonymized, this sort of data would be useful for research as well.
I agree that giving any joe schmuck access to the raw data would result in unnecessary procedures.
I don't know the actual legality. But here's what I do know from attempting to implement multiple projects within the US for clinical care and for clinical research: most providers, hospitals, and pharmaceutical companies we have worked with have balked at the ability to have high resolution data fed into their medical record or research data systems. In most cases, they are worried that if they had the data, they would have to act on it were something to be concerning. And they do not want that responsibility.
Example: we were in talks to feed in near-realtime blood glucose, activity, and blood pressure measurements from patients on a clinical trial. Surely that would be useful for better sussing out the effects of a research drug than the current process (patient comes in every few months to get measured or have their log data entered or meter data downloaded)? Nope, multiple top 5 pharma companies didn't want the responsibility and liability of seeing their data that often, because what if a patient had super low or super high glucose and they later died? You had the ability to act on it but didn't.
They'd rather just not have it at all. It sounds ridiculous. And again, I don't know the actual legal/liability issues, but based on my experience with malpractice cases, I could see a viable case to be made. And there are certainly institutions out there who believe the benefit is worth the cost of that responsibility for certain use cases. But my experiences have definitely made me reconsider some of these things.
All that said, I am definitely in favor of collecting more health data (somehow), because until we have more data we won't know its utility. So far, in general it has been shown that more testing and more information often doesn't lead to better outcomes, but the cost of data is getting lower -- aside from aforementioned liability cost/fear.
I imagine getting something like this implemented at a state level in one or two key states could enable collection of large scale medical data for analysis on behalf of the public.
Edit: Question for you, given your experience do you think it might be possible to convince big pharmaceuticals to collect high fidelity data if said data was collected into wearable devices and only processed after the fact? As analogy to use a "black box" recorder approach rather than flight control systems. Though not sure how far to take that "black box" analogy.
That’s why we use them.
Isn't that a description of literally what every practicing doctors, that isn't involved in research, actually does?
And clinical trials are extremely expensive. Surely there's some utility in gathering data between the extremes of doctor-patient and clinical trial.
We use properly conducted clinical science to give us a framework for interpreting the data we see, comparing data we have to studies with appropriate external validity to allow that comparison.
And no, there isn’t, if an RCT is at all feasible. There was a great paper a few years back that reviewed various case-control studies for conditions that were ultimately explored using RCTs (eg, knee arthroscopy, internal mammary artery ligation). What they found was that the retrospective study consensus was basically a coin-flip with respect to the RCT consensus. They just don’t provide useful data - too many confounders, known and unknown, go unaddressed.
I am extremely skeptical that this is in fact an even approximate description of how medicine actually works. Maybe not what you're doing! But what almost all of the doctors I've ever seen? They're not keeping up with all of clinical science. And even if they were, lots of clinical science is not being properly conducted. Just listen to or read the gripes that the statisticians have!
My previous claim still stands; you wrote:
> I, as a lone doc, am not equipped to make heads or tails of un-randomized un-controlled piles of data.
An individual patient, provides "un-randomized un-controlled piles of data" to their doctor – or, rather, not even piles of data. There's the visual evidence available from looking at and watching a patient when they're visiting your office (or whatever). There are sounds, smells, touch too. Then there's the verbal evidence. Then there's the results of whatever tests it is you order. If that's not un-randomized and un-controlled data, I don't know what is.
There's no framework for interpreting all possible medical evidence, even for the smallest professional specialties. There are no RCT for medical expert systems; none that anyone could legally use instead of a licensed doctor.
And there are lots and lots of interventions that doctors perform that have never been adequately studied in the form of RCTs. Doctors, in the real world, exercise an incredible degree of latitude with regard to what interventions they can perform based on their own personal judgement.
(1) if I see something, I have to act on it, despite not actually having the knowledge of whether it means anything in this information-rich context. I just love things that -look- like information but that don’t actually tell me anything.
(2) we have a LOT of data suggesting that we do more harm than good when we try to intervene or investigate everything. For Pete’s sake, hospice patients outlive those who are actually pursuing life extension! Turns out every treatment has harms, and when you’re applying those harms to no benefit (because the patient is asymptomatic, or the anomalous event was never destined to progress to clinical disease), you just accrue harm that -looks- like care.
3) It is not likely to result in useful diagnosis, because catching shit pre-clinically offers little indication of what will or won’t become clinical. We see this in helical CT: a lot of people became “cancer patients” when helical CT came out, before we realized a LOT of growths recede forever without intervention - we just hadn’t seen them before, on the less sensitive tech.
If I see the data I have to act on it to cover my (American) ass, and I know I will do net harm to my patients. Thanks but no thanks. Do your due diligence: collect the data, analyze it, stick it in a properly randomized prospective study to see if you just trawled your way into an overfitted model, and prove that the intervention for it does more good than harm. THEN I’ll be delighted to use that specific data to pursue that specific intervention. Don’t just measure everything, throw it at my wall like hot spaghetti, and expect me to turn it into dinner.
-my blunt anonymous opinion, since it was asked for
So as someone that would be liable to act on this data, I understand and sympathize that receiving this data would be an unfair burden on you. But what about the rest of us? We're being treated by people that we are expected to trust that actively refuse to learn whether they're actually helping. That seems pretty unfair to the rest of us.
And no, we aren’t actively refusing to learn if we are helping. We want the data collected in a meaningfully structured way that actually answers the question of whether it’s helping. Garbage In/Garbage Out doesn’t answer that question.
You say it’s unfair, but go reread my post. My concern is that my arm gets twisted into doing things that are -bad for you-, not bad for -me-. Whether you like the reasoning or it’s conclusion, nothing in it is about being unfair to you. It’s about trying to do my absolute best to live up to my ethical responsibility to optimize your health outcomes. The only part of it that’s defensive is the part where I don’t ignore test results, because it will get me sued out of medical practice in the rare occasion they’re true positives. And I would like for that not to be the case, but there we are.
I don't disagree. I was claiming that both patients and doctors are caught in this trap, i.e. doctors are legally liable for not offering, or recommending, treatment that isn't warranted.
A false positive from a medical test is much more costly, both in money for a doctor (or doctor's staff member's) time, and anxiety for the patient.
That said, your brother's case makes some sense. It's targeted, and I imagine the action you want to take after getting an alert would be pretty well defined.
I'm glad people are saying this. I'm in a constant fight at work with people who want "more data." What ends up happening most of the time is it's ignored or people do not interpret it correctly because they don't have the ability or context to do it (doing stats wrong for instance).
"Using experts in a variety of fields as test subjects, experimental psychologists have examined the relationship between the amount of information available to the experts, the accuracy of judgments they make based on this information, and the experts’ confidence in the accuracy of these judgments."
"Key findings from this research are: • Once an experienced analyst has the minimum information necessary to make an informed judgment, obtaining additional information generally does not improve the accuracy of his or her estimates. Additional information does, however, lead the analyst to become more confident in the judgment, to the point of overconfidence. • Experienced analysts have an imperfect understanding of what information they actually use in making judgments. They are unaware of the extent to which their judgments are determined by a few dominant factors, rather than by the systematic integration of all available information. Analysts actually use much less of the available information than they think they do."
Which implies they do not know which part of the data is most important. No?
disclaimer: didn't click link/read chapter
AI is not going to "think" like humans. It is not going to be better or worse, just "different". And all experience so far show that the more data the better are the decisions made by AI.
And the experience doesn't show more data is better, it shows that an excess of features leads to overfitting.
People are proposing smart whatever now, and handeaving its value proposition as -eventually- feeding into ML which will -eventually- have benefits to patients.
I don’t think there is disagreement that data + ML is useful, but the value for increasing data because of AI assistance isn’t meaningful in the absence of that AI. Hospitals and insurers have huge data sets available: build a halfway decent AI first, and then argue for the value-add of the data.
Except any department in any hospital or insurer that welcomes you to do research can make that de-identified data available to you with a simple contract.
How do you think college freshmen get in the door to work on research with physicians?
No, it’s not public, but it’s trivially accessible.
But I did. No, I use the word “trivial” knowingly.
The data for insurers is in the form of Billings (which come with Diagnosis codes!), which follows a federally-standardized format (check out the Medicare website) and you can get a sample of 5% of Medicare subscribers’ data off the Medicare site without any hoops to jump through.
I want to say, “if you actually know what to do with it, PM me and we can talk details,” but the fact that you don’t already know about this sort of low-hanging fruit suggests you’re from outside the hc sector looking in.
Reach out, we can talk about what you’re thinking.
Put another way, at the extreme, you can have more and more data but unless you have the right data and are able to interpret it correctly, more data will only blind and confuse.
There is definitely an issue of under-diagnosis for many diseases. More data may not help initially but once we are over the critical mass in the amount of data and sophistication of ML algorithms trained on it, it will get over the hump.
The difference is that you actually hear about the Diagnosis X That Was Missed, but you never hear about That Simple Blood Test That Should Never Have Been Drawn but that Showed Positive for a Marker Common In Disease But Also Found in the General Population so They Needed a Biopsy to Rule Out, and Spent Hundreds or Thousands if Dollars and Lost Several Days of Work For Nothing At All. The latter is much, much, more common.
The “miss” is usually due to missing something in the modern deluge of data coupled to inadequate appointment time/resources, not lack of data availability.
For a real life example: heard about a patient die the other day. Reviewing his records with a student afterwards I saw his potassium had been climbing in his last two panels (though it hadn’t yet gotten high enough to be a red flag), and he didn’t get the blood gas that should have been done as per protocol. Thing is, the data was there, the rules were there, but someone with enough time to meaningfully review it was not. Would more piles of detritus data to sift through made that more likely, or less?
I also remain an ML skeptic for the time being. ML analysis of patient data is a big sophisticated version of retrospective analysis, which for purely statistical reasons is highly, highly unreliable.
A couple months ago I spent a good number of hours over a couple week at local hospital waiting for a close friend's mum to recover from a stroke. I could be mistaken but the only "data analysis" I noticed was taking place in the heads of the doctors and nurses.
The same goes for the physical therapy facility. That is, no one - not even the insurance company - was using similar cases as a guide. Yes, at a high general level they were. But using real data didn't seem to be on their radar.
This void made family decision making stressful and difficult.
Patients are not part of the decision making process, and therefore not privy to the analysis, unless there is a decision to be made that requires their input (eg, two reasonable roads forward that depend on their preferences, or on their risk-tolerance.)
The process is made opaque to patients intentionally.
Nothing like that. No continuity between docs or floors/wards. Just everyone kinda making it up as they go.
It just struck me as another reason why outcomes aren't optimized, and costs, relatively, inflating.
I'm not suggesting doctors are replaceable. But the doppelganger approach to (big) data seems to be ideal.
I understand. That’s part of what is intentionally opaque. We don’t speak like that to anyone but other medical professionals. There’s a lot of Dunning-Kruger in what people think of their ability to parse hc information. As someone who grew up with serious chronic disease, I’ve been on both sides of this fence. It’s not something you really “get” until you’ve had to care for patients.
> costs inflating
Several Health Econ studies have shown that about 70% of hc costs are driven by new technologies (implants, procedures, patented meds.) It has nothing to do with how docs communicate with one another.
And how we communicate is also opaque to patients. Not intentionally, it’s just not a conversation for patients, any more than any two technicians in a software company talking/arguing technical details are having a convo meant for the ears of customers.
The way it should have worked: based on previous labeled examples, computer system should have noticed the uptrend in potassium and flag this client for blood gas, and then make classification based on available and _additional_ data.
To rephrase: algos intelligent enough to be useful in the manner you imagine are a long way off from existing, but data deluge that harms patient care is -already here-. Don’t worsen the deluge until you improve the ability to digest and present it.
That EMR “decision aids” aren’t anywhere near sophisticated enough to tell the difference in the myriad of situations we have is why docs fight so hard against those automated “flags.” We get piles of them, all the time. QI people call it alarm fatigue, they know it’s a thing, and yet... they do it anyway, because it sounds (like in your post) like the obvious thing to flag.
It’s not that AI is inherently incapable of getting there. It’s that when it does is a long way off. Medicine looks a lot simpler from the outside.
Unfortunately the rule doesn't apply in this case. Your "smart" toilet is not trying to diagnose problems. It is trying to infer which ads you will click on the most, to make money for a surveillance company. A toilet that has a handful of true positives, a huge number of false ones, and doesn't kill too many people is pure win: surveillance and pharma both make money, and it's very hard for a consumer to tell if he never had X disease, or if buying the medicine prevented his X disease.
You can get more complex offerings that do perform some more complicated tests if you really want to.
The 'bath room' itself is also typically a single, self contained unit, often in some kind of mold, and including the shower, accoutrements, and bath in a full space designed to get wet and save/share the clean water in the bath.
The 'bath room' usually has a waterproof, electronic control panel which allows you to set timers for filling the bath, set an ambient temperature to keep it at, set the preferred level of water, set different default temperatures for the shower and bath, and typically also access all these features from other places like the kitchen where a companion panel is installed. This is typical of your average house - I just described my wife's parents' place, which was built 10 years ago.
The whole design thing is completely solved there, and why that approach has not been adopted across the U.S. and Europe by now is completely beyond me.
But the killer thing there would be to bring the Japanese approach to the US and Europe and plug in some better smart access features. Or go to Japan and plug them in there, where the infrastructure and demand is already in place [well, Google is probably on it already but...]
There's an episode of Star Trek: Voyager about this[1]. Seven of Nine absorbs months worth of data (way more than a human could actually process) from the ship's computer, and "discovers" that both half of the ship plans on mutinying and that the Captain wasn't actually trying to get them home. Both of which were false.
1. http://memory-alpha.wikia.com/wiki/The_Voyager_Conspiracy_(e...
* Diet balancig? (Hey, eat more cucumbers)
* Calories? (IANAD)
* Also more serendipiteous opportunities we can't foresee now
Or:
(at the doctor's)
t = 0 - Doc - I would need a urine analysis
t = 60s - User - Here you are.
Cons:
* Huge privacy concerns (Hey, you are fired! btw, you also have cancer)
As someone in the industry I do of course have a wishlist, but, you know.
As to clean: a self cleaning toilet and a roomba living in the corner is a huge step away from treating your toilet as an always-on medical lab.
https://www.theonion.com/new-e-toilet-to-revolutionize-onlin...
I think Reliable, cheap, user friendly medical diagnostics would be a game changer, but of course the data analytics to make good use would be different than how current, one-off, medical testing is treated.
> raise flags that must be chased down
If a red flag must be chased down, doesn't that mean it changes your decision?
Every time I hear about all-knowing toilets, I continue to feel it’s putting the cart way ahead of the horse.
There’s a rule in medicine: don’t order a test whose outcome doesn’t change your decision. That’s not about cost-savings, it’s about avoiding false positives that make no sense in clinical context but raise flags that must be chased down.
This sort of thing is literally promising to just continually run un-merited urinalysis left and right.
“More data” is not inherently good. Without a framework of information for interpretation of that data, it will do far more harm than good.
Honestly, the reason so little tech disruption keeps happening in the provision of medical services: no one trying to “disrupt” bothers asking what is or is not useful. There are just some ideas completely out of touch with what people need or want or have friction over, and they just refuse to die because they’re the most obvious ideas to have if you’re unfamiliar with the territory.