With a 50% false positive rate, most of your positives are false positive. You might as well just tell everyone who asks for a test that they're infected.

Having 50% of all tests come back positive is way too high and makes the test basically useless. But having 50% or more of positive tests being false positives is probably the right tradeoff, though you probably want to re-test people who don't show symptoms.

From a public health perspective ... this could be a brilliant solution.

Until they all decide that they’re immune and don’t need to worry about it anymore.

Exactly. I think many people, upon learning they're positive by asymptomatic would (maybe) wait a week to make sure they don't feel ill, and then go out into the world, potentially spreading the virus with abandon because they're among the group that doesn't get ill from it.

You should account for ignorance in public health policy, but you shouldn't rely on it.

Having more false positives than true positives only works if the people taking the test either don't understand statistics or don't know the test accuracy. I for one would probably ignore a positive result if I thought my chances of true positive were 0.001 and my chances of a false positive were 0.5.

That's a great way to lose any remaining confidence the population might have on the establishment. It is also probably unethical so I doubt any doctor would go with it.

It's a perfectly valid concern, but I don't think it needs to turn out the way your fear.

How about we try this: when reporting "facts", how about we try noting that there is uncertainty involved? I know the average Joe and Jane public aren't intellectuals, and it is often how stupid people can be, but on the other hand I think it is also fair to say that we underestimate people.

I think we should at least consider trying this (noting uncertainty in reporting) - it's times like this where we need more ideas and more thinking, not less.

Until Fox News or the governor of West Virginia blows the illusion away.

Huh, I'm lost in this thread.

So, We lie to people and tell em they are infected. And then we are angry that FOX or the governor of WV are going to expose our scam?

What did I miss...

You could tell them the test came back positive. That's not really a lie.

50% is a bit of information. You can repeat the test - even if you consider that some contamination survives repeated tests, if you even get half a bit from subsequent tests you can easily get to eliminate quite a bit of population.

Yea but not nearly enough of the population for this to meaningfully change the response, and now you’ve made the testing results completely meaningless so that we can’t track the disease. Maybe it’s not intuitive to people but we are actually worse off with a test that’s this bad than just the extremely limited testing that’s being done now

I admit it's not intuitive. Tell me more?

I'm assuming there's high false positive rate. If the test has even moderate false negative, then yes it's worse than useless.

High false positive rate (I’m talking like more than 1%) means

(1) you are overwhelmingly unlikely to actually have the disease even if your test comes back positive but you don’t have obvious symptoms

(2) the pool of people testing positive is completely dominated by false positives. The fraction of people testing positive that will actually have the disease would be very small, so you lose the epidemiological benefit of the test (now you can’t actually track the disease well).

So — test is useless for people like you and me who don’t have symptoms or a strong prior probability of having covid BEFORE the test, because even a positive result is overwhelmingly more likely to be wrong, AND now the epidemiologists trying to actually track the disease are instead overwhelmed with false positives so the true signal disappears. It’s bad all around. You could imagine using this as a screening tool, but the false positive rate has to be small enough for that purpose as well (like, less than one percent).

There is a big system-level difference: with 50% false positive, half of the population goes back to living regular lives. False positives stay home, and go to a hospital only if they need a ventialtor (which very few will need since they are not actually sick)

There are 50% false positive, and there are some false negative. And some more not exactly false negative, but simply got sick after the test. And you tell them: we tested everyone, all negatives can just resume normal life. What could go wrong?

If you can't breath you go to the hospital regardless of it being COVID-19.

This is an interesting tradeoff to consider. Under many many circumstances I could see that rate of false positives being awful and intolerable.

In the context of a bona fide pandemic with a massive shortage of desperately needed testing, however, this would be a palatable improvement upon the status quo.

In the case of a generally mild disease, it's much less of a problem. Usually, the problem with false positives is unnessary, risky treatments and overwhelming the medical system. But in this case, you just need to know whether to self quarantine. You only need treatment if symptoms become severe.

Until the global economy unnecessarily collapses.

You also need to consider that when the false positives get over their flu or cold or whatever, they will assume they are immune to COVID.

Really? That test is meaningless. It screws up the numbers, and you might as well do a complete quarantine anyway, since you need to not just quarantine but isolate anyone who’s positive.

Let’s say there’s 100,000 people in the US with covid right now (like 10x the measured number)

FPR =0.5 TPR = 1

Disease rate = 1e5/3.3e8 = 3e-4

P(you have covid| test is positive) = 3e-4/(0.5(1-3e-4) + 3e-4) = 0.06%!!

This is less than useless. Now instead of tracking the disease you are completely overwhelmed with false positives and the result is exactly the same — everyone stay home.

hmm, how many false negatives is it giving?

A 50% false positive rate means that, since most people don't have the virus, about 50% of all the tests will come back positive. You just about might as well flip a coin instead.

This is a classic statistical fallacy. A test that is 50% accurate serves 0 purpose. You might as well consider coin flipping a valid test. If you tested America with this test you’d end up thinking half the country is infected. And within the ones that tested positive only half would actually be and within the negatives half would be and you would have missed them.

A test needs to be materially more accurate than the odds of having the disease to be worth anything.

Classic thought experiment: if 0.0001% of the population has a disease and a test is 99% effective and you test positive, what are the odds you have the disease? Answer: 1%.

Complimentary thought experiment: if an expensive preventive drug was available in limited supply (for 0.1% of the population only) and the earlier you took it the more preventive it was should you give it blindly to as many positives as you can? Probably not because 99% of that would be going to waste. And you would run out of drugs to cover the actual sick. Only 10% of the sick would end up actually getting the drugs.

Isn't the false positive rate the percentage of positives that should have been negative? E.g. if I test 10000 people and 100 of them test positive, but really only 50 of those are sick (and almost none of the other 9900 people are), then I have a false positive rate of 50% and a very useful test. Obviously, this is very different from flipping a coin. And flipping a coin could also have a very high false negative rate.

It's the rate of actual negatives that are perceived as positives.

I make that mistake myself all the time, What helps me avoid it is this: you want a number that is a property of the test method alone, independent of case distributions. A ratio between misidentified positives and identified positives (or true positives) would depend on sample distribution.

The metric you're talking about is called the "false discovery rate": https://en.wikipedia.org/wiki/False_discovery_rate

I also have a hard time keeping these all straight: https://en.wikipedia.org/wiki/Sensitivity_and_specificity#Ap...

50% accurate I'd agree is useless.

But the parent said 50% false positive, presumably with close to a 0% false negative would be VERY useful and save potentially millions of lives. We need enough tests yesterday or so to avoid a repeat of Italy and a 50% false positive rate (with a very low false negative rate) could help do that.

Even the properly-conducted version of this test has a fairly substantial false negative rate, somewhere around the 30% mark, and if it's done by students using samples that might not be taken correctly on a rigged-together testing setup that's going to get worse. Seriously, you might as well flip a coin.

A test with a 50% false positive rate, administered to 100 million people who aren't infected, would say 50 million people were infected.

False positive rate is a confusing term; it means the % of tests that should be negative that report a false positive.

I think the term for what you're probably thinking of (% of positive results that are false positives) is the false discovery rate.

That’s fair, if the assumption is that false negatives are very low or 0 (in which case it’s a super high accuracy test at low contamination rates and super useful).

And yeah obviously we desperately need a decent test like three months ago.

A test might have a high false positive rate, ie “low specificity”, but can still have “high sensitivity”.

That would mean that while you catch most cases of the virus, you’d also get a bunch of false positives. Flipping a coin as a hypothetical test would give false positives and negatives, or low specificity and low sensitivity.

Isn’t false negative specificity and false positive sensitivity?

https://en.wikipedia.org/wiki/Sensitivity_and_specificity

No, sensitivity is TPs / all positives (detected TP and FN). It’s the green half of the diagram in your link. The language is very easy to get tripped up by though. :)

I think I’m getting tripped by false positive, true positive.

A test with high sensitivity will have a low false positive rate.

A test with high specificity will have a low false negative.

So having a test with high sensitivity but low specificity will result in trust in the positives, but not trust in the negatives?

For me it’s easier to think of sensitivity as being “sensitive to the true positive” without saying much about false positive or negative.

A sensitive test will catch many positive cases. It may or may not have false positives though, eg a test that’s always gives the right answer vs a test that always returns positive no matter what.

A specific test will give you few positive results when the true answer is negative. You could use it to rule something out. One test might say “patient has A or B condition”, and a second test with high specificity may then rule out A or B, leaving B or A, respectively, as the probable condition.

I think you need to learn the difference between false positives and false negatives. A 50% false positive rate with no false negatives is absolutely better than 50% false negative and positive.

False negatives miss infected people and you fail to quarantine. False positives just mean you quarantine too many.

You’re not differentiating between sensitivity and specificity - in this case the test is highly sensitive but not specific. That means that it could be effectively hard to rule out cases while failing to provide strong evidence of actual infection in individual cases.

You're applying conventional knowledge to an unusual situation where it doesn't apply. The current test in my country is something like "was sitting within 4 rows of a known patient on a plane". That's obviously got a massive false positive rate but we do it anyway. The action you take when officially tested positive is the same as you take when learning you had contact with a patient - isolate yourself at home. There's no wasted drugs because there's no treatment for it! At worst, you might do an official test, but if those are in short supply, they're not going to be given to anyone who just walks in with a home-made result.

>A test needs to be materially more accurate than the odds of having the disease to be worth anything.

Shouldn't it just need to be better than 50/50?

>Classic thought experiment: if 0.0001% of the population has a disease and a test is 99% effective and you test positive, what are the odds you have the disease? Answer: 1%.

My odds still changed dramatically and the cost of a false positive is just me hanging out at home and not visiting my parents. I'm not getting a biopsy or taking expensive medicene.