News for hackers. Stuff that matters.
Re:Stanford Student Claims to Run Bootleg Covid-19 Testing Lab by bobowzki on Friday March 20, 2020 @01:29AM (#22635197)
If it's so easy to build a website. Why doesn't everyone scale to the size of Google?
Re:Stanford Student Claims to Run Bootleg Covid-19 Testing Lab by mayneack on Friday March 20, 2020 @01:37AM (#22635237)
My guess: non trivial to do it safely and non trivial to get accuracy rates you'd get from a professional lab.

Doing all those things by hand also takes some time. Thinking back to my days in a lab at MIT, sample to PCR probably took a full day (rough approximation). Big labs can probably increase the parallelism a lot.

Re:Stanford Student Claims to Run Bootleg Covid-19 Testing Lab by tjoff on Friday March 20, 2020 @02:10AM (#22635385)
Comparison is flawed.

Getting demand is harder than scaling.

In this case the demand already exist and the money to scale as well.

In short, the question remains. If there is a will to scale, what is the holdup? People with experience?

Re:Stanford Student Claims to Run Bootleg Covid-19 Testing Lab by freepor on Friday March 20, 2020 @03:04AM (#22635619)
I mean if you transferred $500 billion to my bank account and said “clone Google” I would not get very far.
Re:Stanford Student Claims to Run Bootleg Covid-19 Testing Lab by grumple on Friday March 20, 2020 @06:46AM (#22636523)
You might not, but many of us are capable to developing the systems and features that are core to google's business, and for any parts we don't know, we know where to find the people we need.
Re:Stanford Student Claims to Run Bootleg Covid-19 Testing Lab by freepor on Friday March 20, 2020 @06:46PM (#22642496)
But why do you assume that someone like you rather than someone like me is in charge here? Statistically there are orders of magnitude more mes than yous. And empirically, it seems as though the people in charge right now are even more catastrophically incompetent than anyone thought.
Re:Stanford Student Claims to Run Bootleg Covid-19 Testing Lab by jimmaswell on Friday March 20, 2020 @03:16AM (#22635677)
Hire the right people with all that money and you could.
Re:Stanford Student Claims to Run Bootleg Covid-19 Testing Lab by freepor on Friday March 20, 2020 @03:18AM (#22635688)
I wouldn’t be able to evaluate people well enough to hire the right ones.
Re:Stanford Student Claims to Run Bootleg Covid-19 Testing Lab by harry8 on Friday March 20, 2020 @06:40AM (#22636510)
That right there is the problem we face as a nation, Mr President.
Re:Stanford Student Claims to Run Bootleg Covid-19 Testing Lab by astatine on Friday March 20, 2020 @09:15AM (#22637266)
That doesn't take away from the point that there probably exists a non-zero number of people who could convert $500B into something of the scale of Google.
Re:Stanford Student Claims to Run Bootleg Covid-19 Testing Lab by mcovalt on Friday March 20, 2020 @01:39AM (#22635249)
I’d be curious to the answer to that too. I can think of quite a few reasons, but I’m sure there are nuances that are completely unknown to me.

I’m more interested in the parent question at the moment. I know almost nothing about the subject matter. What are the blockers for deploying a simple virus test at the scale we need?

Re:Stanford Student Claims to Run Bootleg Covid-19 Testing Lab by nkrumm on Friday March 20, 2020 @02:52AM (#22635566)
It boils down to people, logistics, and efficiency:

Yes, we can produce the primers, develop the tests, ramp up the supply of RNA extraction kits, swabs, tips, etc. Individually easy, but together more challenging, especially when the entire world is also trying to acquire the same products.

Yes, large, fast and high-scale instruments exist. But, they need to be set up and calibrated. They need adequate power, cooling, water, space, etc. The technicians need training. They need network interfaces, etc.

Yes, it's a simple test. But there are still SO MANY steps needed to generate a result from a sample, and those take people. Just looking at the laboratory workflow: 1) Sample received in shipment -- unbox, scan manifest (if provided) 2) Unbag sample + requisition 3) Check sample label (usually handwritten) and verify accuracy to requisition (possibly handwritten). In an ideal case, orders are coming in as "interfaced orders" (meaning electronically) but many smaller clinics/hospitals will not have this. 4) Enter all demographics of the patient into laboratory system 5) Have second person verify the information 6) Re-bag and send sample to location in lab with test instrumentation 7) Technicians unbag, verify label + order accuracy 8) Decant sample from non-standardized tube sent to lab to standard tube used in lab 9) Scan into liquid handling robot... ...et cetera, et cetera

As I will probably find out in the replies there are "solutions" to all of these problems, and many labs utilize them. However, to scale such a complex system so quickly means that there is no room for a "test environment", and all changes have to be made "in production". Even simple changes to the workflow require days of prep to communicate to all of the staff across three daily shifts-- you get the idea.

source: i'm a laboratory medicine resident at the UW lab. We scaled from 0 to 3000 tests in 2 weeks and it's been an incredible all-hands effort.

Re:Stanford Student Claims to Run Bootleg Covid-19 Testing Lab by Reelin on Friday March 20, 2020 @03:51AM (#22635823)
I appreciate what's going on at UW regarding this and realize that it must have been a huge amount of work to say the least. However, I'd like to point out that most of what you've described is related to the bureaucracy and paperwork of certified medical procedures. Under ordinary circumstances that's all well and good, but given the severity of the situation it seems like it would have been better to eliminate much of that at least a month ago. Put another way, combat medics don't let someone die just because they're short some official FDA approved widget.

Why keep such detailed records? Barcode it on the way in, run the test, and report the result - forget demographic data and any other paperwork, whoever ordered the test can deal with that.

The supplies for RNA extraction are common and readily available (I think all the raw components are mass produced?), but only specific kits are FDA approved.

Only specific swabs are FDA approved. (https://khn.org/news/as-coronavirus-testing-gears-up-special...)

> the government is considering expanding its recommended testing material options to allow for more general nasal swabs to keep up with the increased testing demand

Standard qPCR machines are incredibly common in both academia and industry and have 96 wells at minimum. The protocol published by the CDC has an 80 minute runtime. That's 1000 tests per day for a _single_ low-end machine.

The US response has been an absolute shitshow of bureaucratic dysfunction as far as I'm concerned, and people will likely end up dead as a direct result.

Re:Stanford Student Claims to Run Bootleg Covid-19 Testing Lab by omar_a1 on Friday March 20, 2020 @04:04AM (#22635881)
> Why keep such detailed records? Barcode it on the way in, run the test, and report the result - forget demographic data and any other paperwork, whoever ordered the test can deal with that.

Detailed records and redundancies ensure you don't mix up any of the tens of thousands of samples coming in. These systems are crucial for being able to scale up as they have without complete chaos breaking out in the lab.

You are right that barcodes are far more efficient, after implementation. But implementation takes months, and requires diverting resources that would otherwise be used to run more samples. The lab would face significant downtime putting a barcode setup in place and retraining. (See nkrumm's comment about labs not having a "test" environment.)

> Standard qPCR machines are incredibly common in both academia and industry and have 96 wells at minimum. The protocol published by the CDC has an 80 minute runtime. That's 1000 tests per day for a _single_ low-end machine.

That's only if we ignore the sample preparation time (orders of magnitude longer than analysis time), and the many control wells necessary to ensure the technique was successful. While I suspect these will also be dismissed as bureaucratic inefficiency, they're needed to make sure your newly-installed instrument running 24/7 hasn't broken down, and that the high throughput sample prep wasn't botched for a given prep batch.

The stakes for these tests are high enough that cutting corners on QA/QC isn't acceptable, because it means more lives lost. Doing a mediocre job for testing at this scale will have a very real impact on our population.

Re:Stanford Student Claims to Run Bootleg Covid-19 Testing Lab by Reelin on Friday March 20, 2020 @05:24AM (#22636206)
My calculation (1000 tests per day) left space for 12 controls per batch - is that not sufficient under the circumstances? Alternatively, at 500 tests per day (again, this is _per_ low-end machine!) you could run each sample twice in addition to the 12 controls. (And I would _never_ dismiss controls as an inefficiency, even the most trivial exploratory experiment is highly suspect without one.)

The point I was trying to make with those numbers was just how readily accessible the necessary instrumentation is.

Regarding sample preparation, I'm well aware that it's a time sink (I've done DNA and RNA extractions before). But it's fairly trivial and can be done fully in parallel by multiple people; the primary bottleneck is likely to be available bench space. (Unless the sample prep has been fully mechanized, in which case I'm really not seeing the issue.)

My point being that multiple, parallel sample prep pipelines can feed a single qPCR machine to keep it running just about 24/7.

Regarding barcodes, I didn't mean it had to be fully automated. Just drop the excess data entry and switch to a serial number scheme with built-in redundancy (or at least error detection). Many academic labs employ such schemes by hand.

Given just how dire the circumstances are, I'm afraid I'd have to strenuously disagree that cutting corners on QA/QC (as compared to standard medical diagnostic testing) would be unacceptable. None of what I described is at all out of place for handling research samples, and in my experience those are quite reliable.

I used the combat medic analogy in my previous post for a reason - a significant number of people are likely to end up dead due to our having blindly stuck to the rules. A bit of pragmatism could have saved them, and I view that as a tragic systemic failure on the part of the US government.

Re:Stanford Student Claims to Run Bootleg Covid-19 Testing Lab by omar_a1 on Friday March 20, 2020 @02:30PM (#22640119)
You're saying a lot of things here that don't make any sense.

> I'm afraid I'd have to strenuously disagree that cutting corners on QA/QC [...] would be unacceptable.

> I would _never_ dismiss controls as an inefficiency

So, the QC in QA/QC stands for Quality Controls. So, you're strenuously insisting they're unnecessary, yet vehement that you would never dismiss the thing you just said wasn't necessary? Which one is it?

> But [RNA extraction is] fairly trivial and can be done fully in parallel by multiple people

It takes longer than the instrument analysis was my point. That's the typical bottleneck in running samples, not your instrument sample throughput like you claim. So even if you can run 1000 samples on the instrument, if it takes longer to prepare them (which it always does), then that's your rate limiting step.

Unless, of course, you're working in this magic lab of yours where new, fully-trained techs suddenly materialize any time there's a spike in sample volume (whereas new benchspace doesn't? Might wanna take that up with your deity lab manager).

> Regarding barcodes, I didn't mean it had to be fully automated. Just drop the excess data entry and switch to a serial number scheme with built-in redundancy (or at least error detection)

With all hands on deck who do you think will have time to do this? Barcode migrations are a lot of work. Adding the word "just" to what I've described doesn't make something trivially easy.

Though apparently this whole process is trivial, from the sample check-in to the RNA extraction to the instrument analysis. Why not just fire the UW resident and all the staff, so that you can swoop in save the world, since it's all so terribly easy for you? I look forward to touring this fully-automated magic lab of yours.

Re:Stanford Student Claims to Run Bootleg Covid-19 Testing Lab by Reelin on Friday March 20, 2020 @09:57PM (#22643586)
I'm fairly certain that everything I said makes sense, so I'll try to clarify a bit.

First though, it seems a misunderstanding has developed. I was not calling the competence of the UW lab into question! They are bound by various state and federal regulations and I'm certain are doing the absolute best that they can under the circumstances. The examples I provided were to illustrate just how absurd the current state of affairs is, and to make it clear that bureaucratic ineptitude is to blame.

We obviously both agree that QA/QC is necessary; note that my back-of-the-envelope calculations included controls. I don't pretend to know precisely what requirements FDA regulations place on various diagnostic tests. What I do know, and attempted to illustrate in concrete terms, is that it is physically possible to scale testing in a sufficiently reliable manner using common instrumentation and bulk reagents. Any genuine attempt to explain US testing shortages _must_ account for this fact.

Yes, parallelizing sample prep requires a sufficient number of properly trained technicians. Presumably the US government is capable of locating and making use of qualified personnel in an emergency situation such as this? (To start with, literally any graduate student whose research includes running molecular biology protocols is likely over qualified for sample prep.)

Regarding serial numbers and labeling, it is indeed trivial. You can literally keep a list on paper and label tubes by hand (I did this for years in an academic lab). This step is fast compared to overall sample prep time.

I have no idea where you got the idea of a barcode migration from. Remember, I'm not describing a clinical lab operating according to FDA regulations; I'm describing a sufficiently reliable setup that closely resembles an academic research lab and would maximize throughput in an emergency situation. The only goal is to receive a package, label a corresponding tube, run a test, and report the result.

(BTW, I claimed that bench space for sample prep, not instrument throughput, was likely to be the bottleneck. In fact, instrument throughput not being the bottleneck was one of my primary points.)

Re:Stanford Student Claims to Run Bootleg Covid-19 Testing Lab by omar_a1 on Friday March 20, 2020 @11:01PM (#22643984)
If you're basing your sample throughput calculations on instrument throughput (1000/day), you're implicitly assuming that instrument run time is the limiting factor, and the rest of the operations can match it.

I'm also confused by your apparent assumption that bench space is a universal constant that cannot be changed, even under a global pandemic, while lab personnel can be scaled up effortlessly.

> Regarding serial numbers and labeling, it is indeed trivial. You can literally keep a list on paper and label tubes by hand (I did this for years in an academic lab). This step is fast compared to overall sample prep time.

Scientists don't keep records on scraps paper for a reason, and I'm shocked your PI didn't read you the Riot Act for shoddy record-keeping.

With thousands of samples coming in, this is how you end up completely losing track of your samples. Once again: it's not trivial, particularly at this scale.

> I have no idea where you got the idea of a barcode migration from.

> Why keep such detailed records? Barcode it on the way in, run the test, and report the result - forget demographic data and any other paperwork

Re:Stanford Student Claims to Run Bootleg Covid-19 Testing Lab by Reelin on Saturday March 21, 2020 @04:56AM (#22645390)
I was attempting to illustrate the absurdity of the past month and a half by performing a back-of-the-envelope calculation, not provide a detailed business plan for launching a biotech startup. FFS

I never claimed that bench space was a universal constant, nor that hiring personnel is effortless. Rather, I attempted to make it clear that from both scientific and logistical perspectives the barrier to effective testing is quite low.

The point of the instrument throughput calculation was to make it clear that instrument availability can't possibly be the limiting factor because a single low-end one that many academic labs already have on hand can facilitate an incredible number of tests each day. That's important because you can't depend on being able to procure new instrumentation in a timely manner during a crisis so any estimates ought to assume that you're stuck with whatever you already have.

We see reagent shortages reported in the news, but that can't possibly be the limiting factor either because (as far as I know) only common chemicals (plus primers and probes) are required. (Actually you could probably dispense with the probes if you were willing to run an awful lot of gels, but that would further complicate the pipeline and require more labor so better not.)

So then we might suppose that personnel and training is the bottleneck, but that can't be it either because the government has deep pockets and just about any practicing molecular biologist is over qualified to perform such a trivial procedure.

Looked at this way, it should be abundantly clear that US policy is entirely to blame.

I also never said to use scraps of paper in lieu of proper record keeping. The obvious interpretation of what I wrote is to go line by line in a lab notebook, which is something I have in fact done before (obviously not on the scale described though). It works quite well in the long term provided that your serial numbers incorporate either a day or a page number and are strictly sequential. It's hardly the only way to go about it; I only mentioned it because the original post that I responded to explicitly called out record keeping as a necessarily time consuming and complicated part of the process. I was illustrating that once again, that is only true when complying with existing US regulations.

Re:Stanford Student Claims to Run Bootleg Covid-19 Testing Lab by omar_a1 on Saturday March 21, 2020 @02:42PM (#22648806)
What was described above wasn't "US Administrative Bureaucracy" so much as "this is how busy labs are run." It can be challenging to understand that if you're coming from a small research lab where QA/QC boils down to recalibrating when you feel like it, and sample records are maintained by transcribing loose paper into Excel.

The bureaucratic aspects are to ensure you don't mix up the large volume of samples, and the quantitative QA/QC metrics are needed to objectively esablish what level of error is considered acceptable. Good record-keeping and QA/QC are all necessary when you're dealing with healthcare, particularly largescale healthcare.

That doesn't mean US policy or the administration's handling of this crisis was anything close to acceptable, but the issues there are pretty far removed from routine laboratory sample check-in and record-keeping.

Re:Stanford Student Claims to Run Bootleg Covid-19 Testing Lab by wbl on Friday March 20, 2020 @11:20AM (#22638239)
Then let's be about it. South korea did this, why can't we?
Re:Stanford Student Claims to Run Bootleg Covid-19 Testing Lab by bhaumik on Friday March 20, 2020 @02:03AM (#22635357)
Supply chain issues, for one.

Eg. Not enough cotton swabs: https://www.politico.com/news/2020/03/16/coronavirus-testing...

Re:Stanford Student Claims to Run Bootleg Covid-19 Testing Lab by makomk on Friday March 20, 2020 @06:09AM (#22636387)
Yeah, by a bit of absolutely terrible luck, the preferred swabs for this are produced by a factory in Lombardy, Italy. There are other manufacturers of swabs that could be used but they don't seem to have the production capacity.
Re:Stanford Student Claims to Run Bootleg Covid-19 Testing Lab by hcknwscommenter on Friday March 20, 2020 @04:54AM (#22636088)
Seriously? There are approximately 1 Billion cotton swabs in this country (U.S.). This is not the issue. The issue is gross federal gov't incompetence.
Re:Stanford Student Claims to Run Bootleg Covid-19 Testing Lab by Reelin on Friday March 20, 2020 @05:44AM (#22636293)
It's somewhat more nuanced than that. The currently approved swabs have limited availability, and indeed there are others (unapproved) that would work. However it seems you can't use just any - they have to consist of synthetic fiber, can't have a wooden shaft, and can't contain calcium alginate. This rules out many of the swabs that a healthcare facility might have on hand.

(https://khn.org/news/as-coronavirus-testing-gears-up-special...)

Re:Stanford Student Claims to Run Bootleg Covid-19 Testing Lab by joe_the_user on Friday March 20, 2020 @01:34AM (#22635226)
Indeed given many lives lie in the balance, shouldn't there be something like a bootcamp get smart people up to speed on this quickly?
Re:Stanford Student Claims to Run Bootleg Covid-19 Testing Lab by omar_a1 on Friday March 20, 2020 @02:18AM (#22635410)
I doubt it's the lab techs that are in short supply. It's probably the consumables and reagents that go into the test that are out of stock. Lab procedures scale pretty well (to an extent - not the scale of the pandemic). Lab work is often feast or famine in terms of sample volume. During normal times, spikes usually just means the folks running the assays will be particularly busy.

With all that said, there's still a lot of specialized skills required to competently run these tests. You'd still need significant training if starting from nothing. But there are enough un/underemployed lab techs out there that finding more techs won't be the bottleneck.