We're trying to make our SARS-2 tests better than we should

Ok, that's a somewhat provocative title but I think it's basically accurate.  During this pandemic the US in particular has had a problem with erring towards the side of better, more expensive tests when what we've really needed has been cheaper tests.

An inauspicious start

The US got off to a really bad start testing for Coronavirus.  Scott Gottlieb was head of the FDA until a year ago and, in contrast to many other Trump appointees, was generally acknowledged as very competent.  Back in February he put together a Twitter thread outlining what was happening in terms of testing which I'll summarize here.  Normally hospitals with PCR machines, which is most medium sized or larger ones, can just go ahead and develop their own tests for viruses.  However the CDC thought it would be important to have widely available testing kits which would be of a higher quality than what hospitals could do themselves.  So they asked HHS to declare that Covid-19 was a public health emergency so they would be allowed to start working on their kits, which HHS did on January 31st.  But because a public health emergency was declared around Covid hospitals were no longer allowed to do their own testing.

That would have been fine if the CDC was able to produce a test quickly but of course we all know it wasn't.  Rather than trying to implement the simple one part test that the WHO recommended and which was essentially what hospitals would have done themselves in the absence of a declaration of emergency it developed a more complicated three part test.  That more complicated test took several weeks to develop.  And as we all know that more complicated test just didn't work.  Well, the first part which detected SARS-2 worked.  The second part which detected SARS-2 as well through a different method worked for most hospitals but not for others.  But it was the third part of the test that really caused problems.  That part was there to see if the patient might have another infection that might be confusing matters.  And the validation step for that third part had some faulty reagents that failed to validate the test at basically all the hospitals that got the CDC testing kits. 

If the testing was done at some third world country then the CDC could probably have just told hospitals to use the first part and ignore the rest.  But in the US we expect well tested medicine and the FDA hadn't approved that test procedure.  So all those test kits couldn't be used.  We did continue to have hospitals ship samples to the CDC to be tested by them but the bandwidth there was very limited.  This was the era when you could only get tested in the US if you had recently been to China.  Because there wasn't enough testing capacity at the time for anything like the number of tests that were needed some arbitrary criterion had to be used to winnow them down and I suppose that one was better than assigning testing resources by lottery.

By February 29th, a month after the emergency was declared, the FDA decided it would allow about 5,000 of the most sophisticated laboratories in the US to start doing their own testing and on March 16th they decided that basically any certified laboratory should be allowed to test for Covid.

Here's a more detailed timeline of all of this for the curious.

In retrospect I think it clearly wasn't true that

 "In the U.S., we have policies in place that strike the right balance during public health emergencies of ensuring critical independent review by the scientific and public health experts and timely test availability.  The CDC test is a high-quality test, and it’s important to remember that false negatives or positives can be detrimental to making sure we are treating patients early, without delay, and also not quarantining healthy individuals.”

as FDA Commissioner Stephen Hahn said.  We clearly error too far on the side of test quality.  If you only have a hundredth of the number of tests you need your tests effectively have a 99% false negative rate and nothing we could have ended up with would have been nearly that bad.

Current problems

We've solved those problems that plagued our early response but there are still issues.  As contact tracing efforts in the US have ramped up we've started to get backlogs delays in getting results back from tests have started to increase.  And that's a real problem.

When you first get infected with Covid-19 you, of course don't notice anything at first.  But after a few days (or possibly longer) the amount of virus in your respiratory system starts to ramp up very quickly.  Eventually they hit a peak and start slowly going down.  And generally only at this point, most of a day after peak infectiousness, do you start to develop symptoms.  That's what makes this disease so much harder to contain than SARS was.  

The ideal test would be cheap and plentiful enough that you could test everybody every day.  It would give you a result quickly after taking the test.  It wouldn't give false positives or false negatives.  And it would be sensitive enough to detect an infection no matter how early in its progression.  There doesn't seem to be any technology that can give us all of that but there are potentially better tradeoffs we could make.

PCR works by taking a sample that might have viral RNA and, if it is present, doubling the amount again and again in cycles until the level of RNA is detectable.  How many cycles it takes, the cycle threshold or CT, is a measure of how much viral DNA there was.  PCR machines can generally go for up to 40 cycles leading to them being able to detect very small levels of virus.  But they're big machines and the samples have to get to them and it often takes at least a day to get the result back.

On the other hand we've long used a different sort of test for things like the measles, flu, or pregnancy tests.  These have antigens for specific proteins and change color in contact with them. However, they do come with a downside.  Without replication they are less sensitive than a PCR test, maybe only going down to a CT of 35 or so.  This means that they will miss some infections that PCR test would catch.  But that probably isn't as important as it looks.

To have a reasonable chance of infecting someone else you need a certain amount of virus, probably a CT value of less than 30 or 25 for a reasonable chance - we aren't sure exactly where the cutoff would be yet.  So while the antigen will miss a good fraction of infections it won't miss any more infectious people than a PCR test would.  As virus levels are initially climbing there's a window where a PCR test can detect an infection but an antigen test can't.  However, people generally spend less time in this window than the difference in turnaround time between a PCR test and an antigen test.  So if someone is being continuously tested with both methods the PCR machine will get a sample it can detect from them first but they'll hear back from the antigen test before that.

And of course antigen tests can be much, much cheaper than PCR tests, possibly less than $10 in mass manufacture, and could be theoretically self administered every day whereas PCRs require trained personnel that can't really be scaled like that.

There are cases where PCR is better.  If someone has been sick with pneumonia for over a week and just taken a turn for the worse and ends up at an ER then by that point detecting the virus may require a PCR test.  In a strictly medical sense of treating a patient at a hospital they work better.  But in a public health sense of halting the spread of a disease within a community sense they're worse.

We've already seen a lot of negative media coverage about faster but less sensitive tests.  FDA probes accuracy issue with Abbott’s rapid coronavirus test.  Coronavirus Antibody Tests: Can You Trust the Results?  These pieces correctly point out that there are infections these test wouldn't detect but not the reasons we shouldn't care.  So there's political pressure not to approve these because the reasoning takes a little while to explain.

Also, the paradigm the FDA works in is mostly hospital treatment, public health interventions aren't central to what it does.  So I worry that if some test is great for public health purposes but poor for medical purposes and some confused intern could possibly use it instead of the right test in a medical setting then the FDA won't consider approving it.

But I really hope I'm wrong because this could be a very important intervention.  I just don't think history is on our side.