In this study, we report a standardized cross-evaluation of LFAs on the same pre-pandemic SARS-CoV-2-negative and PCR-confirmed SARS-CoV-2-positive samples, and rate their reproducibility, usability, and performance characteristics. Overall, the LFAs showed a higher propensity for false negative than false positive readings. Results are public: https://covidinnovation.partners.org/evaluation/. We use the Simoa technology(29) to measure the concentrations of anti-spike protein IgG and IgM antibodies and extrapolate the assay limit of detection. We also established a web tool to aid users in understanding the likelihood they have anti-SARS-CoV-2 antibodies given a positive test result. This resource of performance characteristics of several LFAs and a tool for result interpretation, can both be used for immunosurveillance and future home testing applications.
LFAs are tractable tools to estimate community seroprevalence, especially with anticipated seasonal fluctuations in the transmission dynamics of SARS-CoV-2 and other viruses that cause the common cold, which confound the symptomatologic diagnosis of COVID-19(32). As new waves of the COVID-19 pandemic resurge around the globe(1), and with commencing vaccinations against SARS-CoV-2 infections(12–14, 16), there is a renewed interest in serological tests to detect anti-SARS-CoV-2 antibodies(20, 33). Affordable LFAs offer an attractive option for monitoring the presence and longevity of anti-SARS-CoV-2 antibodies, and determining population-level herd immunity(34). LFAs also obviate the need for complex laboratories to process the samples(35). As the pandemic expands to previously unexposed communities, it is critical to use simple tools to monitor exposure dynamics and seroconversion in SARS-CoV-2-exposed individuals, as well as vaccine-induced immunity(16). We tested a mixture of LFAs targeting SARS-CoV2 nucleocapsid, spike proteins, or both. Moderna’s mRNA-1273 and Pfizer-BioNTech COVID-19 vaccines encode SARS-CoV2 spike proteins to induce anti-spike antibodies(14, 36). Therefore, LFAs targeting the spike and nucleocapsid proteins of SARS-CoV2 could be used to differentiate vaccine- and infection-induced antibody responses, respectively.
Rigorous evaluation of these LFAs by manufacturer-independent parties is important. The US FDA independently reviews medical products before commercialization. The FDA used Emergency Use Authorization (EUA) authority to accelerate the implementation of diagnostic products during the pandemic. Commercial manufacturers were required to submit a completed EUA request(22). Unfortunately, the rush to market introduced many tests that did not meet typical US or international standards(37). Therefore, the FDA and international regulatory agencies continue to update guidelines for authorization of new serological tests. Our evaluation plan mirrored the FDA guidelines for evaluating serological tests(22). We included 10 HIV-positive samples to test whether they have higher false positive results in SARS-CoV2-negative samples(38), and did not detect higher false positive results .
The mere detection of IgG or IgM responses does not guarantee that neutralizing antibodies are present at protective titers (7, 39). The study demographics suggest a slight over-representation of African Americans among cases, as reported(40). However, the sample size was underpowered to formally determine the effect of race on test performance. In our analysis, IgM detectability was less sensitive and reproducible than IgGs across multiple LFAs, possibly due to both lower IgM titres, and lower limits of detection for the IgM LFAs. Waning antibody responses have been reported in some SARS-CoV-2-infected individuals(41–44). Furthermore, reported cases of re-infection with SARS-CoV-2 suggest that prior exposure, and even seroconversion, do not universally protect against SARS-CoV-2 infection(44, 45). This could result from low antibody titers as shown in an immunocompromised patient(46), low durability of infection-induced antibodies(42–44), or low neutralizing potential of SARS-CoV-2 infection-induced antibodies in some individuals(34). IgG and IgM antibodies may also target irrelevant epitopes outside the spike and receptor binding domains, and consequently be less efficient at intercepting infection by the virus(47). The WHO cautions against interpreting presence of antibodies, even neutralizing ones, as lower risk of re-infection and transmission(48). The presence of antibodies could be however used for rapid immunosurveillance to monitor extent of population transmission, particularly in asymptomatic but SARS-CoV2-exposed individuals(3, 33).
One major concern about the deployment of these tests is the misinterpretation of positive results(7, 39). As more tests move towards FDA approval of home use, clear scientific communication about the result interpretation becomes more crucial(23, 39). A positive serological result does not necessarily mean active infection(23, 31, 49). Although combined use of molecular and seroconversion results can be used to confirm the diagnosis of symptomatic and hospitalized individuals(35), a positive serological test in the absence of symptoms dissociates the presence of the antibodies from the time of infection(44). Additionally, it is important to understand the implication of false positive and false negative results, particularly in the context of a low to mid-prevalence disease such as COVID-19(25). Low prevalence decreases the negative predictive value of a test, but increases the false positive rates(25). A false positive serological test result may prematurely instill confidence that one has immunity against SARS-CoV-2 infection, thus resulting in behavioral changes that increase risk of transmission(50). Hence, the probability that a person without antibodies will test negative on a serological test is more important than test sensitivity(48–50).
Our study presents a few limitations. Although we successfully benchmarked the performance of the LFAs to a quantitative assay(29), we did not determine the neutralizing potential of these antibodies. Secondly, samples were acquired when PCR testing was restricted to severely ill patients. For epidemiological studies and population surveillance, it will be important to evaluate assay performance on asymptomatic individuals.