Performance and usefulness of a novel automated immunoassay HISCL SARS-CoV-2 Antigen assay kit for the diagnosis of COVID-19

Here, we aimed to evaluate the clinical performance of a novel automated immunoassay HISCL SARS-CoV-2 Antigen assay kit designed to detect the nucleocapsid (N) protein of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). This kit comprises automated chemiluminescence detection systems. Western blot analysis confirmed that anti-SARS-CoV antibodies detected SARS-CoV-2N proteins. The best cut-off index was determined, and clinical performance was tested using 115 nasopharyngeal swab samples obtained from 46 patients with coronavirus disease 2019 (COVID-19) and 69 individuals who tested negative for COVID-19 through reverse transcription quantitative polymerase chain reaction (RT-qPCR). The HISCL Antigen assay kit showed a sensitivity of 95.4% and 16.6% in samples with copy numbers > 100 and < 99, respectively. The kit did not cross-react with human coronaviruses causing seasonal common cold and influenza, and none of the 69 individuals without COVID-19 were diagnosed with positive results. Importantly, 81.8% of the samples with low virus load (< 50 copy numbers) were diagnosed as negative. Thus, using HISCL antigen assay kits may reduce overdiagnosis compared with RT-qPCR tests. The rapid and high-throughput HISCL SARS-CoV-2 Antigen assay kit developed here proved suitable for screening infectious COVID-19 and may help control the pandemic.


Results
Western blot. To examine whether the previously produced anti-SARS-CoV antibodies (ab clone# 2-3 and ab clone# 2-12) 24 can detect SARS-CoV-2, western blots were performed using His-tagged recombinant N proteins of SARS-CoV-2 (ACRO Biosystems, Newark, DE). Figure 1 shows that the preanti-SARS-CoV antibodies (left two strips: ab clone# 2-3 and ab clone# 2-12) and anti-His antibody (the third strip from the left) detected the SARS-CoV-2 antigens but not negative control (the far-right strip). The original stained membrane is shown in Supplemental Figure 1.

Determination of the association between Ct values and viral loads.
Since RT-PCR results can be affected by different laboratory settings 25 , relationship between Ct values by a RT-qPCR method 26 and viral copy numbers was determined in our laboratory. SARS-CoV-2 Positive Control RNA (1 × 10 5 copies/µL; JP-NN2-PC, Nihon Gene Research Laboratories) was sequentially diluted (50 to 5,000 copies/sample) and subjected to RT-qPCR. Figure 2 shows the standard curve of Ct values as a function of copy numbers using four RNA samples for each copy number. The plots were fitted with linear regression, yielding the slope of − 3.43 and the intercept of 50.1 (r 2 = 0.999).

Reproducibility.
To check the reproducibility of the HISCL SARS-CoV-2 Antigen assay kit, we measured one sample containing recombinant SARS-CoV-2 antigens (109.5 ± 36.5 pg/mL, ACRO Biosystems, Newark, DE) (positive control sample) and one sample containing only buffers (negative control sample). The tests were repeated ten times for each sample and all experiments were conducted in triplicate. Table 1 shows the COI values for both samples. The negative control samples showed 100% agreement with COI of zero and the positive control samples showed the average COI of 27.7 to 28.8 with a coefficient of variation (CV) percentage of 1.3-2.5.
To check reproducibility between measurements in five different days, we performed assays using the negative and positive control samples twice daily for 5 days. Table 2 shows that the COIs of the negative and positive control samples were consistent (CV% < 10).

Cross-reaction.
To check for specificity, various corona and influenza viral antigens were used to measure COIs in the HISCL assay. Table 3 summarizes the COI measurements. The COI showed high values for SARS-CoV at a concentration of 0.25 ng/mL and positive values for MERS-CoV at a low concentration. In contrast, the COI values stayed in the negative range for other coronaviruses and influenza viruses, indicating that the HISCL SARS-CoV-2 Antigen assay kit does not react with seasonal common cold and flu viruses.

Determination of the cut-off value.
To optimize the cut-off antigen level for accurate diagnoses, we first determined the relationship between COI and viral loads using a total of 84 nasopharyngeal swabs collected from 17 patients at Kobe City Medical Center General Hospital and 67 commercially available samples (Cantor Bioconnect, Santee, CA, USA): 30 positive samples and 54 negative samples.
Chemiluminescence counts were measured by HISCL SARS-CoV-2 Antigen assay kit using these samples and recombinant SARS-CoV-2 proteins (0, 20, and 100 pg/mL). Viral loads (copy numbers) were calculated based on the relationships between Ct values and RNA content (Supplemental Table 1). Sensitivity and specificity were calculated at different cut-off values depending upon the RT-qPCR results (i.e., copy number 0 as negative). In determination of the threshold, we put priority on obtaining best specificity according to the Antigen Testing Algorithm by the Center for Disease Control (https:// www. cdc. gov/ coron avirus/ 2019-ncov/ lab/ resou rces/ Antig en_ Testi ng_ Algor ithm_ 2020-12-14_ v03_ NO_ DRAFT_ SPW_ 508. pdf). When the cut-off for the antigen level was set to 3.65 pg/mL, sensitivity of 80% and specificity of 98.2% were obtained (Table 4). Thus, chemiluminescence count using 3.65 pg/mL antigen was set to a COI value of 1.0 in the HISCL SARS-CoV-2 Antigen assay kit. Figure 3 shows that the ROC analysis yielded an AUC value of 0.8988 ± 0.0464 (95% confidence interval, 0.808-0.990, p < 0.0001). Table 5 summarizes the concordance between the two tests in three viral load ranges using the cut-off value in validation samples.   Table 6 summarizes the concordance between the two tests in the two viral load ranges. The antigen tests showed 95.4% positive results in samples whose viral copy numbers were higher Table 1. Within-run reproducibility. The COI values of samples without or with SARS-CoV-2 antigens were measured a total of 30 times (three sets of 10 consecutive measurements). COI cutoff index, SD standard deviation, CV coefficient of variation.  Table 2. Between-run reproducibility. COI cut-off index, SD standard deviation, CV coefficient of variation.       www.nature.com/scientificreports/ than 100. In samples whose copy numbers were lower than 99, the antigen test showed negative results in 20 samples and positive results in four samples.

Discussion
In this study, we evaluated the performance of HISCL SARS-CoV-2 Antigen assay kit using chemiluminescent enzyme immunoassay in comparison with RT-qPCR test results. This test was approved by the Pharmaceuticals and Medical Device Agency of Japan in November 2020, and it is currently commercially available (Sysmex, Kobe, Japan). This automated test can process 200 samples per hour, and the run time is only 17 min. Therefore, it can be used as a high-throughput screening test.
Since the early phase of this pandemic, RT-qPCR tests have been used for diagnoses, epidemiological studies, and political decision making [27][28][29] . Severe cases with high infectivity usually show high viral loads (i.e., low Ct values) and vice versa 30 . However, the interpretation of positive results needs extra caution because Ct values can vary due to differences in PCR conditions 31 . Importantly, infectivity cannot be determined solely based on RT-qPCR results 11 .
To examine the correlation between RT-qPCR and our new antigen detection assay, we used SARS-CoV-2 RNAs as a standard. Our assay showed 95.4% concordance in the samples with a COI > 100, and the concordance decreased to 16.7% in samples with a COI < 99. LUMIPULSE assay (Fujirebio, Tokyo, Japan) showed 100% concordance in samples with > 100 copies 23 . This may be due to the small sample numbers used in both the studies, but it may also be attributed to the difference in the standard RNA (AccuPlex SARS-CoV-2 Reference Material Kit; SeraCare, Milford, MA, USA) and RT-qPCR methods (QuantiTect Probe RT-PCR Kit and Applied Biosystems 7500 Fast Real-Time PCR System versus StepOnePlus Real-Time PCR System).
Nonetheless, this difference should not be problematic in clinical practice since the viral loads in infectious patients were usually much higher than copy numbers of 100. For example, Yu et al. reported that copy number of SARS-CoV-2 in sputum samples in early-phase infection was 46.800 ± 17.272, and 1.252 ± 1.027 in the recovery phase 32 . Wölfel et al. reported that the pharyngeal virus shedding was high during the first week of symptoms, with a peak at 7.11 × 10 8 RNA copies per throat swab on day 4 33 . Conversely, many studies also reported that there was no apparent correlation between viral loads (copy numbers) and severity of disease or mortality in patients with COVID-19 disease 34,35 . Indeed, the viral loads in our study were much lower than those in the previously reported cases, and mild cases showed similar COIs compared to moderate-critical cases (Supplemental Table 2). These data, including our data, indicate that infectivity and viral loads are not proportional. Thus, we propose this antigen test is considered as a useful screening test. However, diagnosis and clinical decisions should not be made upon any single test. Therapeutic strategies should be determined based on systemic physiological parameters, such as respiratory rate, blood oxygenation, heart rate, and blood pressure, along with the screening test.
There were a few limitations of this study: (1) this study was performed only in Japan with a relatively small number of patients; (2) we could not test the assay for mutated pathogens that might not be bound by our antibody; (3) cross-reactivity with unknown pathogens could not be excluded. However, diagnoses of COVID-19 Data are plotted as logarithmic scales and the plots were fitted with a power approximation. Table 6. Relationship between the SARS-CoV-2 viral loads and COI by the HISCL SARS-CoV-2 Antigen assay kit. SARS-CoV-2 severe acute respiratory syndrome coronavirus 2, RT-qPCR reverse transcription quantitative polymerase chain reaction, Ag antigen.  (4) we could not examine chronological changes of the antigen test results.

RT-qPCR SARS-CoV-2 Ag test
In conclusion, considering the current chaotic situation across the world due to COVID-19, which is in part caused by detecting non-infectious patients with extremely low viral load as positive cases by RT-qPCR, our antigen detection assay may be more suitable than RT-qPCR for mass screening as it would help to moderate COVID-19 overdiagnoses. Further validation of the kit with a larger number of samples and SARS-CoV-2 variants is warranted.

Methods
Patient samples. This research related to human use complied with all the relevant national regulations, institutional policies and was conducted in accordance the tenets of the Helsinki Declaration. It has been approved by the authors' institutional review board at Juntendo University Hospital, Tokyo, Japan (IRB #20-036, Chair: Atsushi Okuzawa, MD) and Kobe City Medical Center General Hospital, Hyogo, Japan (IRB #zn200615, Chair: Yasushi Naito, MD). This study used an opt-out consent method. The Institutional Review Board of Juntendo University and the Institutional Review Board of Kobe City Medical Center General Hospital waived the need for informed consent. Therefore, written informed consent was not required.
Classification of disease severity was determined according to the Clinical Spectrum of SARS-CoV-2 Infection (https:// www. covid 19tre atmen tguid elines. nih. gov/ overv iew/ clini cal-spect rum/). To determine the relationship between photo counts and viral loads, a total of 84 nasopharyngeal swabs were collected from 17 patients at Kobe City Medical Center General Hospital and 67 commercially available samples (Cantor Bioconnect, Santee, CA, USA; validation dataset). For clinical performance analysis, a total of 115 nasopharyngeal swabs were collected from 115 patients at Juntendo University Hospital (test dataset). Western blot. The detection of SARS-CoV-2 Antigens by SARS-CoV antibodies was confirmed via western blot analysis using the previously produced anti-N capsid SARS-CoV antibodies 24 . Histidine (His)-tagged SARS-CoV-2N proteins (10 ng) were loaded on NEXT Page II gel (5-20% gradient; Gellex, Tokyo, Japan), and electrophoresis was performed using iBind Western Device (Thermo Fisher Scientific). Two lots each of anti-SARS-CoV antibodies (0.5 μg/mL) and horseradish peroxidase (HRP)-conjugated anti-His antibodies (Penta-His Antibody; Cat. No. 34660, Qiagen) were used as primary antibodies. Anti-IgG antibodies (Medical & Biological Laboratories, Nagoya, Japan) were used as the secondary antibodies. Immobilon Western HRP substrate (Millipore, Burlington, MA, USA) was used for chemiluminescent detection.

SARS-CoV
HISCL SARS-CoV-2 Antigen assay kit. Previously produced monoclonal antibodies against the SARS-CoV N proteins 24 were used to probe the SARS-CoV-2N proteins (ACRO Biosystems, Newark, DE, USA). The HISCL SARS-CoV-2 Antigen assay kit is a chemiluminescent enzyme immunoassay that uses a HISCL automatic immunoassay analyzer (Sysmex, Kobe, Japan). Figure 5 presents a schematic illustration of the procedure: (1) the samples were incubated with biotinylated SARS-CoV-2 antibodies (R1: antibody 2-3) at 42 °C for 3 min; (2) the mixtures were incubated with streptavidinbonded magnetic particles at 42 °C for 2 min; (3) after protein separation and washing, alkaline phosphatase (ALP)-bound SARS-CoV-2 antibodies (R3: antibody 2-12) were added and the mixtures were incubated at 42 °C for 3 min; (4) after magnetic separation and washing again, the chemiluminescent substrates were added and the mixtures were incubated at 42 °C for 5.5 min; and (5) chemiluminescence signals (CDP-Star, C0712, Sigma-Aldrich, St. Louis, MO, USA) were measured using the photo counter of HISCL-800 (Sysmex, Kobe, Japan). The level of SARS-CoV-2 Ag was indicated as cut-off index (COI), calculated by the difference in the luminescence intensities in the buffers with and without the SARS-CoV-2 antigens.
Human sample collection. Human samples were obtained using nasopharyngeal cotton swabs following the standard method 36 . The swabs were immersed in 0.5 mL phosphate-buffered saline or viral transfer medium Figure 5. Assay protocol of the HISCL SARS-CoV-2 Antigen assay kit. The biotinylated SARS-CoV-2 Ag antibody (R1) was allowed to react with the sample at 42 °C for 3 min. Then, streptavidin-bonded magnetic particles (R2) were added and allowed to react at 42 ℃ for 2 min. After protein separation and washing, ALP-bound SARS-CoV-2 Ag antibody (R3) was added and reacted at 42 °C for 3 min. After another magnetic separation and washing, buffer solution (R4) and the chemiluminescent substrate (R5) were allowed to react at 42 °C for 5.5 min, then the luminescence intensity was measured. SARS-CoV-2: severe acute respiratory syndrome coronavirus 2; ALP: alkaline phosphatase; Ag: antigen. The illustration was drawn by A.K. using Microsoft PowerPoint 17.0 (Microsoft Corporation, Redmond, Washington: https:// www. micro soft. com/ en-us/ micro soft-365/ power point).