Multicenter evaluation of EIAs and CLIAs in the detection of hepatitis C virus antibody among blood donors in China

Background: Recent data on anti-HCV screening assays among large population of blood donors is limited. The present study aimed to perform a multicenter evaluation of EIAs and CLIAs for detection of anti-HCV among blood donors in 16 different Chinese blood establishments. Methods: A total of 1,309 samples including 582 anti-HCV screening negatives and 727 positives collected from 15 blood establishments all over China. Ten different anti-HCV assays (eight EIAs and two CLIAs) were evaluated in 16 different blood centers/banks. Con�rmatory testing was performed using recombinant immunoblot assay and HCV RNA tests. Results: The plasma panel contained 963 negative samples, 261 positives, and 85 indeterminate samples, based on the results of con�rmatory test. False positive rate of screening tests was 39.67% (382/963) and the positive prediction value was only 35.76% (260/727). Among ten anti-HCV assays, Roche and InTec had the highest sensitivity (98.47%), while KHB and Wantai (indirect) had the highest speci�city (99.23%). Then we analyzed the combined performance of these assays with two assays’ strategy widely used in China: Ortho or Abbott together with InTec could �nd all the true positives and Wantai (indirect) with Livzon (sandwich) got a highest speci�city of 97.80%. Indeterminate samples showed quite different signal to cutoff (S/CO) ratios tested by Roche compared with con�rmed positives (4.84 vs 19.36, p<0.0001), and higher S/CO ratios than con�rmed negatives (4.84 vs 2.94, p=0.020). Conclusion: False reactivity in anti-HCV screening should be treated as urgent issue. RIBA indeterminate donations may be a special group, while it still worth to be further studied.


Background
Hepatitis C virus (HCV) infection is of special concern due to a rising burden of its related cirrhosis, hepatocellular carcinoma (HCC), and liver related death worldwide [1][2][3].In 2015, the global prevalence of viremia HCV is estimated to be 1.0%, which means approximately 71.1 million population worldwide are suffering from Hepatitis C and HCV-related diseases [4].Especially in China, the proportion is about 0.7% (more than 9 million people are infected).
HCV is also a blood-borne virus.Laboratory diagnosis of HCV infection is usually made on the basis of the detection of circulating antibodies and/ or HCV RNA [5].Serological tests are of great importance to preventing transfusion transmitted infections, especially because large majority of HCV infection are anti-HCV reactive and HCV RNA negative [6].Enzyme immunoassay (EIA) and chemiluminescent immunoassay (CLIA) are two kinds of widely used serological screening methods.In China, serological screening strategy of HCV infection involves two different anti-HCV EIA assays that are approved by National Medical Products Administration (NMPA) and if they show inconsistent results, twice repeated tests using the reactive one are performed.When any of the two EIAs is repeatedly reactive, the donation is disquali ed regardless of the NAT results and the donor is deferred for safety [7].CLIA, which has been widely used in recent years, can automatically and quickly test anti-HCV antibodies from serum/plasma samples with higher sensitivity and speci city but haven't been used in Chinese blood establishments yet.
Recent data on anti-HCV screening assays among large population of blood donors is limited..This infomation is of critical value to analyze and nd difference among different assays and improve assays' performance.The present study aimed to perform a multicenter evaluation of enzyme immunoassays and chemiluminescent immunoassays for detection of antibodies against hepatitis C virus among blood donors in 16 different Chinese blood establishments.

Plasma panel
A total of 1,309 plasma samples collected by 15 different blood establishments during Apr 2015 to Sep 2015, including Changchun (abbreviated as CC), Chongqing (CQ), Hebei (HB), Heilongjiang (HLJ), Henan (HN), Heze (HZ), Jining (JN), Jiangsu (JS), Kuming (KM), Liaoning (LN), Shandong (SD), Shenzhen (SZ), Tianjin (TJ), Tongzhou (TZ), Xiangyang (XY) blood center/blood bank.Among them, there were 582 anti-HCV negative randomly selected donations (HBsAg, anti-HCV, anti-HIV, and anti-TP antibodies were all negative with unquali ed ALT level) and all the 727 anti-HCV positive donations detected in blood establishments during the period were included.All the plasma bags were sent to the Chinese National Center for Clinical Laboratories (NCCL), con rmed, split, numbered, stored at -40°C, and then sent back to all the participants.The transportation of the plasma panel was at 2-8°C and within 3 days.Sample source were listed in Table 1.All these samples were from 1,309 blood donors, no consecutive samples from one donor included.

Study design
16 different blood centers or blood banks over China participated into this multicenter evaluation, including CC, CQ, Cangzhou (CZ), HB, HLJ, HN, JN, JS, KM, LN, SD, Shanghai (SH), SZ, TJ, TZ, XY blood center/bank.The moment all these establishments above received the panel, they used one or two different EIAs to test it within 3 days by automated detection systems.Each sample was only tested once by one assay.All the results were sent back to NCCL within 10 days.Besides, NCCL tested all the samples once by two different CLIA assays simultaneously.All results were performed according to the manufacturers' instructions.

Con rmatory tests and evaluated assays
All the donation samples were subjected to con rmatory testing with anti-HCV recombinant immunoblot assay (Mikrogen Diagnostic, Martinsried, Germany), which showed the results as positive, negative, and indeterminate (IND).All the indeterminate samples were further tested HCV RNA by Cobas Taqscreen MPX Test, Version 2 (Roche Molecular systems, Branchburg, NJ).All results were performed according to the manufacturers' instructions.Technical speci cations of con rmatory assay see Table 2.There were 10 different commercially available anti-HCV assays in this study.Technical speci cations of evaluated assays see Table 2.

Data and Statistical analysis
Since plasma panel was tested by different blood establishments and different evaluated assays in the study, majority results on one sample would be treated as a "true value", if not all results from different blood establishments were consistent.
The sensitivity, speci city, Youden's index (The formula was below) and Kappa coe cient of each assay and combined assays were calculated by SPSS software (v.21.0).Results were analyzed with One-Way ANOVA analysis and Chi-square test.p<0.05 was considered as statistically signi cance.Sensitivity = number of positive results/ number of con rmed positive in %; Speci city = number of negative results/ number of con rmed negative in %; Youden's index = Sensitivity+Speci city-1

Con rmatory results of plasma panel
In 1,309 plasma samples, 963 were true negative, 261 were true positive by con rmatory testing.85 samples were indeterminate by immunoblot and HCV RNA were also undetectable.The nal results and validation algorithm see Figure 1.In 727 anti-HCV initial reactive samples only 260 were true positive, indicating the average positive prediction value (PPV) of anti-HCV screening tests was only 35.76% (260/727) and the average false positive rate was as high as 39.67% (382/963).Besides, we also found one true positive samples from Liaoning blood center were undetected by screening assays.The con rmatory results from each establishments was listed in Table 1.

Performance of evaluated assays
Among all the true negatives, 724 (724/963, 75.18%) were detected negative by all 10 assays.209 (209/261, 80.08%) positives were detected positive by all assays.376 samples showed discrepant results, among which there were 52 con rmed positive samples could not be detected by all evaluated assays (Table S1).InTec and Roche detected 48 samples, while Abbott only detected 29 ones.All samples had positive results of antibody against HCV protein Core 1, while other antibodies were weaker than con rmed positive samples with consistent results by all evaluated assays.On the basis of assay's results of 963 true negatives and 261 true positives, we calculated the sensitivity, speci city, Youden's index and Kappa Coe cient.The results showed assays' sensitivity varied from 91.19% to 98.47%, InTec and Roche owned the highest sensitivity (98.47%), while sensitivity of Abbott was the lowest (91.19%).As a routine strategy in China, two different kinds of EIAs are used simultaneously.Thus we calculated the combined performance index between two different assays.The combined sensitivity showed that when we use InTec and Ortho, 100% positives could be detected.Similarly, different assays showed different speci city (92.94%-98.23%),KHB was the highest, while InTec was the lowest.When Wantai (sandwich) and Wantai (indirect) were used together, we could get the highest combined speci city (97.92%).Youden's index, a way of summarizing the performance of a diagnostic test, equals sensitivity plus speci city minus 1. Roche got the highest index (96.08%)and the two EIAs, KHB and Livzon (indirect), owned the highest combined Youden's index.Results of Kappa coe cient, a statistic which measures inter-rater agreement, indicated that Wantai (indirect) had perfect agreement with con rmatory results, and Wantai (indirect) and Livzon (sandwich) together owned the highest kappa coe cient.As a whole, CLIAs didn't perform better than EIAs on the detection of anti-HCV in this study.All the detail results see Table 3.
Different assays showed quite different results on these indeterminate samples: of all the indeterminate samples, 45 were detected by Abbott, while Wantai (sandwich) and Murex got only 25 positives.In the aspect of antigen distribution among different assays, most assays showed similar results as the total, except Murex.Helicase single in Murex positive indeterminate samples accounted for a main proportion (40%), next is core1 single (32%) and Helicase/NS3 format (28%).
Besides, we also analyzed the S/CO ratios of Roche and Abbott among 367 samples with discrepant results.There was statistically signi cant between the indeterminate and positive (4.84 vs 19.36, p<0.0001), between negative and positive (2.94 vs 19.36, p<0.0001) and between determinate and negative (4.84 vs 2.94, p=0.020) detected by Roche.However, we didn't nd any difference among the three groups when detected by Abbott (positive 1.40 vs indeterminate 1.74 vs negative 1.55, p=0.567) (Figure 2)

PPV of CLIAs on different S/CO ratios
In order to determine the relationship between PPV and S/CO ratio on two CLIAs, we analyzed the assays results when different S/CO ratios were as the cutoff values of the two CLIAs.When S/CO ratio went to 1.0, Roche's PPV on detection of anti-HCV was 92.09% (256/278), and Abbott was only 78.55% (238/303).When PPV was above 95% (246/258 on Roche, 196/205 on Abbott), cutoff S/CO ratios need to be more than 8.2 on Roche, and more than 4.2 on Abbott.When PPV was 100% (65/65 on Roche, 177/177 on Abbott), cutoff S/CO ratios need to be 46.8 on Roche, and 8.0 on Abbott.Results of ROC and AUC indicated that the two assays both showed a good AUC: Roche was 0.994 and Abbott was 0.984.However, the best S/CO was quite different on the two: S/CO ratio=1.515 was the best cutoff for Roche, on this S/CO the assay's sensitivity was 98.08% and the speci city was 97.82%.Since the relatively low sensitivity of Abbott, the best S/CO was only 0.385 (sensitivity=99.23%,speci city=86.50%).

Discussion
False positive reactions in anti-HCV screening assays is a well-known concern and an obstacle to improve assays' performance all along.In the present study, we found that the average false positive rate of anti-HCV screening assays was as high as 52.54%, which meant large amount of donations were discarded and quali ed donors were deferred unfortunately.It is reported in Uganda, 7.6% (76/1000) were serologically reactive by Ortho, but none were con rmed [8].Similarly, Schroter M, et al [9] also found false positive results by widely used anti-HCV EIAs were at least 10%, which, compared to other virus screening assays, was unacceptable.In another study in Africa, only 13% of initial anti-HCV positive samples were con rmed by NAT or immunoblot and 2.46% (409/16,613) of blood donations were wrongly discarded [10].
Whereas the false positive rate in this study might partly result from the serological screening strategy in China.Use of two EIAs no doubt would relatively increase the sensitivity but sacri ce the speci city to a certain extent.In addition, among all screened positive samples in the panel, 40% were repeated reactive by single assay, which contributing only 1% con rmed positivity and leading to the high false positive rate.In the next place, samples used in screening tests were directly from blood collection tubes, while samples in the plasma panel were from plasma bag, which contained about 20% blood preserving uid, diluting the samples and resulting in missing detection of some weak positives.Besides, researchers also found in low risk populations, 40-50% of screening reactives were negative in supplemental immunoblot tests, which was similar to our results [11].
Secondly, performance analysis of ten different assays indicated that generally all assays' performance was worse than reported [12][13][14][15][16][17][18][19][20] or claimed by their instructions, which may be mainly due to sample source.Large amount of false positive samples increased the di culty of testing a lot and false positive results of an assay during the rst screening are likely to be false positives again.Speci cally, Roche got the highest sensitivity (98.47%) and Youden's index (0.961) but speci city (97.61%) was lower than four EIAs (KHB 98.23%, Wantai (indirect) 98.23%, Wantai (sandwich) 98.03% and Livzon (sandwich) 98.13%).The other CLIA, Abbott didn't perform well: it had the lowest sensitivity, but its results were highly comparable to those of Ortho, which also has been shown in other studies [18][19][20][21].Therefore, Two CLIAs especially in the respect of speci city didn't show much better performance than EIAs, no matter imported assays or domestic ones.
We then analyzed the combined performance of any two EIAs assays.Interestingly, not two assays both with good sensitivity had the best combined sensitivity, but Ortho or Abbott together with InTec had a 100% sensitivity indicating these two assays just made up for each other's missing detection.In the terms of speci city, though use of two assays would de nitely decrease detection speci city, we found that Wantai (indirect) together with Livzon (sandwich) got a 97.80% speci city, even better than only using one assay (InTec 92.94%, Ortho 90.13%, Livzon(indirect) 93.15%, Murex 97.20%, Roche 97.61%, Abbott 93.25%).
Based on the performance of ten evaluated assays in the study, we recommended Roche for screening anti-HCV in blood donations and to those could only use EIA assays, Wantai (indirect), Wantai (sandwich), and Livzon (sandwich) also performed well.If blood establishments use two assays' strategy when testing anti-HCV antibody, Wantai (indirect) with KHB, Wantai (indirect) with Livzon (sandwich), or Wantai (sandwich) with Livzon (sandwich) would be a better choice.In addition, a con rmatory testing is necessary for the initial positive samples.
Usually, an immunoblot test and/or NAT are served as a way to con rm screening reactive samples.However, the recombinant immunoblot assay as a con rmatory method for serological screening test of HCV, is of high cost and laborious to be used in every laboratory in China and these con rmatory tests often give many indeterminate and/or HCV RNA negative results, indicating the donor may be a resolved HCV infection, infection with a different genotype, early seroconversion, occult HCV infection or just nonspeci c reactivity [22], which makes us much di cult to estimate the donors' true status.Like our results, we got 85 (6.49%, 85/1,309) indeterminate/HCV RNA negative samples, and most (53/85, 62.35%) were due to isolated reactivity to antigens from the core1 region of the HCV genome.We also found these indeterminate samples showed quite different average S/CO ratios detected by Roche compared with con rmed positives (4.84 vs 19.36, p<0.0001), and higher S/CO ratios than con rmed negatives (4.84 vs 2.94, p=0.020).Similarly to Kiely P's results [22], the antigen distribution and band intensity were different between indeterminate samples and con rmed positives.All these results indicated indeterminate samples were a special group different from negative and positives.
Finally, we evaluated the corelationship between PPV and S/CO ratio on two CLIAs.When S/CO was 8.2 on Roche and 4.2 on Abbott, the PPV could achieve more than 95%.The S/CO ratio of ARCHITECT anti-HCV established for FDA-approved is more than 5.0 [5], similar to our results.But there is no such approved ratio on Elecsys Anti-HCV II.Some studies suggested that the value predictive of a true positive ≥95% of the time with the assay could be set at an S/CO ratio of 20.0, much higher than our study [14,16], which meant it was required to verify the value in further study.
There were some limitations in the study.First, not all the information of anti-HCV positive donors were available in this study, therefore we didn't know the true infection status of these donors.Secondly, different automated sampling and testing equipment were used in the 16 different blood establishments involved in the evaluation study, and all these difference may in uence the detection accuracy.

Conclusion
Based on the multicenter evaluation of 10 common used anti-HCV screening assays, false reactive problem of anti-HCV screening should be solved urgently.Assays showed different performance on sensitivity and speci city and CLIAs didn't performed better than EIAs, especially on speci city.Indeterminate samples may be a special group, should be further studied.
Sensitivity (red), specificity (green), Youden's index (blue), and Kappa Coefficient (purple) of single assay and two combined assays were shown.The darker the color, the better performance the assays have.The number in the black box shows the best performance (sensitivity, specificity, Youden's index, or Kappa Coefficient) of all the combined assays.

Figures
Figures

Figure 1 Con
Figure 1

Table 2 .
20. Zachary P, Ullmann M, Djeddi S, Meyer N, Wendling MJ, Schvoerer E, et al.Evaluation of three commercially available hepatitis C virus antibody detection assays under the conditions of a clinical virology laboratory.J Clin Virol 2005;34(3):207-10; discussion 16-8.21.Oethinger M, Mayo DR, Falcone J, Barua PK, Gri th BP.E ciency of the ortho VITROS assay for detection of hepatitis C virus-speci c antibodies increased by elimination of supplemental testing of Technical specifications of evaluated assays and confirmatory assay No. Abbreviations of assays ECLIA: electrochemiluminescence immunoassay; CMIA: chemiluminescent microparticle immunoassay.‡ N/M: not mentioned in the package inserts.

Table 3 .
Performance index of different assays