Number of Experimental Assays Compared to Computationally Guided Prediction Assay Projections
To experimentally determine epitopes for 757 peptides spanning the whole HIV-1 proteome for clades A and D as well as both time points of the 6 individuals required a total of 4230 test assay wells. For each test subject these included 9 antigen proliferation wells, 384 culture ELISPOT wells and an average of 164 epitope mapping ELISPOT wells (Range; 148 – 186 test wells). Using the 22 HLA alleles represented study subjects we were able to computationally predict 95% of the experimentally mapped epitopes. This approach could have reduced the test assays by eliminating all the T-cell antigen proliferation and culture ELISPOT steps totalling to 3258 assay wells (77%) and leaving only 972 (23%) epitope mapping assay required. Applying a pooling strategy to the computational predictions similar to that used in the experimental pooling where each pool contained approximately 20 peptides with a coverage of 3 per peptide, the 923 potential peptides (95% of experimental peptides for epitope mapping ELISPOT derived from the 972 (23%) eligible epitope mapping peptides) would make at most 46 pools. Consequently the computational prediction approach could have reduced the experimental assays by at least 80%.
Magnitude of Epitope Predictions are Variable across HLA Alleles, HIV-1 Proteins and Clades
The input HIV-1 subtypes A1 and D consensus whole proteome sequences evaluated for potential 9, 10, 11, 12, 13 and 14-mer binders to the 22 HLA alleles represented in the six patients, varied in the distribution of predicted binders across HIV-1 genes and HLA alleles. All the peptide hits predicted for 10 through 14-mer were also all predicted in the 9-mer set except for two 14-mer peptides. An expected positive correlation for protein size with number of predictions as well as with HIV-1 protein length was observed as illustrated by Spearman’s rank order correlation; rs=0.88 (Figures 1, A and B). NetMHCpan4.0 predicted 95% (88/93) of the experimentally mapped peptides as binders and missed 5% (5 out of 93) for the 12-time points of the 6 participants (Table 2). MHCflurry predicted 91% (85/93) of the experimental peptides and had a lot of similarity to NetMHCpan4.0 for the predicted HLA. NetCTL was the least performing tool with only 15% (14/93) predicted experimental peptides (Table 2).
Comparison of the various epitope prediction length set showed that the 9mer setting was ideal for NetMHCpan4.0. The number of predictions were 88, 79, 55, 39, 39 and 37 hits out of 93 for 9, 10, 11, 12, 13 and 14-mer epitopes respectively. Increasing the prediction length from 9mer through 14mer resulted in a smaller number of predicted binders as illustrated in Figure 2. Since we held the assumption that our wet experimental data was the gold standard we evaluated the sensitivity and specificity of the NetMHCpan4.0.The computational predictor had more predicted binders than those determined by the experimental mapping as presented in the confusion matrix in Table 1. The experimental positive’s count also shown in table 2 under column “Hits” shows the test peptide count (1through 88) that contained the computational 9-mer sequence. Multiple computational epitopes may be contained in a single experimental peptide, as shown in the column “NetMHCpan4.0 9-mer Epitope Prediction” in Table 2. Overall HIV-1 Clade A 9-mer predictions were fewer in number than clade D (figure 1, C) though the difference did not approach statistical significance.
Comparison of Experimentally Mapped Epitopes with in-silico Prediction
The experimental peptide mapping data was derived from a baseline time point corresponding to HIV-1 Fiebig stages IV, V and VI (table 4) and a later time point. Ninety-three (n=93) epitopes were experimentally mapped of which 12 were recognized at both baseline and later time points, 34 only at baseline and 54 only at the later time point. Comparison of the ranked computational score for Netmhcpan4.0 binders of early (n=34) versus later peptides showed that the later time point predictions were stronger binders reaching statistical significance(Wilcoxon signed rank p-value=0.0000005) (figure 3). NetMHCpan4.0 ranked binders as those predicted to be in the top 2% and assigned a score of 0.2 or below. Any binder within the top 0.5% and assigned a score of 0.05 or below was ranked as a strong binder. Considering only the 9-mer computational predictions, peptides that were derived from the same 17-mer experimental peptide were determined by a BLAST mapping to their derivative sequences. The 17-mer peptides were then classified into a confusion matrix (table 1) as either true positives, false positive, true negative and false negative. From the classification the true positive rate (sensitivity) was plotted against the false positive rate (1-specificity) for only 9-mer predictions using an ROC curve and the AUC attained reached 0.928 (Figure 4). Only 9-mer length epitopes were considered in the ROC analysis as increasing the length to 10-mer through 14mer NetMHCpan4.0 predictions neither raised the number of predicted binders nor improved the hit rate as all their predictions contained the sequence already predicted in the 9-mer set except 1 14-mer peptide (hit 72 in table 2).
Comparison of the ELISPOT magnitude of response (spot forming units) did not show any association to either NetMHCpan4.0 scores or MHCflurry1.2.0 affinity values. Similarly a comparison of the latter 2 computational predictors did not show any association between their assigned “affinity” values.
We observed a negative correlation for the number of computationally predicted epitopes with the length of input for netMHCpan4.0 (Figure 2). NetMHCpan4.0 registered the highest concordance to the wet experiments followed by MHCflurry1.2.0.