The accuracy of virtual fractional flow reserve from invasive angiography: importance of the method of reconstructing three-dimensional coronary artery anatomy.

Background Three dimensional (3D) coronary anatomy, reconstructed from coronary angiography (CA), is now being used as the basis to compute ‘virtual’ fractional flow reserve (vFFR), and thereby guide treatment decisions in patients with coronary artery disease (CAD). Reconstruction accuracy is therefore important. Yet these methods remain poorly validated. Furthermore, the magnitude of vFFR error arising from reconstruction is unkown. We aimed to validate a new method for 3D CA reconstruction and determine the effect this had upon the accuracy of vFFR. Errors arising from the epipolar line projection method used to reconstruct 3D coronary anatomy from CA are small but result in clinically relevant errors in vFFR simulation, amounting to approximately 40% of the total error associated with vFFR.


INTRODUCTION
Invasive coronary angiography (CA) remains the standard method for assessing coronary artery disease (CAD) and guiding its treatment with percutaneous coronary intervention (PCI). Standard CA provides a two-dimensional (2D), dynamic, arterial 'luminogram' which is interpreted visually by the operator. By applying mathematical algorithms, 3D coronary anatomy can now be reconstructed using computer software from orthogonal 2D projections. 3D CA reconstructions more accurately capture lesion length, plaque eccentricity and correlate better with functional measures of disease, which is frequently overestimated using 2D QCA. 1,2 Moreover, by applying the governing equations of fluid dynamics to these reconstructions, physiological parameters such as pressure and flow can be predicted.
3D CA reconstructions are therefore now being used as the basis for the computation of clinical indices of coronary physiology such as fractional flow reserve (FFR). 3 This is a significant development, because FFR improves patient outcomes and is considered the 'gold standard' method for selecting appropriate cases for PCI in international guidelines. [4][5][6] The ability to compute FFR using 3D reconstructions from CA, known as angiography-derived or 'virtual' FFR (vFFR), is anticipated to widen access to the benefits physiologically-guided intervention considerably, allowing the clinical benefits of FFR to become available to patients without the need for to pass an invasive pressure wire. 7 It is, therefore, important that vFFR is accurate.
Compared with invasively measured FFR, vFFR has an error range (95%CI) of FFR ±0.14. 8 vFFR errors arise from inaccuracies in the 3D CA reconstruction and in the assumptions and simplifications in the mathematical solution. 7 Despite the importance of accurate 3D CA reconstruction in applications which are used to compute vFFR, there is a paucity of published data validating their accuracy.
Furthermore, the amount of vFFR error attributable to the reconstruction method is unknown.
The aims of this study were to: 1. Validate the geometric accuracy of a new method for reconstructing 3D coronary artery anatomy from standard CA.

2.
Quantify how much vFFR error results from errors in the reconstruction method.

Study design
The study was performed at the University of Sheffield. Clinical data were collected from adult patients at the South Yorkshire Cardiothoracic Centre at Sheffield Teaching Hospitals (STH) NHS Foundation Trust, as approved by the NHS regional ethics board. Participating patients provided informed constent. All methods were carried out in accordance with relevant guidelines and regulations. First, we developed a new method of 3D CA reconstruction. Next, the accuracy of this reconstruction method was validated using phantom arterial models which were imaged with CA according to standard imaging protocols and then reconstructed.
The geometry (surface similarity, centreline and diameter) of the reconstruction was then compared with the known geometry of the phantom models. This step validated the geometric accuracy of the reconstruction method. We then applied computational fluid dynamics (CFD) modelling to the reference files (the files from which the phantoms were 3D printed), and to the reconstructed arteries, to predict vFFR. By comparing the vFFR results, we were able to quantify the amount of vFFR error attributable directly to the reconstruction method. A summary flowchart illustrating the study design is shown in Figure 1. A detailed explanation follows below.   Two angiographic projections were selected, ≥30° apart, ensuring optimal   visualisation of the artery and stenosis, during

Creating the coronary phantom models
Two types of arterial phantom models were created. To assess the method's accuracy in reconstructing vessel diameter, particularly in the stenosis region (minimal lumen diameter), phantom models were created from 3.2 mm diameter aluminium rods, which were curved to mimic the contour of the epicardial surface.
Both concentric and eccentric stenoses were fabricated by hand. Seven fabricated stenoses represented anatomically mild, moderate and severe disease, ranging from 40 to 90% diameter stenoses. The aluminium rods were chosen because, unlike the 3D printed models, they were strong enough to maintain shape and integrity even with severe stenoses. Diameter measurements were made with a high-precision digital caliper (Mitutoyo, KA, Japan) as the average of three readings. The second type of phantoms were 3D-printed using as their basis, the CAs of patients with chronic coronary syndromes in whom FFR had also been measured. These solid phantom models were used to assess the accuracy of the reconstruction's overall surface topography. These were based on the angiograms of patients with chronic coronary syndromes, mean age 66 years, who were being assessed for PCI with FFR guidance. Three left and three right branched coronary arterial (LCA, RCA) models were printed using stereo lithography (Rep Rap X400 PRO 3D printer) in polylactic acid (PLA) doped with stainless steel to mimic the radiodensity of contrastfilled vessels under CA imaging. The six models comprised a total of 15 individual branches, which were analysed separately, as they normally would be clinically and physiologically.

Angiographic imaging
All phantom models underwent routine coronary angiography (multiplane 2D The pixel:mm ratio for the Philips angiography system was 1 pixel =0.314 mm.

Ensuring the phantom models were radiographically realistic
To ensure the phantom models were radiographically realistic under angiographic imaging, we sampled the radiodensity of the raw CA images (pixel greyscale value, MATLAB, MathWorks Inc) at ten equally spaced intervals along the length of the main vessel of each phantom type (metal and 3D printed model angiograms). These were compared to a similar number of sample points (n=90 points in total) from the main vessel of a patient coronary angiogram to enable a comparison. Differences were compared by one-way ANOVA.

Accuracy of the 3D printed phantom
Our validation relied upon the 3D-printed phantom models being a true likeness of the files from which they were printed because these reference files were the goldstandard comparitor. To evaluate this, the phantom models were imaged in a 320slice computed tomography (CT) scanner (Aquilion Genesis, Toshiba Medical Systems, Japan) at STH. The 3D rendered geometries were extracted using Materialise Mimics software (Materialise, Belgium) and compared with the reference files they were printed from, by calculating the global Hausdorff distance within MeshLab. This was to validate the use of the 3D reference files as the gold-standard comparator.

Validation of the reconstruction method
Reproducibility of the method: It was important to understand the magnitude of error associated with the comparison method itself, i.e. the highest achievable accuracy. To assess this, we compared each 3D reference file against itself within MeshLab and measured the global Hausdorff distance between the two surfaces.
Theoretically, comparing two identical surfaces should result in a Hausdorff distance close to, or equal to, zero. This analysis demonstrated the best achievable Hausdorff distance and the error inherent to the sampling method within the MeshLab software.

The phantom arterial models
The aluminium stenosis models comprised four eccentric stenoses and three concentric with stenoses ranging from 44.7% to 77.2% representing the mild, moderate and severe clinical range. The seven alluminium and fifteen patientspecific 3D-printed phantom models were each imaged three times, generating 66 analyses in total ( Table 1).

Accuracy of the phantom models
When comparing the CT imaging results against the reference files, the Hausdorff distance was 0.61 mm (±0.16 mm, 95% CI 0.45 -0.78 mm). We concluded that the printed phantoms were an accurate embodiment of the reference files from which they were printed from and, therefore, that the reference files could be used as the 'gold standard' reference comparator for the validation analyses.

Reproducibility of the printed phantoms
When the fifteen 3D arterial files were superimposed and compared against themselves (i.e. the same arterial 3D file opended twice and overlayed in 3D space), the Hausdorff distance was 0.28 mm (±0.20 mm). This defined the best achievable accuracy with this method and therefore the inherent error associated with the sampling method itself. This provided context for assessing the validated accuracy of the reconstruction method

Accuracy of the method in reconstructing minimal lumen diameter
For the aluminium stenosis phantom models, compared with the digital caliper measurements, the average error in minimum diameter measurement was 0.05 mm (±0.03). The error as a percentage of the minimum diameter measurement (i.e. max stenosis) was <1% (±0.87%, 95% CI 0.13% to 1.61%). There was no statistical difference between the caliper measurements made on the phantom models and those from the reconstructions created by the novel method (P=0.93). Model analysis is detailed in Table 1.

Accuracy of the method in reconstructing luminal surface topography
For all 45 3D CA reconstructions (reconstructed from the 3D-printed phantoms), the     Table 3. Per-vessel vFFR measurements for reconstructed and reference meshes. The observed error in physiological simulation for 3D reconstructed vessels when compared to their reference (print mesh) counterparts is also reported.

DISCUSSION
We have developed and validated a method for reconstructing 3D coronary artery anatomy from standard CA suitable for using as the basis for the computation of Reconstructing 3D coronary anatomy from conventional (2D) CA is challenging.
Epicardial coronary arteries are only 2-5 mm in diameter, smaller in stenosed regions, tortuous, and constantly moving due to cardiac, ventilatory and patient activity. In addition, the x-ray table is moved between and during image acquisitions, arteries overlie each other, and an individual projection may not adequately show all regions of interest. Early innovations attempted to overcome some of these challenges by rotating the C-arm while acquiring images, or by simultaneous biplane acquisition but these systems are not widely available, are cumbersome, and associated with significant practical shortcomings. 10,11 A strength of the methods described this study is that a standard multiple single-plane CA is all that is required to reconstruct the coronary anatomy.
Uniquely, but importantly, our study associated geometric with computed physiological error. The physiological (vFFR) error arising purely from errors in the reconstruction was FFR ±0.06 (Bland-Altman 95% limits of agreement). Whilst this is clinically relevant, it needs placing in context. A recent meta-analysis of thirteen studies of vFFR reported average error of vFFR ±0.14. 8 The previously reported error of the current vFFR method was FFR ±0. 16. 9 Similarly, a study of vFFR computed from CT angiography reported an error of ±0.15 (24). 12 Thus, our data suggest that approximately 40% of the error associatated with vFFR is attributable to errors in the reconstruction process. This corresponds with the findings of published vFFR sensitivity analyses which highlight that the major source of vFFR error arises from the selection and tuning of boundary conditions to represent the microvascular resistance and flow, and not the anatomical reconstruction per se. 7,13,14 Thus, the present study supports the notion that, in the context of the computation of vFFR from coronary arterial reconstructions, the greatest scientific challenge remains the selection and application of appropriate boundary conditions, because these are the main contributor to the remaining error.
In the context of the MLD, which is important when computing vFFR, our method was associated with <1% error compared with digital caliper measurement. In terms of surface similarity, the lowest possible achievable error (when we compared identical surfaces) was 0.28 mm (±0.20 mm). The error of our reconstruction method (0.65 mm ±0.30 mm) was therefore only 0.37 mm above the best achievable. Given that the pixel sizing of the Philips angiography system was 1 pixel = 0.314 mm (at 512 by 512 pixel resolution), our model was accurate, at worst, to two pixels; and, at best, to a single pixel. This is reassuring because no system can be more accurate than a single pixel. It would therefore be interesting to test the method on higher resolution detectors (1024 by 1024). 15 Furthermore, the error we observed in our method was almost identical to that of high-resolution CT reconstruction (0.65 mm ±0.30 novel vs 0.61 mm, ±0.16 CT), an imaging modality used routinely in clinical practice for coronary angiographic reconstructions and vFFR computation. 16 Our analysis was more detailed and pragmatic than previously published studies. physical coronary phantom models to assess the accuracy of their 3D arterial reconstructions. However, these studies considered only the accuracy of reconstructed centreline data, using Euclidian distance measurement. [17][18][19] This is different to our study because we also integrated an assessment of the luminal surface topography (which incorporates centreline accuracy), stenosis capture, and physiological accuracy. Furthermore, the models used in these studies were relatively simple, contrast-filled tubes with narrowings made to mimic stenoses, not reflecting actual patient anatomy. One of the challenges of reconstructing 3D models from 2D angiographic images is in capturing vessel curvature and dealing with images that foreshorten the arterial anatomy. It would not be possible to assess this in these phantom types. In their comparison of centreline data, Yang et al, described their average positional accuracy (distance between reconstructed and true phantom centrelines) to be 0.665 mm, similar to the average error in surface reconstruction reported in our study. 19 Centreline comparison provides an analysis of the system's ability to capture the curvature of the vessel, however it gives no information on the quality of surface topography or diameter accuracy. Shechter et al subjected their phantoms to magnetic resonance imaging and used the resultant reconstructed centreline data as their comparator. 17 Arguably, using this method, errors in the process of obtaining centreline data for the imaged phantoms may influence the results; it is more a comparison of reconstructions rather than validation against a ground truth comparator.
Other Although this is ideal for concentric lesions, it is not representative of real-world atherosclerotic plaque that is usually eccentric, at least to a small degree. 24 It is, therefore, possible that most tools for this type of 3D reconstruction are introducing a small magnitude of error as a result of circular diameter approximation. Galassi et al refined their 3D reconstruction algorithm to better account for more complex luminal contours by using a Non-Uniform Rational B Spline contour, allowing for flexible freeform diameter reconstruction. 15 A comparison of luminal diameter was made against intravascular imaging using optical coherence tomography. In a sample of vessels with true diameters ranging between 0.72-1.03mm, the average error was 0.29mm despite the use of 'flexible' luminal modelling. Our method still supports the analysis of eccentric stenoses, including those in the current study, because the centreline deviates accordingly. Nevertheless, this is a potential limitation, and there are algorithms that compute non-circular cross-sections: we would observe that there is simply not enough information in two projections to identify the shape of the crosssection uniquely, although the more sophisticated processes certainly produce better approximations than axisymmetric. 15 However, the primary purpose of this process was to produce geometries to support physiological computation, and separate exploration of the sensitivity of this parameter to the assumption of circular crosssections had indicated that this is a minor limitation in comparison to other sources of error, especially estimation of the distal myocardial resistance. 13 The sample size in our study was small. However, it was larger than those used in other validation studies in this area. It also utilised patient-specific 3D-printed models. [17][18][19] This enabled a unique, clinically relevant analysis. Although the phantom models were static, this is unlikely to affect results because our tool incorporates a correction for movement in any plane (x,y or z) alongside ECG-gating which was designed and implemented to compensate for cardiac and respiratory motion and for radiographic panning.

CONCLUSIONS
We have developed and validated a tool that reconstructs 3D coronary arterial anatomy from standard CA. The method was simple to use and compensated effectively for inter-and intra-acquisition movement. Errors in reconstruction were small and led to small but clinically relevant errors in vFFR accounting for around 40% of the total error associated with vFFR.