Study Design
Three datasets of human sublingual microcirculation were considered in this validation study. Two datasets were previously acquired from patients undergoing cardiac (P1), or general (P2) surgery. Data from the P1 dataset has previously been published.26 The third dataset was prospectively acquired from three healthy volunteer participants to further investigate the effect of video length on agreement between AVA 3.2 and AVA 4.1. Volunteer participants did not receive any anesthesia or undergo any surgical procedure (V1).
Outcome Variables
The variables of interest in the software comparison were perfused vessel density (PVD), total vessel density (TVD), and proportion of perfused vessels (PPV). Unless otherwise specified, PVD and TVD are reported in mm/mm2, whereas PVD is reported as percentage.
Procedures
Measurements
The Microscan (MicroVision Medical BV, Amsterdam, The Netherlands), a commercially available first generation sidestream darkfield (SDF) imaging microscope, was used to capture all microcirculation videos. Images recorded with the Microscan have a resolution (horizontal x vertical) of 1.45 μm/pixel x 1.58 μm/pixel, and a size of 720 x 480 pixels, resulting in a field of view measuring 1.04 mm x 0.76 mm29. Images were recorded at 30 frames per second (fps), converted from analog to digital format using the ION Video2PC converter, and saved in AVI format on a dedicated research computer.
Video quality was evaluated using the six MIQS criteria: illumination, duration, focus, content, stability and pressure. There is not specific MIQS criterion assessing contrast. Nonetheless, videos with poor contrast were typically excluded from analysis due to scoring of their illumination or focus quality scores. Two experienced operators (CG and MB) independently analyzed each of the videos for quality, and videos with a score of 10 on any of the six dimensions of MIQS were excluded. Videos containing bubbles, saliva or blood outside vessels were also excluded. Disagreements were resolved by discussion and if consensus was not reached, the video was excluded.
The exact number of frames for each microcirculation video was recorded. Multiple videos in the P1 and P2 datasets had less than 90 frames, which is the threshold for acceptable video length in the MIQS. Therefore, we recorded videos ranging from 100 to 600 frames in the V1 dataset to investigate whether video length would have a significant effect on agreement. After a V1 video was recorded and passed quality scoring, AVA 3.2 was used to splice the video into increasingly shortened videos by removing a specified number of frames at the end of the video. For example, a 300 frame (10 seconds) video would be used to generate another video with 250 frames, 200 frames, 100 frames, etc.
After undergoing quality scoring, video files with acceptable scores were imported into AVA 3.2 to undergo semi-automated analysis according to consensus guidelines17, and into AVA 4.1 to undergo fully automated analysis. Video files did not undergo down-sampling or any other processing prior to analysis with either software package. The same calibration videos were used for AVA 3.2 and AVA 4.1. Values for TVD, PVD and PPV were then exported into Microsoft Excel (2019).
Validity of AVA 4.1
AVA 4.1 validation was carried out in two steps: 1) Measurement of agreement between AVA 3.2 and AVA 4.1 analyses on all three datasets, 2) The ability for both AVA 3.2 manual analysis and AVA 4.1 automated analysis to discriminate between two established microcirculatory states: pre- and post-induction of general anesthesia.
Datasets P1 and P2 had previously been analyzed using AVA 3.2. The appropriate calibration measures for the retrospective datasets were obtained from the original AVA 3.2 analysis reports and were entered into AVA 4.1 settings prior to automated analysis.
V1 was analyzed with AVA 3.2 using the aforementioned validated referent methodology. All three datasets were then analyzed using the automated software, AVA 4.1. The measurements from both the manual and automated software packages for PVD, TVD, PPV were recorded for all datasets.
Statistical Analysis
Statistical analyses were completed using R statistical analysis software31or Microsoft Excel (2019). Significance level was set a priori at p = 0.05. Raw data and analysis scripts are available upon reasonable request.
All variables from AVA 3.2 and AVA 4.1 were evaluated using histograms and Kolomogorov-Smirnov tests were used to confirm normal distributions. Homoscedasticity was tested using F-tests, with the null hypothesis that variance did not differ between comparison groups (e.g. comparing PVD before and after induction of general anesthesia in dataset P1). For groups meeting normality and homoscedasticity assumptions, paired t-tests were used to compare microvascular variables preceding and immediately following induction of general anesthesia in the P1 and P2 datasets.
To compare the level of agreement between AVA 4.1 and 3.2, the intraclass correlation coefficient (ICC) was calculated using a two-way analysis of variance (ANOVA). The ICC for each variable, TVD, PVD, and PPD were calculated separately. All ICC values are reported along with the 95% confidence intervals. ICC values below 0.40 are considered as “poor”, between 0.40 and 0.59 as “fair”, between 0.60 and 0.74 as “good” and greater than 0.74 as “excellent”.32
Additionally, the method proposed by Bland and Altman was used to assess the agreement between the two methods of analysis.33 For each microcirculatory variable, a Bland-Altman plot shows the difference between measurements taken by AVA 3.2 and AVA 4.1 for each video versus the average of the measurements taken by the two methods, along with limits of agreement. The agreement between the two methods is summarized by calculating the mean difference between the two methods along with 95% confidence interval. The limits of agreement (LOA) further extend the confidence interval for the mean difference to account for sampling error.