Combination of Digital Image Processing and Statistical Data Segmentation to Enhance SPR and SPRi Sensor Responses

The work reports the combination of basic digital image processing (DIP) techniques and statistical segmentation strategy (SDS) to improve surface plasmon resonance curve (SPRc) and SPR imaging (SPRi) sensors' performance. The SPR image is used for sensing and monitoring biological events in the so-called SPR imaging process. In the traditional SPR process based on the attenuated total reflection (ATR) method, the image is used to create the SPR curve, and the curve features tracking is employed on sensing applications. The SPR curve features are enhanced after the pixels of the SPR image have been processed with low-complexity filters in the spatial domain (brightness, contrast, threshold, and morphological). The bootstrap was used as a statistical processing approach, selecting lines and columns from the image that was less affected by imperfections and noises in the image detector, and consequently reducing the SPR sensor instrumentation disturbances. Experimental tests with reversible binding water-mixture were performed, and both image and statistical processing were reported. The combination of DIP and SDS approaches improves the extraction of the curve features, increasing the performance in terms of resonance position sensitivity to 81%.


Introduction
Surface plasmon resonance (SPR) is a promising physical phenomenon for the development of instruments and tools to be applied for sensing biological events. Extremely sensitive to refractive index variations of the substances under analysis [1,2], the SPR sensors monitor the resonance position through the minimum reflectivity value obtained when illuminating a multilayer structure. Typically, the attenuated total reflection (ATR) configuration is employed [1,2], with a multilayer composed of a thin metal slab between two dielectric mediums, one being an optically transmissive material and the other being an analyzed substance [3].
A 1D or 2D image detector captures the reflected light after going through the multilayer. The generated image is used directly for sensing in the so-called SPR imaging process (SPRi) [4,5] or is graphically constructed previously before sensing, forming the SPR curve (SPRc). In both cases, the quality of the measurement is related to the image quality provided by the SPR sensor. As an example, 2D image samples with and without SPR excitation are present in Fig. 1. In the resonance condition, a low energy region is noticed and indicates the resonance position according to the substance refractive index. The SPRc and SPRi sensors track this position/intensity for sensing proceedings.
The imperfections and noise of the experimental set-up of the SPR sensor are present in the generated image. As indicated in Fig. 1c, scratches (A), stains and shading (B), mechanical misalignment (C), false minimum/resonance (D), and non-uniform response € deform the SPR image/ SPR curve, causing detection of the resonance position difficult. In addition, the arising noises during the SPR sensor usage, related to the processes of transformation, generation, conversion, amplification, and digitization of incident light are also expressed in the SPR image. Brightness, contrast, and sharpness issues are commonly noticed in SPR images.
Algorithms to improve SPRc data extraction [6], curve characteristics enhancement [7], and for better selection of the region-of-interest (ROI) in SPRi [8,9] are early reports. Attempts to model and remove image sensor noise [10], with machine learning approach [11], or based on transformation domain operation in SPR images for intensity processing [12] and noise removal [13] can be found elsewhere. Transformation domain operation, with non-linear filters as notch, fan-sector, and specific point filters [14] can be also applied for SPR image enhancement. In addition, the subdivision/segmentation of the SPR image into multiple spots can mitigate the non-uniformity of the image detector since the imperfections are spatially distributed in the detector area. Improvements in the signal-to-noise ratio [7] and in the dynamic response of the SPR sensor [15] can be obtained. As illustrated in Fig. 1d, the image can be segmented into several spot-regions (Si) to obtain the best SPR curve/SPRi ROI. In general, this is a process performed manually [3,15] or through complex methods [9,16] that require elaborate calibration and parameter acquisition procedures for the segmentation model.
Here, we introduced a spatial domain digital image processing (DIP) combined with statistical data segmentation (SDS) approach for enhancing both SPRc and SPRi. Traditional enhanced, smooth, and morphological operators are applied in the pixel matrix of the image. Next, the bootstrap method [17] is applied in the processed image to resample the pixel distribution of the SPRi/SPRc ROI. The approach is demonstrated with reversible binding solution at angular interrogation mode SPR operation. The new sensor performance indicated 81% improvement in the resonance angle ( R ) shift measurement.

Experimental Report
The SPR sensor based on the well-known PPBIO prism [18,19] was used to obtain the SPR images. The experimental setup is shown in Fig. 2. The PPBIO has a trapezoidal structure, in which the light beams enter and exit parallel to the prism, avoiding rotational parts. The metallic layer is deposited directly on the prism, avoiding gel/oil for coupling. The PPBIO is capable of operating in the spectral interrogation (WIM) and angular (AIM) modes [19]. In AIM mode, the PPBIO provides an angular range of 62° to 73°, ideal for measuring aqueous substances. The sensor has a cylindrical lens LJ190L1 (for focusing) and a plano-convex spherical lens LA1951 (for collimating the light beams); the 670 ± 10 nm laser diode L670P005 and the two-dimensional CMOS camera DCC1645C with 1024 × 1280 pixels resolution, all commercialized by Thorlabs. The Acquisition, Control, Processing and Display (UACPD) unit embedded the combined DIP and SDS proposed approach. Furthermore, the geometrical arrangement (see Fig. 2c) provides the alignment of the optical coupling in relation to the sensible Fig. 1 Image a without and b with surface plasmon excitation and resonance position indication. Details for the image pixel representation (SPRi) and the corresponding SPR curve. c Examples of degradations/ imperfections in SPR image. d Logical spots regions obtained with the image segmentation areas (spot), each one corresponding to a physical portion on the metal-sensitive surface of the prism. The PPBIO concept provides a self-alignment of the spots, making possible parallel measurements with multispot technique (dispenser instrument and appropriate flow cell may be required). The multispot processing is composed of different sensitive areas, matched pair with its corresponded 2D image segmented region, and which indicate the amount of substances that can be simultaneously monitored by the SPR sensor. For more details of the PPBIO prism, we indicate [3,6,15,18,19].
The experiments have carried out with BK7 glass PPBIO (BK7 SCHOTT with 1.52 refractive index at 670 nm), covered with a thin 50-nm gold layer using the sputtering DC Cryofox Explorer 600LT from Polyteknik. The substances deionized and degassed water (H 2 O) with electrical resistance of 18.7 MΩ/cm obtained with the Milipore Direct Q3 system, and solutions of PBS (phosphate buffered saline) and hypochlorite (Hypo.) were used. The substances have the following refractive index (RIU): H 2 O = 1.33001, PBS = 1.33151, and Hypo = 1.34501.
The SPR images for each substance are shown in Fig. 3. Color or gray scale images can be used without compromising sensor performance. Substance with a higher refractive index shifts the resonance position to the left so that the minimum intensity of reflected light is located at the initial pixels. The pixel-angle relation [3] used for sensor calibration is θ(p) = − 0.007p + 72.765, where p is the pixel position.

Spatial Domain Processing of SPR Image
The raw SPR images were processed with methods for adjusting brightness and contrast, segmentation by a threshold, and noise/imperfections smooth with erosion and dilation morphology methods. References in the literature for the mathematical formulations of these techniques and simple implementation algorithms can be found elsewhere [14]. Figure 4 presents the DIP application on the images of each substance and their respective SPR curves. Visual inspection, SPR curve depth, and minimum position were evaluated. Corrections differences in intensity level (brightness) and between the upper and lower levels (contrast) are necessary for compatibility between light source power (mA), camera integration time (ms) and ambient light (nW or MW). In general, SPR images have low average intensity (dark images) due to low power applied in order to avoid noise and fluctuations in the laser/LED output [20]. Thus, the brightness and contrast operations aim to increase the intensity of the image's pixels.
Segmentation threshold was applied above to the pixel average intensity, in opposite processing to brightness and contrast. Despite visually enhancing, the resonance region and the depth of the SPR curve did not change. Furthermore, morphological processing was evaluated. Erosion reduces or sharpens objects in an image, whereby details smaller than the structuring element are filtered out (removed) from the image. Thus, pixel-area shrink in size, and holes within those areas (resonance

Statistical SPR Image/Data Segmentation
The enhanced SPR image with the erosion method is then segmented into several regions (Si) to obtain the best SPR curve. An image segmentation process can be used to remove degradations/imperfections, and to perform a Erosion processing gives the highest SPR curve dip, and it does not disturb the curve features such as width, asymmetry, and slope [5] multisport sensing strategy [3,15]. The statistical segmentation allows a reorganization of pixels to obtain a better dark region/dip of the resonance position. The bootstrap statistical method [17] was used in SDS. The basic idea of this method is to estimate the desired characteristics for the SPR curve, allowing to approximate the desired pixel distribution of the image by an empirical distribution based on a finite-size sample (B). The sampling is performed with replacement [21]. As a computational technique that estimates values for population parameters, the bootstrap allows flexibility, speed, and fewer assumptions in determining of the estimates. In addition, it is able to be applied to any type of estimator, independently of its generating rule, enabling the estimation of bias, prediction errors and confidence intervals. Here, confidence intervals within the region of interest are generated, allowing inferences (significance tests), and information extraction about minimum points in the images of each analyzed substance.
The bootstrap procedure is synthesized in Fig. 5 flowchart. First, for a given input data set with n values (SPR image or SPR curve pixels), the method generates B samples, with the same size n, for each one with replacement. Next, the interest measure m, mean or median for example, is computed for all samples.
These computed values are sorted. At this point, bootstrap has generated B sorted values (m 1 , m 2 ,…,m B ) for the interesting measure. Lastly, given a confidence error α, the boundaries of the confidence interval are determined by the values at the positions B (α/2) and B (1-α/2) in the sorted set generated.  [17,21]. As illustrated in Fig. 6a, the α confidence interval is defined in which the upper (SUP) and lower (INF) limits delimit the percentiles of the bootstrap estimate that will be used to represent the new SPR curve. Here, the SDS was investigated for 0.01, 0.05, and 0.1 intervals in samples of sizes 2000, 4000, and 8000. Figure 6b shows the resonance position ( R ) extracted from the new SPR curves obtained with the SDS. For the pair α = 0.05 and B = 2000, it is possible to obtain the greatest variation in the resonant position among the analyzed substances (see Fig. 6c). Table 1 summarizes resonance angles and sensitivity values for the DIP and SDS combined application. Manual segmentation in two (512 lines) and four (256 lines, see Fig. 1d) were performed to compare with the bootstrap. The range of refractive index (RI) between H 2 O → PBS is 1.35RIU, and 1.5RIU between H 2 O → Hypo. This corresponds to an angle variation of 0.98° and 1.71°, respectively, for the DIP (erosion) + SDS (bootstrap: α = 0.05 and B = 2000) processed images. The values of angular sensitivity S , expressed as the ratio of resonance angle variation ( Δ R ) by the refractive index changes ( ΔRI ), i.e., S = Δ R ∕ΔRI , reaches 390°/RIU for the proposed approach. The performance obtained with the DPI + SDS approach increases 28% related to manual segmentation of ¼ of the SPR image, and 81% to the raw image.
The proposed method can be able to work with parallel data as well. For this, two different scenarios can be considered: (1) the manipulated image has different regions, which are independent and each one can have different substances, at the PPBIO multispot processing. In this case, each region can be treated separately and simultaneously; (2) the image refers to only one substance but it is too large, or logically segmented to serve as a calibration/reference signal, and it needs parallelism to be processed. In this case, the image can be also separated into different regions, which are processed independently. In the end, statistical methods for the combination of confidence intervals, for the same population, can be used to provide a final result.

Conclusions
An approach for processing the response of SPR and SPRi sensors based on spatial domain processing of the SPR image, combined with the statistical segmentation of image pixel values was presented. The dip accentuation of the SPRc and visual aspects of SPRi was obtained with the application of a morphological erosion filter. Erosion thins objects in an image, accentuating the region relative to the resonance position. However, with this fine-tuning, a displacement of the resonance position is noted after the operation. A statistical segmentation was performed using the bootstrap method. The bootstrap acts in a reorganization of the SPR image pixels to obtain a better SPR curve/ image, avoiding manual segmentation. The resampling with parameters α = 0.05; B = 2000 showed a significant improvement in the sensitivity value, attesting to the good performance of the proposed DPI + SDS approach.
Author Contribution All authors contributed to the study conception and design. Gleice M. de Medeiros: software development, investigation, data visualization, and validation. Leandro C. Souza: development, reviewing and editing, SDS specialist. Daniel F.L. de Souza: development, reviewing and editing, DIP specialist. Leiva C. Oliveira: supervision, conceptualization, development, writing -original draft, writing -review and editing, SPR specialist. All authors commented on previous versions of the manuscript and read and approved the final manuscript.