Learning high-order geometric flow based on the level set method

Recently, the development of deep learning has accomplished unbelievable success in many fields, especially in scientific computational fields. And almost all computational problems and physical phenomena can be described by partial differential equations. In this work, we proposed two potential high-order geometric flows. Motivation by the physical-information neural networks and the traditional level set method (LSM), we have integrated deep neural networks and LSM to make the proposed method more robust and efficient. Also, to test the sensitivity of the system to different input data, we set up three sets of initial conditions to test the model. Furthermore, numerical experiments on different input data are implemented to demonstrate the effectiveness and superiority of the proposed models compared to the state-of-the-art approach.


Introduction
Recently, the development of artificial intelligence (AI) has received comprehensive attention and assistance through the breakthrough of deep learning (DL) technology. DL has accomplished unbelievable success in many fields, especially in scientific computational fields, including computer vision, medical imaging, and control problem [1][2][3][4][5][6].
Fortunately, almost all computational problems and physical phenomena can be described by differential equations, especially geometric partial differential equations (GPDEs). Such multi-phase flows play an increasing role in several scientific and engineering applications [7][8][9]. Also, GPDEs has essential application in computer vision, such as Liang et al. [10] used PDE for facial image analysis. Moreover, Chen et al. [11] developed geodesic paths for image segmentation. In addition, PDEs are widely used in medical image analysis. For example, Chen and Amini [12] developed the geometric deformable models for MRA images. Also, Salinas and Fernández [13] used PDE approaches for medical image enhancement. Furthermore, Karasev et al. [14] proposed an interactive PDE control of active contours for medical image segmentation. Besides, PDE has important applications in the field of control. Such as Biroon et al. [15] developed a PDE approach for real-time detection and isolation. Moreover, a robust cooperative output regulation for a network of parabolic PDE systems was proposed by Deutscher [16]. Furthermore, Song et al. [17] used the PDE systems for fuzzy event-triggered control. More references about the applications of PDE in control are shown [18,19].
The PDEs can be solved analytically and numerically. However, so far, the analytical solutions of many PDEs have not been solved. Therefore, in terms of PDEs that have not been analytically solved, we can only understand the physical phenomenon described by them from the perspective of the numerical solution. Usually, the numerical approaches are used to discretized the solution domain and construct algebraic equations, after that, they are solved analytically or iterative. The traditional numerical methods include the finite volume method (FVM), finite difference method (FDM), finite element method (FEM), and so on. The computational cost of the equations becomes extremely expensive when the number of equations increases. Moreover, the GPDEs are generally defined on the surface (manifold), which can fall into the curse of dimensionality. Furthermore, GPDEs solutions can be distinctively different, and there is no general approach that applies to all kinds of GPDEs.
Happily, since the universal approximation theorem (UAT), which is the fundamental theoretical basis of DL. And it opens another door for the numerical solution of GPDEs. According to the UAT, the complex or even dynamical GPDEs can be approximated via a DNN. To find the solutions, the DNN is trained on the solution domain of the GPDEs. Moreover, a surface (manifold) can be implicitly represented by a level set method (LSM), which was first introduced by Osher and Sethian [20], and has a significant impact on the computational field. Currently, DNN has applied successfully for solving PDEs, such as hidden physics models [21] and physics-informed neural networks (PINNs) [22].
In this work, motivated by hidden physics models, PINNs, and LSM, a robust deep LSM learning of the data-driven high-order GPDEs method is proposed. Our contributions are summarized as follows: -We focus on the data-driven and learning methods to solve two GPDEs: (1) Quasi Xuguo flow [23], (2). High-order surface diffusion flow of Cahn-Hilliard model. -Theoretically, our framework is flexible to adapt to different high-order GPDEs, which enables effective integration of traditional LSM and DNN to improve computational efficiency. Our experiments confirm this property.
The rest of the paper is organized as follows, in Sect. 2, we first present some related works about learning highorder geometric flow. Furthermore, we introduce the proposed algorithm about learning high-order geometric flow based on the level set method in Sect. 3. After that, in Sect. 4, we describe the experimental settings and experimental results. Finally, we summarize this paper and present the limitations of the current study, and present several future research directions in Sect. 5.

Related work
In this section, we briefly review the most relevant studies from the following three aspects: (1) The manifold learning and some definitions, (2) The level set method, and (3) The deep learning-based approach for solving PDEs.

Manifold learning and some definitions
Manifold learning is a method for nonlinear dimension reduction. Algorithms for this task think that the dimension of several data sets is only artificially high. Set S = u (x, y) ∈ R 3 : (x, y) ∈ D ∈ R 2 be a sufficiently smooth, regular and the parametric surface. And let g = u x , u y be the coefficients of the first and second fundamental forms of surfaces with u x = ∂u ∂ x , u y = ∂u ∂ y , ∂ 2 u ∂ x∂ y = u xy , where ·, · refers to the usual inner product in Euclidean space R 3 . Definition 1 (Tangential gradient operator). Suppose |∇φ| = 0 on some open neighborhood Ω of the level set = {(x, y) : φ (x, y, t) = 0}, f is a differentiable function on Ω, then the tangential gradient operator ∇ S acting on f is given by ∇ S f = P∇ f , where P = I − n ⊗ n = I − nn T is the projection operator to the tangential plane of the surface , n = ∇φ |∇φ| , and I refers to the identify mapping.
where div is the usual divergence operator.  φ (x, y, t) = 0}, f is twice differentiable function on Ω, then the Laplace-Beltrami operator Δ S acting on f is given by Recently, manifold learning has made great progress. For example, Bachmann et al. [1] developed constant curvature graph convolutional networks (GNN), bridging the gap that popular GNNs consider the data only via Euclidean geometry and associated vector space operations. Also, Sanchez-Gonzalez et al. [3] used the GNN to simulate complex physical phenomenon. Moreover, Chen et al. [43] utilized the convolutional kernel networks for learning the graph-structured data.

Level set method
The LSM was first developed by Osher and Sethian [20], which made a huge impact on computational methods for interface motion. Since the surface manifold can be implicitly represented by the LSM, therefore, which is widely used in various fields including computational geometry, fluid mechanics, computer vision, and materials science [49]. Moreover, Fedkiw et al. [50] used the LSM for representing the dynamic implicit surfaces. More recently, Lin et al. [51] developed the LSM for solving constrained convex optimization. More about the LSM please refer to [31,[52][53][54][55][56][57][58][59][60][61] and Table 2.

Deep learning-based approach for solving PDEs
Recently, with the rapid development of DL, the numerical methods of PDEs have made significant progress. Advantageously, as mesh-free approximators, compared with the traditional mesh-based numerical schemes include FVM, FDM, and FEM, the DNNs are inherently mesh-free function-approximators. Such that they not only can avoid the curse of dimensionality but also approximate the solutions of PDEs on complex geometries effectively. One remarkable application of DNNs is the physics-informed neural networks (PINNs) [22], which can solve both forward and inverse problems with the desired accuracy. Also, Sirignano et al. [62] developed a method called DGM for solving PDEs. Moreover, Long et al. [2] proposed PDE-Net for learning PDEs from data. For more examples on solving differential equations with DL, please refer to [21,41,44,[63][64][65][66][67][68] and Table 3.

High-order quasi Xuguo flow
In this section, we describe more details about the algorithm of learning high-order geometric flow based on the LSM. In [23], Xu and Zhang constructed the Quasi Xuguo flow from the perspective of computational geometry. However, the higher-order geometric flow is not solved analytically and numerically in that paper, according to [23] and LSM, the high-order GPDE is we obtained as follows, where Δ s refers to LBO, ∇φ ε = φ 2 x + φ 2 y + ε, κ = div ∇φ ∇φ ε refers to the mean curvature (MC), and ε > 0. According to [69], take κ as an example, the LBO can be explicitly formulated as following, (2) To obtain the order reduction of GPDEs, the new variables q = Δ s κ, w = Δ s q are introduced. Hence, Δ s q and Δ s w can be explicitly formulated via the same approach as Δ s κ. Therefore, Eq. (1) can be reformulated as follows, Motivated by PINNs, the GPDEs are encoded into the loss function, and the partial derivatives can be computational via automatic differentiation (AD). For convenience, the following symbols are introduced as, Based on the analysis above, then the total loss function is obtained as follows, where I C and BC refer to the initial and bound conditions, respectively. In this work, the initial conditions are discussed in more detail later, and the boundary condition is the periodic boundary one. Usually, the systems of PDEs are viewed as the PDEconstrained optimization problem, where Θ is the set of model parameters, and L is the loss function. When we replace Θ with N N Θ , the following physics-constraint optimization learning one is obtained, The gradient descent approach can be used for solving it, and the parameter update equation is, where L refers to Eq. (6). The algorithm is summarized as in Algorithm 1, and the schematic illustration of our LSM-physics constrained learning (PCL) framework for learning GPDE is shown in Fig. 1.

High-order surface diffusion flow of Cahn-Hilliard model
Such as [7], we want to approximate the surface diffusion flow using LSM and high-order LBO of κ, to smoothing the system of PDEs, the Heaviside function is introduced as, where ε is a small positive constant. Then, the proposed new Cahn-Hilliard model reads as, Since the Definition 1-2, we can get a smooth vector field v defined on Ω, and the divergence operator div S acting on v, which has the following explicit representation, x v x +φ x φ y (vx +v y )+φ 2 Algorithm 1: The framework of learning highorder geometric flow based on the LSM.

Experiments
To test the robustness of the flows, we add different degrees of Gaussian noise to the initial conditions, and the noise levels are 0.3, 0.5, and 0.9, respectively. Furthermore, the metrics are shown as follows, 1. Peak signal-to-noise ratio (PSNR) [75] is an engineering term for the ratio between the max-   imum possible power of a signal and the power of corrupting noise that affects the fidelity of its representation. PSNR u * ,û = 10log 10 255 2 mn u * −û 2 2 , here, u * ∈ R m×n is the clean solution image, u ∈ R m×n is the learned solution image, and u ∈ R m×n is the noisy data.
2. The structural similarity index measure (SSIM) [75] is a method for predicting the perceived quality of digital television and cinematic pictures, as well as other kinds of digital images and videos. SSIM is used for measuring the similarity between two images. SSIM measures the two patches, x   and y correspond to the same spatial window of the images X and Y , respectively. The SSIM value for the patches x and y is given as . μ x refers to the mean of x. σ x refers to the standard deviation of x. σ xy is the cross-correlation of the mean shifted images x−μ x and y−μ y , and the C i , (i = 1, 2) are small positive constants. Therefore, SSIM is used to measure the similarity between two images of learned solutions.

Normalized Root Mean Square Error (NRMSE)),
given a ground-truth image y(x) and a generatedŷ (x), NRMSE can be defined as follow, In this section, we describe more details about the experimental setup and results while test the performance of our method. We train the deep neural network models on our equipment with a GeForce RTX 1080 super GPU. The software is developed based on the PyTorch framework [76].  [74], DnCNN [73], CBDNet [70], RDNet [71], and VDNet [72] to test the robustness of our framework for various networks. We choose 50 boundary sampling points, 50,000 inner sampling points, and 5000 initial sampling points for training. Examples of the visualization process of evolutionary for high-order flows with initial conditions are shown in Figs. 2, 3 and 4. And the training loss under different situations is shown in Fig. 5. Also, the comparison of the training loss values and times for high-order flows under different initial conditions are shown in Tables 6  and 7. The initial conditions of u are given in Table  8. Furthermore, comparison results of SSIM, PSNR, and NRMSE for high-order flows under different initial conditions (FNN) is shown in Table 4-5. To study the influence of various optimizers on the algorithm system, we used the following optimizers to test the system, including Adadelta [77], Adagrad [78], Adam [79], and RMSprop [80]. And the training loss under different situations is shown in (f) of Fig. 5. We found that, except for Adadelta, other optimizers converge very quickly.
About parameter settings, it should be noted here that during the training process, the initial condition weighting parameter is 100. Moreover, the boundary value condition and PDE-loss weighting parameters are 1. In all experiments, ε in the Heaviside function is 0.5, and ε in the initial condition 3 is 0.01. Furthermore, the learning rate is 0.001, γ in Eq. (11) is 0.1, and the number of iterations is 1000.

Conclusion
In this work, we explore the problem of high-order Quasi Xuguo flow and high-order surface diffusion CH flow with different initial conditions based on deep LSM. Also, we use different initial conditions that aim to study the sensitivity of the algorithm on different input data. Moreover, to verify the robustness of the algorithm, we used two networks for testing our algorithm. Theoretically, almost all networks fit our framework. Besides, through the combination of the traditional LSM and DNN, such that our framework becomes a powerful tool for solving high-order GPDEs. Furthermore, we study the influence of different optimizers on the learning results and convergence of the algorithm. And these optimization meth-ods mainly include Adadelta [77], Adagrad [78], Adam [79], and RMSprop [80]. The test results show that Adam is more suitable for our framework. In future work, we will use real-world data to test our framework and solve practical problems in computer vision such as denoising, inpainting, reconstruction, and segmentation problem.