Mathematical engineering modeling and optimization of convolutional neural networks for common data areas in multitasking systems

doi:10.21203/rs.3.rs-3138131/v1

Download PDF

Research Article

Mathematical engineering modeling and optimization of convolutional neural networks for common data areas in multitasking systems

https://doi.org/10.21203/rs.3.rs-3138131/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

This paper presents an in-depth study and analysis of modeling optimization in mathematical engineering for common data areas in multitasking systems using convolutional neural networks. A study of multi-convolutional neural network task computation is developed, and a multi-convolutional neural network task switching architecture is designed based on FPGA. To address the phenomenon of failure correlation among different functional types of components in complex systems, a basic model of the functionally related voting system is defined, and the component functional dependence mechanism is used to guide the construction of the system's dynamic Bayesian network topology and the generation of node conditional probability tables. On this basis, the state probability distribution of system nodes is calculated based on dynamic Bayesian network inference to realize the dynamic assessment of system reliability with online state data. The results of the algorithm show that the method can effectively realize both function-related voting system reliability modeling and dynamic updating of the complex system reliability index using online state data. Meanwhile, the method applies a deep learning technique from a 3D tree model library to learn the 2-dimensional shape of 3D shape mapping, which results in more natural and reliable depth information. Then, a complete 3D tree model is generated by combining a procedural tree modeling approach under the constraint of 3D shapes, where 3D tree models with different levels of detail can be created by different semantic strokes. Finally, this study experimentally verifies the efficiency and effectiveness of the method in terms of recovering 3D tree models from a single image.

In recent years, convolutional neural networks have made remarkable achievements in image recognition, speech recognition, object recognition, and other fields with their excellent recognition ability, which is driving the rapid development of artificial intelligence and gradually bringing intelligent application products into public life. With deeper research on deep learning and the demand for increasingly complex work tasks, the scale of the convolutional neural network is increasing, and the amount of data and computation are similarly increasing dramatically [1]. A convolutional neural network is a feed-forward network with no data feedback between layers, which are independent of each other, and the computation within layers follows a highly parallel structure. The commonly used CPU is a computing architecture designed to perform logical processing, and the computational functions are implemented by software; however, this approach is not suitable for tapping the parallelism of convolutional neural networks and therefore cannot meet the requirements of the application scenarios in terms of power consumption and speed. Moreover, in practical application scenarios, there are often multiple convolutional neural networks or multiple convolutional neural network tasks that need to be processed at the same time, such as a situation in which a convolutional neural network recognition task has not been computed but there is another recognition task that needs to be computed [2]. Due to the limitation of computing resources and bandwidth resources, the hardware system cannot respond to multiple neural network task requests in parallel, and the tasks can only be computed serially in sequential order, which makes the system's real-time flexibility very poor.

3D visualization is the process of creating graphics and rendering designs using 3D software. In short, the data of various industries are presented in three dimensions through 3D visual effects. With the rapid development of global technology, in many fields, two-dimensional flat displays can no longer meet the needs of users of visual scenes [3]. Three-dimensional visualization is widely used and plays an important role in games, film and television, animation, architecture, industrial manufacturing, medicine and other fields. Along with social development and construction needs, geotechnical engineering and construction urgently need to transform two-dimensional plane data into three-dimensional visualization data; therefore, the study of three-dimensional modeling visualization technology for tunnels, pits, and other projects has become an important element. With the rapid development of science and technology, a single simple design model in the past has been unable to address the increasingly complex product design needs, and it is unable to effectively organize the complex relationship between the various subsystems and components in the system to express their mutual relationship [4]. Such a rough design pattern can easily generate considerable useless self-iterative development, which leads to higher costs and longer production cycles. Therefore, to address the design needs of such complex products, a mature methodology of systems engineering has been used for product design.

System reliability models characterize the statistical laws of the system state, which evolve over time during service and are the cornerstone of reliability analysis and health management in complex systems. When building a system reliability model, engineers usually calculate the reliability indices of each bottom-level unit within the system first and then calculate the system reliability indices based on the state mapping relationships between the system and its component units at each level. In the case of a reliability block diagram in which the bottom unit of the system is a component, for example, a component reliability model that reveals the random evolution of the component’s state over time is constructed first, and the model parameters can often be provided directly by experts or estimated based on a batch of degradation test or accelerated degradation test data from the same type of component. Component reliability model parameter estimation methods have been widely studied in the field of reliability. Most of the past product designs exist in the form of drawings or models, which are usually stored in databases and provided for retrieval, and these knowledge bases and design repositories contain a large amount of mineral data that has not been used effectively before. Machine learning and data mining can be used to discover the characteristics of the design data from the data itself to guide a new round of design.

Xu's [5] white paper states that inefficiencies, errors, and delays account for almost one-third of total annual construction spending in the U.S. Designing with BIM can predict potential errors and inefficiencies and adjust designs promptly to reduce the risk of project failure and loss, and a survey shows that 48% of U.S. construction offices already use BIM methods. In accord with the development trend of today's construction industry and the progress of technology, the construction industry has realized the whole BIM application process in the actual construction process. To achieve the interoperability of BIM technology and software and a parametric model of information expression, an extended BIM technology using Industry Foundation Classes (IFC) files was used to achieve the design phase of different software adaptations. Guo [6] constructed a BIM-based parametric prototype system for calculating the life-cycle carbon emissions of industrial buildings, and Yu [7] built a museum-shaped roof based on a Rhino software parametric model for rapid positioning of detailed components. Liang et al. [8] also applied innovative fabrication and assembly design concepts to the parametric modeling process of prefabricated buildings, and Dhillon [9] digitally designed the geometry of large-span building surfaces and optimized the model to reduce the number of iterations in the schematic design and improve the design quality of the schematic drawings. Dubey [10] conducted a parametric analysis of the model simulation for BIM building design, the linkage capability of different components, and the synergistic relationship between them and concluded that parametric analysis and design play a crucial role in the design of complex building forms, scheme design, and engineering management.

In the case of constructing 3D models from point clouds without any additional information (e.g., point normals), a Delaunay-based interpolation reconstruction algorithm is usually used; the basic idea behind this approach is that the reconstructed 3D surface is formed by a variant of the Delaunay triangulation profile [11]. The advantage of this method is that if the input surfaces are sampled densely enough, they have provable quality assurance in terms of the geometry and topology of the reconstruction [12]. However, these requirements may be too stringent for point clouds collected from actual scenes, and therefore, it is impractical to use these methods for point clouds with large data defects obtained from scanning. Another important class of methods for constructing 3D surfaces from point clouds is usually based on algorithms with implicit surface functions, where the input is a normal point cloud [13]. This type of model is a typical model developed based on the concept of data-driven modeling, which has unique advantages in terms of addressing complex problems, but this type of model often depends on the number and quality of samples as well as the solution process. The results are often unstable and unsatisfactory, while the deep learning network developed in recent years seems to be expected to break through. Another example of this type is the support vector machine model, which is a machine developed based on statistical learning theory [14]. Compared with neural network models, the generalization performance of such models has been greatly improved, but the selection of model parameters is still a problem to be studied.

After the above introduction, it can be seen that the current acceleration research around hardware platforms mainly focuses on the optimization and implementation of a particular convolutional neural network model or a specific computing layer so that the convolutional neural network can complete the computation with lower power consumption and faster speed and does not consider the situation in which multiple convolutional neural network tasks or multiple convolutional neural network models are deployed on hardware platforms for computation. In practical application scenarios, it is often the case that one convolutional neural network recognition task has not yet finished its computation when another recognition task initiates a computation request. For a hardware platform with limited resources, it is difficult to respond to the computational requests of multiple tasks in parallel. In this paper, we investigate how to compute and switch between multiple convolutional neural network tasks on an FPGA platform as well as how to use hardware control to improve the real-time performance and flexibility of the system.

3.1 Data convolutional neural network algorithm design

The convolutional layer consumes the majority of the computational time and computing resources, so the feature analysis of convolutional computation is an important research element for accelerating convolutional neural networks. The essence of convolutional computation is multiplicative accumulation operations, and thus, parallel computation can be achieved. In the convolutional layer, multiple input feature maps are often computed with multiple convolutional kernels to obtain multiple output feature maps, so there is a possibility of multiple parallel computations.

When the parallelism of convolutional computation is n, it means that n multiplication accumulation computations can be performed in a single clock cycle [15]. Figure 1 shows the structure of parallel computation inside 3 convolution windows, where the computation of the convolution kernel and the corresponding elements in the convolution window can be realized in one clock cycle. In addition, $X_{n}^{i}\left( {i \in N} \right)$ denotes the neurons in the nth row and column i of the input feature map, and ${W_0}\sim {W_8}$ denotes the weights of the corresponding points in the convolution kernel. The neurons in the nth row are computed to obtain ${X_n},0 \times {W_0},1 \times {W_1},2 \times {W_2}$, and this process is then repeated for the other rows. The computational results obtained from all rows are summed to obtain the partial sum of a single convolutional window, which realizes parallel computation within the convolutional window.

The neurons on a certain output feature map all come from the same convolutional kernel with the computed output of the input feature map. Therefore, until all of the input feature maps are traversed in a convolutional window, the computation of the required weight parameters is constant, and the new weight data are read when the corresponding convolution of that convolutional kernel has been computed. Suppose the convolutional kernel size of a certain neural network is l×l, and each clock cycle requires l new input neuron data to be loaded. When the convolution kernel completes all of the computations, it needs to reload l×l weights. A single neuron or weight data needs b bytes to be stored, the number of input feature map and convolution window computations is $k * k$, the number of input feature map channels is m, and the number of output feature maps is . The data to be read for a particular input feature map convolution computation are as follows:

$$b=l(k * k) - {k^2}$$

Convolutional computation produces intermediate computation results that are often stored in on-chip storage space, and this cache space only needs to be sufficient to store the intermediate computation results produced by one input feature map. When the size of the parallel computation structure is the same as the size of the convolutional core, the scheme of parallel computation inside the convolutional window is most effective in speeding up. However, when the FPGA hardware resources are insufficient or there are multiple sizes of convolutional kernels in the neural network model, the parallel computation structure and the size of the convolutional kernels cannot be the same, and in this case, the computational efficiency decreases [16]. When computing a certain output feature map, all of the input feature maps correspond to the same convolutional kernel, so only l×l weight parameters need to be loaded alternately after completing the corresponding computation; in addition, a small amount of on-chip storage space is needed to save the weight parameters used in the current computation, so the on-chip storage resource occupation rate is lower for the method of internal parallelism in the convolutional window.

$$\frac{{\partial \sigma (x)}}{{\partial x}}=\sigma (x)(1+\sigma (x))$$

Although parallel computations are performed between convolutional windows at different locations on the input feature map, there is partial overlap of adjacent convolutional windows, which means that the input neuron data from the overlapped part can be reused in the computation of different convolutional windows. To reuse the input neurons, an additional data reading module needs to be designed with corresponding control logic in the control module, but it will reduce the off-chip reading operation of the data. A data cache copy is added to the module that stores the input neuron data to retain the data to be reused. When the neuron data need to be read, a multiplexer (MUX) is used for data selection control to provide the data to different computation units. Usually, when the convolutional neural network computes to the later layers, the size of the input feature map decreases continuously, the number of convolutional windows that can be computed in parallel on the same input feature map decreases, and the number of convolutional windows cannot always be integer-divided by the parallelism. At this time, the acceleration efficiency decreases, and the parallel computation structure is wasted.

$$coth(x)=\frac{{{e^{ - x}}+{e^x}}}{{{e^x} - {e^{ - x}}}}$$

In practical application scenarios, there are multiple concurrent requests from hardware systems for inference of convolutional neural networks, which may be computations of the same convolutional neural network model or multiple convolutional neural network models [17]. In this case, FPGA platform acceleration not only considers how to accelerate the inference process of neural network models through parallel computing but also how to schedule the task computation when multiple tasks are concurrent. According to the previous section, the FPGA bandwidth and computational resources are limited, so if multiple convolutional neural network tasks are computed in parallel, the resource requirements increase exponentially, especially the bandwidth resources.

$$x_{j}^{l}=f\left( {\sum\limits_{{i \in {M_j}}} {x_{i}^{k} * l_{{ij}}^{k}} - b_{l}^{{ij}}} \right)$$

The CPU is responsible for receiving the tasks, configuring the task sequence, and writing the task inputs to off-chip storage. In a conventional "CPU + FPGA" acceleration architecture, the CPU needs to be involved in the entire computational process of the convolutional neural network, i.e., both computation and data access need to be performed under the control of the CPU. Although logic control is the strong point of the CPU, the inference of convolutional neural networks is highly repetitive, and the process is fixed; therefore, it is not difficult to implement the control process by using hardware, and the hardware is faster. In the architecture presented in this paper, the CPU only needs to perform one write operation for each task to be computed, and the rest of the work is left to the control module. In this way, the CPU does not need to be occupied all the time and can enter a low-power state or perform other tasks after completing task writing. The power consumption of the hardware is less than that of the CPU under the condition that the same control logic is implemented.

$$P{C_l}=\left( {F \times F \otimes FM_{l}^{2}} \right) \oplus F{M_l}$$

The size of the input feature map of the convolutional layer is $M \times M$, and the size of the convolutional kernel is $N \times N$, where and can be configured according to the needs of different convolutional layers with specific values provided by the task sequence module. Similarly, the pooling and fully connected layers also use similar configurable operational units. Considering that there is a data dependency between the operational layers of the convolutional neural network, i.e., the output of the nth layer is to be used as the input to the n + 1th layer, there is no computational overlap between different operational layers, so there is no conflict when the configurable operational units are reused.

FPGAs, as general-purpose logic devices, do not have many advantages over ASICs in terms of storage bandwidth resources. FPGAs with high bandwidth resources are expensive, and the large number of off-chip data accesses means increased power consumption. Therefore, when designing computing units, it is important to minimize the number of off-chip storage accesses due to data reuse and reduce the bandwidth requirements for off-chip storage accesses [18]. The parallel computation between input feature maps has the highest bandwidth demand, and the parallel computation between output feature maps has a high requirement for on-chip storage resources; however, the internal parallel design of the convolutional window has the highest data reuse rate and meets the design requirements of this study. Therefore, this study adopts the structure of parallel computation within the convolution window, i.e., the convolution operations of all corresponding points within the convolution window are computed simultaneously. When all of the convolution windows on one input feature map are calculated, new convolution kernel data are loaded for calculation; when all of the convolution kernels corresponding to one input feature map are calculated, new input feature maps are loaded for calculation until the calculation of all input feature maps is completed, as shown in Fig. 2.

With a multilayer autoencoder architecture, the same operations can be repeated in the encoding and decoding stages as needed. By stacking multiple hidden layers in the encoder and decoder, a deep autoencoder can be constructed. Deep autoencoders suffer from gradient disappearance as well as gradient explosion problems common to deep learning, and certain measures are required to mitigate these problems. Generative adversarial networks (GANs) and variational autoencoders (VAEs) are more often used in generative tasks than autoencoders. The GAN generates content from the input noise; however, when generating specific content, the entire distribution needs to be searched, and it is difficult to randomly select specific features (noise) for a generation.

The concept of generative models has a long history in machine learning research and is commonly used when modeling data with base-mate conditional probability density functions. Generative models are probabilistic models with joint probability distributions over observations and target values that have not been widely used before, and deep learning has driven the application of generative models in various scenarios. Deep learning is a data-driven technique, and the increase in the number of samples can introduce significant performance improvement, but the collection and labeling of the preliminary data require a great cost [19]. Therefore, learning reusable feature representations from many unlabeled datasets has become an important research direction. Deep learning algorithms involve optimization in many aspects, the most important of which is the neural network training process. Even a single neural network training problem requires considerable computational resources and a long training time, which incurs extremely high costs for the application of deep learning algorithms on the ground, so research on optimization algorithms for deep neural networks and lightweight neural network techniques is crucial. Optimization algorithms for deep model training differ from traditional optimization algorithms in several aspects, and machine learning usually acts indirectly. In most machine learning problems, the focus is on some performance measure P, which is defined on the test set and may be unsolvable, and therefore, P can only be optimized indirectly.

3.2 Mathematical engineering modeling and optimization analysis of public data in multitasking systems

The DBN can handle the information correlation of system state monitoring data in time and space dimensions, and the EM algorithm can solve the parameter estimation problem under incomplete state monitoring data. Therefore, the simultaneous application of the EM algorithm and DBN model to learn the system state mapping relationship is worthy of research. However, the existing EM algorithm is only suitable for estimating time-varying parameters, and the time-domain consistency of the parameter estimation results cannot be guaranteed when used for node CPT parameter estimation [20]. On the other hand, in the case of scarcity of available system state monitoring data, the state monitoring information at different moments is correlated. Blindly ignoring the above information correlation cannot make full use of the valid information in the monitoring data. Therefore, under the framework in which the DBN is the system reliability model, this section proposes a customized EM algorithm to ensure the time-domain consistency of the subsystem and system node CPT parameter estimation results by dividing the DBN topology into multiple V-shaped basic structures and integrating the monitoring information of the same V-shaped basic structures at different moments with the idea of parameter modularity.

The decomposition the DBN is a multiple V-shaped structure. For any node $X(t)$, the basic structure of itself and its parent node $X(t)$ is called a V-shaped structure. If all nodes in the DBN contain state data, the V-shaped structure of node $X(t)$ is independent of other nodes in the whole DBN. On this basis, the unknown parameters in the CPT of node $X(t)$ can be estimated based on the state data in the V-shaped structure. However, due to the incompleteness of the system state monitoring data, the above V-structure independence assumption no longer holds. To solve this problem, this algorithm fills incomplete state monitoring data into complete state monitoring data by considering the distribution expectation of nodes without state monitoring information as their monitoring information [21]. Notably, the CPT parameters of the variable nodes corresponding to the same object on different time slices should be the same. Therefore, this method integrates the state monitoring information embedded in the V-shaped structures of the same object at different time slices and then unifies the CPT parameters of the nodes corresponding to the same object to ensure the time-domain consistency of the parameter estimation results. Using the three-part system shown in Fig. 3 as an example, the CPT parameters of nodes 1S(t) and S(t) in the DBN of this system need to be estimated.

The unknown parameters are estimated using a customized EM algorithm under DBN parameter modularity. In the EM algorithm, the incomplete data are populated with complete data under each time slice using the distribution expectation, and on this basis, the unknown parameters in the CPT of the subsystem and system nodes are estimated. It is worth noting that the EM algorithm continuously updates the estimates of the unknown parameters through iterations of the E and M steps. The incomplete state monitoring data of the system are populated with the complete monitoring data at each time slice using the distribution expectation values. For the subsystem and system node that need to estimate the CPT parameters, the joint state probability distribution of this node and its parent node are calculated using Eq. (6), and the state monitoring information of the V-shaped structure under the corresponding nodes of the same object in different time slices is integrated.

$$E\left[ {N\left( {{S_m}=k,pa({S_m})=j} \right)\left| D \right.} \right]=\sum\limits_{{n=1}}^{N} {\sum\limits_{{t=0}}^{T} {\left\{ {{S_m}(t)=k,pa({S_m})=j} \right\}} }$$

$$\theta _{{m,j,k}}^{2}=\frac{{E\left[ {N\left( {{S_m}=k,pa({S_m})=j} \right)\left| D \right.} \right]}}{{\sum\limits_{{t=0}}^{T} {\left\{ {{S_m}(t)=k,pa({S_m})=j} \right\}} }}$$

The system state monitoring data are obtained based on the simulation of the real parameters of each node’s CPT. It is worth noting that to verify the generality of the proposed method in terms of learning both deterministic and probabilistic system state mapping relations, the real values of the parameter vectors of the subsystem and system node CPTs in the three-component system are randomly generated by using the Monte Carlo simulation method, and both deterministic and probabilistic state mapping relations are covered.

$$s(u,v)=\frac{{\left| {{V_u} \cup V_{u}^{\prime }} \right|}}{{\left| {{V_u} \cap V_{u}^{\prime }} \right|}}$$

$${p_{u,v}}=\frac{{\sum\nolimits_{{w \in W}} {s(u,v){r_{u,w}}} }}{{\sum\nolimits_{{w \in W}} {s(u,w){r_{v,w}}} }}$$

The original input to the modeling method is the field point cloud of the target area, so an online incremental point cloud acquisition subsystem is built first. The captured point cloud data are presented in the head-mounted display and overlaid on the physical world, and the user can directly perceive the data acquisition process, which provides a more convenient and intuitive data acquisition experience. In addition, the system does not capture data in real time but provides the initiative to the user to capture data. Whenever the user thinks he needs to capture data, he can capture the current frame with a simple gesture command, which can effectively avoid capturing useless data and wasting system computing power [22]. Although HoloLens 2 itself provides a certain amount of arithmetic support, a large amount of data processing is still more difficult, and it often manifests in heat and system lag.

The captured point cloud data are then transferred to the server side, and an iterative nearest-point-based point cloud alignment algorithm is executed to align the current point cloud with the historical point cloud to reduce the alignment error caused by pose estimation in the holographic lens. The density-based denoising algorithm is then used to remove the outlier points in the field point cloud to obtain a cleaner field point cloud, as shown in Fig. 4. Finally, the processed point cloud data are transferred to HoloLens 2 for presentation to the user.

The technique of extracting edges from 2D images is now relatively mature, but the edges extracted from 2D images usually contain many nongeometric edges, such as textured edges on the surface of objects or edges unique to a particular viewpoint [23]. We only want to extract the geometric edges of the scene and ignore the texture edges, so we analyze the edge information directly based on the 3D point cloud, specifically using an algorithm based on the angle and half-disk criterion to extract the cloud boundaries of the field points.

$$\xi _{p}^{n}=\left( {{Q^n},{Q^{n+1}}} \right)_{\Gamma }^{2}$$

$${\left( {X_{h}^{2},v_{h}^{2}} \right)_{{\Gamma _h}}}+\frac{2}{3}t\beta {M_p}\left( {{\nabla _{{\Gamma _h}}}W_{h}^{2},{\nabla _{{\Gamma _h}}}W_{h}^{3}} \right)={\left( {g_{h}^{n},v_{h}^{2}} \right)_{{\nabla _{{\Gamma _h}}}W_{h}^{2}}}$$

For fluid surfactant systems, the nonlinear coupling term poses a great obstacle to the system’s energy stability. Therefore, the proof of energy stability for second-order discrete formats is an unsolved problem. However, in numerical experiments, it has been verified that both formats satisfy unconditional energy stability. Common geometric shapes such as circles and rectangles are often found in manufactured objects, and drawing these shapes can have a positive effect on the modeling process. However, if a different type of shape is specified each time a stroke is drawn, it can block the fluidity of the modeling [24]. To reduce this problem, we use a neural network to predict the shape type of the strokes, parameterize the strokes to the specified type of geometry, and reconstruct the strokes.

Points are then sampled uniformly on all 3D contour lines, and these points are triangulated to create the mesh model. Finally, the user can optionally specify that one or both ends of the created mesh are hollow. In the specific implementation, to enhance user perception, the mesh model is updated in real time as the trajectory strokes are drawn and reported back to the user in the head-up display.

3.3 Experimental design

In the experiment, the 3D model was rendered with multiple views. Each 3D model was rendered with a projection about the Z-coordinate axis every 120 degrees, and 3 multiple views corresponding to the 3D model were obtained.

In the experiment, the learning rate of the conditional adversarial module is 0.0002, and the size of the batch normalization layer is set to 1 for instance normalization. The adaptive time estimation optimization algorithm of micro-batch stochastic gradient descent and momentum parameters is used. Both the dropout layer and the batch normalization layer are added to improve the module's generalization performance.

The activation function of each layer of the residual weighting module uses an exponential linear element activation function. The optimizer adopts an adaptive time estimation optimization algorithm, dropout is set to 0.8, weight attenuation is set to 5e-5, weighted weight loss function is set to 1e-2, and the center update rate is 0.9.

4.1 Data convolutional neural network algorithm results analysis

The control module detects that there is a task to be computed with higher priority in the task sequence, immediately controls the task switching, and saves the current intermediate computation results and the status of the interrupted task. Since the switching point is randomly distributed at any moment of the inference process of the convolutional neural network, it is called the arbitrary moment switching mechanism. As shown in Fig. 5, task 1 is an M-layer convolutional neural network, task 2 is an N-layer convolutional neural network, and the priority of task 2 is higher than that of task 1. Suppose that the inference calculation of task 1 is in progress in the computing module, and at a certain moment when the control module detects the existence of task 2, it immediately interrupts task 1, saves the intermediate calculation results, and switches to the inference calculation of task 2. When the calculation of task 2 is finished, the inference calculation of task 1 is resumed. The control module switches tasks as soon as it detects the presence of task 2, and the switching point can be any point in the inference process of task 1.

The advantage of the arbitrary moment switching task is that high-priority tasks can be responded to immediately, and the system has good real-time performance. However, since the switching point is random, if the switching process occurs in a hidden layer of the calculation process, the amount of data that the system needs to save is uncertain, and the calculation state is difficult to predict; therefore, the arbitrary moment switching mechanism is more difficult to implement in the control module. Moreover, the control part of this design is mainly implemented by hardware, and since complex control is not its strong point, the arbitrary moment switching mechanism is not suitable for the design requirements of this study, as shown in Fig. 5.

The absolute value of the deviation of the system reliability under the two failure criterion criteria tends to 0 as time increases, and the absolute value of the deviation of the system reliability at different moments and sample sizes is always smaller. This phenomenon also proves that the parameter estimation results obtained based on the subsystem node and system node CPT parameter estimation methods proposed in this paper can accurately assess the system's reliability.

As we mentioned earlier, when we design the business logic, the logic flow should proceed from the front-end interaction to the semantic data of the elements, and then, based on the style data of the elements, the drawing data of the elements are derived before they are finally resent to the front-end graphics display. The specific drawing data, such as the vertex coordinates of the line, should not be the object of the direct operation of the business logic. Therefore, on this level, we must organize the entire line into multiple segments, each of which is a separate graphical element that can be interacted with as an independent graphical control similar to any other graphical element, but the segments interact with each other, as in the case of two connected segments, to ensure that when the vertex coordinates of one segment are changed, the vertex coordinates of the other segment are changed. In addition, each line segment is an independent unit of operation, but at the same time, it is semantically the same line, which requires that they are connected in some way, at least to the extent that all other line segments belonging to the same line are available from one line segment. The above problems dictate that we cannot simply split the line segments but must organize them based on the linkage.

A round of forward propagation and backward propagation is called a cycle, and the current acquired weights and biases are tested every 60,000 cycles according to the size of the training set to evaluate whether the recognition accuracy of the convolutional neural network model meets the requirements. There are 10,000 images in the test set, and they are completely independent of the training set and do not overlap with each other. When the recognition accuracy reaches 98.5% or reaches the upper limit of 100 iterations, the training of the current network model is completed, and the current weights and biases are saved. Usually, the number of training iterations in the handwritten digit recognition network does not reach 100. If the network accuracy is still relatively low after reaching 100 iterations, it means that the network model has problems such as gradient disappearance or an excessive learning rate during the training process, and the network model needs to be adjusted and retrained, as shown in Fig. 6.

When making recommendation decisions, filtering is performed based on a threshold value of 0. As seen in the experimental results and Fig. 6, the performance is largely unaffected when the threshold is set low because the data in this community are sparse and the combined prediction probability between most users and code posts is 0. The performance decreases significantly after 0 and continues to increase to 0.8 because few corresponding prediction probabilities exceed this value and the number of generated candidate recommended items decreases. Furthermore, there is even a situation in which all candidate posts are marked as not recommended.

In addition, the recommendation content generated by using the rule-based machine learning approach can find the basis for score accumulation from the data of multiple modules and thus has good interpretability. There is no dependency and coupling among the modules, and the assembly can be flexibly adjusted to generate recommendation content for users from multiple perspectives in a collaborative manner.

However, the rule-based approach to fusion requires a certain amount of experience, and the generated recommendation results are sometimes too limited to the user's historical interaction records. Although restrictions can be imposed by additional rules, some rules may not be suitable for all users, and their design requires careful refinement, such as expert knowledge, and does not have domain-independent adaptability, which invariably increases the cost of the system.

4.2 Mathematical Engineering Modeling and Optimization Results for Public Data in Multitasking Systems

Based on the building information model, the construction volume calculation method with building components as entities can count the geometric and engineering information of all components containing multiple categories, multiple materials, and different sizes and different positions, and through the statistical calculation of their geometric information, including their volume information and surface area, it is very convenient to estimate the workload for different dimensions of the whole construction project. By calculating their geometric information, including their volume information and surface area, among other variables, it is very easy to estimate the workload of the whole construction project in different dimensions and therefore combine it with the construction schedule of the whole project, and it is possible to obtain the budget needed for the whole project in each stage and realize cost control. Based on the model and the list tables exported from the ArchiCAD platform’s list function, the data can be written in Navisworks. The data in the inventory are used to calculate the real-time quantities and correlate them with the progress indicators and the main resource consumption indicators. Data integration is enhanced to rationalize the use of data and to facilitate the viewing and analysis of all data information.

Quantification can calculate the quantity of work in the scenario based on the 2D drawings imported by the project browser in different formats through specific modules. It allows for real-time calculation and control of resource consumption, reducing unnecessary costs and optimizing internal and external resources. It also improves the uniformity, accuracy, and consistency of construction documents and significantly simplifies the entire construction process for the entire company and team, reducing waste during the construction process, improving the efficiency of the process to a certain extent, and reducing changes in the engineering design, as shown in Fig. 7.

The numerical simulation shows that when the system reaches a steady state, the fluid is distributed on the eight corners and twelve ribs of the square surface with the highest curvature, while the surfactant is still distributed in the narrow region of the fluid interface. The effect of the mean curvature on the system is enhanced by increasing the coupling strength, and the same phenomenon is observed when the system reaches the steady state; however, the steady state is reached faster. From the different phenomena simulated by different coupling strengths, the mean curvature affects the final distribution of the fluid, and the larger the coupling strength is, the faster the system reaches the steady state.

After the completion of the whole project, the quality of the project needs to be accepted, which is also an essential stage in the application of BIM technology. There are many factors affecting the project quality and safety management and restricting the progress of the project, and the common factors include unreasonable drawing designs, insufficient research on project-related information, and unreasonable use of existing information. Therefore, based on the collected data, the corresponding stages are modeled and combined with the Gantt chart, data model, and virtual manipulation and analysis simulation technology so that project engineering managers can act according to the presented data by comparing different dimensions and can use software to integrate the relevant information to achieve the purpose of intelligent three-dimensional data analysis. At the same time, it can provide both effective guidance before the project starts and control during the project, and it can summarize the quality management after the project is completed, as shown in Fig. 8.

The initial velocity field evolves into a circular flow with time. However, when the penalty factor is ωn = 0, the velocity field is not tangent to the surface and does not generate a circular flow, which is not consistent with the physical evolution. These results show that the method used in this study can simulate the physical evolution of the fluid flow well and produce the ideal circulation flow on the circle. In addition, the variation in kinetic energy is investigated in this section. The fluid produces small disturbances at the partition, and the interface is rolled up to form a vortex shape with time. At the same time, the distribution of surfactants changes, but they still converge in a narrow region at the fluid interface.

4.3 Performance Comparison

As can be seen from Fig. 9, compared with traditional feature extraction algorithms, multi-view convolutional networks can achieve better recovery performance. The algorithm uses two AlexNets to extract convolution features of two-dimensional projections and sketches respectively. Then a metric network is constructed for the two types of convolution features through a fully connected layer. This hybrid network structure can learn the semantic mapping structure of sketch feature space and view feature space, so it has a very obvious role in improving the recovery performance.

Different from this network structure, FPGA-based multi-convolutional neural network uses conditional adversarial network and residual network to directly form a unified feature representation space, so as to obtain the global optimal feature distribution. The experimental results show that the network structure of FPGA-based multi-convolutional neural network can get better recovery effect. The mAP result is increased, which greatly improves the interclass separability and class clustering of sketch convolution features.

For 3D model recovery, how to allow users to quickly and conveniently recover the required model is one of the primary problems to be solved. Because sketch drawing is simple and convenient, it has become an important interactive means for users to carry out 3D restoration. However, it is possible to evaluate the similarity between the sketch and the 3D model through the view, and then return the restored results. However, compared with the model view, the hand-drawn sketch has great irregularity, which greatly affects the recovery accuracy. Different people have different painting styles for the same model, and there is also a big gap in the painting ability of different people, which leads to great differences in the same type of sketches in hand-drawn sketches, and this difference even leads to great difficulties in artificial classification and recovery. In recent years, the results of 3D model recovery competition also show that even if the convolutional network is used to extract features, the sketch-based 3D model recovery still cannot achieve the desired recovery accuracy. Therefore, how to reduce the visual feature difference between freehand sketch and model view and improve the feature clustering is an urgent problem to be solved in 3D model restoration.

This paper makes sketch space and view space have consistency in function distribution through training, and reduces the difference in visual features between sketch and 3D model view. Different from previous studies, the algorithm directly embeds the view space generated by the sketch into the view convolutional network, avoiding the need to integrate the sketch and view results through a separate transformation network. The experimental analysis shows that the hybrid convolutional network structure directly embedding the sketch into the view improves the feature separability of the network and significantly improves the recovery accuracy.

In this paper, a concatenated convolutional network structure is proposed for semantic correlation between sketch and 3D model view. The view generation error is introduced into the target loss function to improve the feature clustering. In the algorithm implementation, this paper generates the similarity between the sketch and the view of the adversarial network. Then the convolution features are extracted by residual network and weighted loss function. The experimental analysis shows that this cascade network structure can effectively improve the clustering of sketch and 3D model view, so as to obtain better recovery accuracy.

For 3D model sketch recovery, a major problem is the need for multi-view rendering to support recovery accuracy. In the follow-up work, it is necessary to further improve the generated adversarial network, so that different sketches can approximate the optimal view of the 3D model. In this case, the recovery efficiency can be greatly improved without reducing the recovery accuracy. On the other hand, how to establish a semi-supervised learning mechanism for 3D model sketch recovery to solve the problem of limited sketch samples is also a problem that needs to be solved.

In this paper, the hardware deployment and multitask switching of convolutional neural networks combined with FPGA platforms are studied according to the computational and structural characteristics of convolutional neural networks, considering the application of multiple tasks in real-world scenarios. To realize switching and computation between multiple tasks, the "task sequence + control module" structure is adopted. The priority of each task to be computed is set first, and then, the control module schedules the switch according to the priority of the tasks in the task sequence according to the idea that the higher priority tasks are computed first. Since there is no parallelism between the layers of the convolutional neural network and the same type of hidden layers are computed in the same way, the computation module is implemented with configurable computation units. There are three basic units in the operation module, namely, the convolutional unit, pooling unit, and fully connected unit, and the specific configuration of the computational parameters is determined by the control module. This paper compares three kinds of multitasking switching mechanisms: the arbitrary moment switching mechanism has good real-time performance but needs to save more intermediate data, which is difficult to implement for complex networks; the ordinary layer switching mechanism has slightly worse real-time performance, needs to save less intermediate data, and is less difficult to implement; and the real-time layer switching mechanism has good real-time performance, does not need to save intermediate computational data, and is less difficult to implement. Therefore, in this paper, we consider the real-time complexity of the system and use a combination of ordinary layer switching mechanisms and real-time layer switching mechanisms to realize switching and computation between multi-convolutional neural network tasks.

IFC Industry Foundation Classes

GANs Generative adversarial networks

VAEs variational autoencoders

Competing interests

The authors declare that they have no competing interests.

Acknowledgments

The authors would like to thank the anonymous reviewers for their valuable comments and suggestions that helped improve the quality of this manuscript.

Funding

This work is supported by Natural Science Research Project of Education Department of Anhui Province (2022AH051379, 2022AH051376 and KJ2021A1110), Doctoral Research Foundation of Suzhou University(2023BSK023) and University-level quality engineering project(szxy2023jyjf082).

Authors’ contributions

All authors contributed to the work. Biao Lu conceived and designed the study. Deliang Liu carried out most of the analyses. Biao Lu drafted the manuscript. Deliang Liu and Wei Zhou performed the experiments. All authors read and approved the final manuscript.

Availability of data and materials

Not applicable.

Wenzel J, Matter H, Schmidt F. Predictive multitask deep neural network models for ADME-Tox properties: learning from large data sets[J]. Journal of chemical information and modeling, 2019, 59(3): 1253-1268.
Guo S, Zhang B, Yang T, et al. Multitask convolutional neural network with information fusion for bearing fault diagnosis and localization[J]. IEEE Transactions on Industrial Electronics, 2019, 67(9): 8005-8015.
Çimen H, Çetinkaya N, Vasquez J C, et al. A microgrid energy management system based on non-intrusive load monitoring via multitask learning[J]. IEEE Transactions on Smart Grid, 2020, 12(2): 977-987.
Hadsell R, Rao D, Rusu A A, et al. Embracing change: Continual learning in deep neural networks[J]. Trends in cognitive sciences, 2020, 24(12): 1028-1040.
Xu X, Yoneda M. Multitask air-quality prediction based on LSTM-autoencoder model[J]. IEEE transactions on cybernetics, 2019, 51(5): 2577-2586.
Guo S, Lin Y, Li S, et al. Deep spatial–temporal 3D convolutional neural networks for traffic data forecasting[J]. IEEE Transactions on Intelligent Transportation Systems, 2019, 20(10): 3913-3926.
Yu S, Chai Y, Chen H, et al. Fall Detection with wearable sensors: A hierarchical attention-based convolutional neural network approach[J]. Journal of Management Information Systems, 2021, 38(4): 1095-1121.
Liang X. Image‐based post‐disaster inspection of reinforced concrete bridge systems using deep learning with Bayesian optimization[J]. Computer‐Aided Civil and Infrastructure Engineering, 2019, 34(5): 415-430.
Dhillon A, Verma G K. Convolutional neural network: a review of models, methodologies and applications to object detection[J]. Progress in Artificial Intelligence, 2020, 9(2): 85-112.
Dubey S R, Chakraborty S, Roy S K, et al. diffGrad: an optimization method for convolutional neural networks[J]. IEEE transactions on neural networks and learning systems, 2019, 31(11): 4500-4511.
Xiao Z, Hu Z, Geng L, et al. Fatigue driving recognition network: fatigue driving recognition via convolutional neural network and long short‐term memory units[J]. IET Intelligent Transport Systems, 2019, 13(9): 1410-1416.
Lee J H, Han S S, Kim Y H, et al. Application of a fully deep convolutional neural network to the automation of tooth segmentation on panoramic radiographs[J]. Oral surgery, oral medicine, oral pathology and oral radiology, 2020, 129(6): 635-642.
Sun J, Tárnok A, Su X. Deep learning‐based single‐cell optical image studies[J]. Cytometry Part A, 2020, 97(3): 226-240.
Huang Z, Tang J, Shan G, et al. An efficient passenger-hunting recommendation framework with multitask deep learning[J]. IEEE Internet of Things Journal, 2019, 6(5): 7713-7721.
Ozcan T, Basturk A. Transfer learning-based convolutional neural networks with heuristic optimization for hand gesture recognition[J]. Neural Computing and Applications, 2019, 31(12): 8955-8970.
Wang Y, Zhang M, Lin A, et al. Mining structure–property relationships in polymer nanocomposites using data driven finite element analysis and multi-task convolutional neural networks[J]. Molecular Systems Design & Engineering, 2020, 5(5): 962-975.
Liu Z, Shen C, Fan X, et al. Scale‐aware limited deformable convolutional neural networks for traffic sign detection and classification[J]. IET Intelligent Transport Systems, 2020, 14(12): 1712-1722.
Liu S, Xia Y, Shi Z, et al. Deep learning in sheet metal bending with a novel theory-guided deep neural network[J]. IEEE/CAA Journal of Automatica Sinica, 2021, 8(3): 565-581.
Khan A, Sohail A, Zahoora U, et al. A survey of the recent architectures of deep convolutional neural networks[J]. Artificial intelligence review, 2020, 53(8): 5455-5516.
Rajesh Kumar T, Suresh G R, Kanaga Subaraja S, et al. Taylor‐AMS features and deep convolutional neural network for converting nonaudible murmur to normal speech[J]. Computational Intelligence, 2020, 36(3): 940-963.
Jin N, Zeng Y, Yan K, et al. Multivariate air quality forecasting with nested long short term memory neural network[J]. IEEE Transactions on Industrial Informatics, 2021, 17(12): 8514-8522.
Murugan R, Goel T, Mirjalili S, et al. WOANet: Whale optimized deep neural network for the classification of COVID-19 from radiography images[J]. Biocybernetics and Biomedical Engineering, 2021, 41(4): 1702-1718.
Luo C, Yu L, Yan J, et al. Autonomous detection of damage to multiple steel surfaces from 360 panoramas using deep neural networks[J]. Computer‐Aided Civil and Infrastructure Engineering, 2021, 36(12): 1585-1599.
Huang Z, Shan G, Cheng J, et al. TRec: an efficient recommendation system for hunting passengers with deep neural networks[J]. Neural Computing and Applications, 2019, 31(1): 209-222.

Download PDF

Editorial decision: Major revision
11 Aug, 2023
Reviewers agreed at journal
13 Jul, 2023
Reviewers invited by journal
13 Jul, 2023
Editor assigned by journal
10 Jul, 2023
First submitted to journal
06 Jul, 2023

You are reading this latest preprint version

Mathematical engineering modeling and optimization of convolutional neural networks for common data areas in multitasking systems

Status:

Version 1

Abstract

Figures

Introduction

Related Works

Methods

3.1 Data convolutional neural network algorithm design

3.2 Mathematical engineering modeling and optimization analysis of public data in multitasking systems

3.3 Experimental design

Results

4.1 Data convolutional neural network algorithm results analysis

4.2 Mathematical Engineering Modeling and Optimization Results for Public Data in Multitasking Systems

4.3 Performance Comparison

Discussion

Conclusion

Abbreviations

Declarations

References

Status:

Version 1