Image Modification Using Deep Neural Cellular Automata

doi:10.21203/rs.3.rs-1503832/v1

Download PDF

Research Article

Image Modification Using Deep Neural Cellular Automata

https://doi.org/10.21203/rs.3.rs-1503832/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Abstract — Art Style Transfer is part of the rapidly growing AI Art community of recent times. Pioneered by Gatys et al, this class of methods makes it possible to convey styles, textures, patterns, and more. to a target image. The expressive masking feature of mainframe vision models such as VGG19 is used as a lossy function. The method of performing this transformation takes many forms, from the original method that directly optimizes the image pixels to more recent forms that form the CNN form to create a generic transport network. The method presented in this article is similar to more recent methods, but takes advantage of a new class of deep learning methods, deep neural automation. This new method provides the ability to convert any image into a target type that, like the CNN method mentioned previously, uses the same automatic data update rules over and over again. This paper contains how to use NCAs to transform images. also contains the Gatys et. al. type style transfer and the other a OpenAI CLIP based version where a prompt can be given to train NCAs to perform that transformation.

Cellular Automata ( CA )

convolutional neural network CNN

Neural Automata

The neural CA framework is a new way of modelling biological events in multicellular animals. An artificial neural network is employed as an update rule for mobile auto-data in these systems. End-to-end discriminating systems, or neural CAs, are systems in which neural parameters may be learnt for fulfil specified task. Here Neural CA used to control the cartpole in this study. Environmental observations are fed into the input cells, and the reading system is based on the cell value output. We used deepQ learning to train the model, which uses the output cell state as an estimate of the Q value that needs to be tuned. We discovered that the compute power of auto-mobile data was maintained throughout hundreds of thousands of iterations, resulting in extremely accurate results. Strategies involved in Image Processing edge detection, classification, and texture processing.

The goal of edge acquisition is to process a two-dimensional image and determine with a computer where there are edges or borders in the image. The edges are part of an image that contains important visual information as it relates to the geometric diversity of objects. The research presented in this paper is based on the application of digital imaging techniques that work well with bio-inspired systems theory. The context of theoretical and practical research conducted in the field of image processing is supported by the idea that cell edge detection techniques can have similar effects on gradient and Laplacian-based archeological methods ^[2].

We use NCA to the job of texture generation in this paper. Rather than manufacturing pixel-perfect reproductions, this operation entails duplicating the overall look of a texture template. We'll concentrate on texture losses that allow for some uncertainty. We study NCA models' acquired behavior after training them to recreate textures and find a few unexpected results. We argue that the cells learn dispersed, local algorithms as a result of these experiments. To do this, we apply an old trick: we employ neural cellular automata as a differentiable image parameterization^[12]. All of these methods function in a similar manner. In computer vision, neural networks contain a detailed internal representation of the pictures they examine. This format may be used to define the attributes we want a picture to have (for example, style), and then to optimize the input image to have those properties. Because the networks are differentiable with respect to their inputs, this type of optimization is possible: we may gently adjust the picture to better suit the desired attributes, and then apply these modifications iteratively via gradient descent. The state space is given by a grid with the width and height of the image that is to be edited with 18 channels, 3 rgb of the stating image that cant be overwritten by the NCAs, 3 rgb channels of the image that is being iterated on and 12 hidden channels. The three channels that correspond to the original unedited image so that the NCA have access to the unaltered image as reference.

To address this issue, we present a parametric texture model in this paper. Instead of using a model for the early visual system to describe textures, we employ a convolutional neural network as the foundation for our texture model, which is a functional model for the entire ventral stream. As a result, we have a texture model that is parameterized by spatially invariant representations based on the convolutional neural network's hierarchical processing architecture.

A). Convolutional neural network

We employ the VGG-19 network, a previously presented and thoroughly documented convolutional neural network trained on object recognition.

After a sequence of convolutional layers, a max-pooling layer is added. After each of the first three pooling layers, the number of feature maps is doubled. When paired with spatial down sampling, this improvement decreases the number of feature responses by a factor of two. The network architecture is schematically illustrated, as well as the number of feature mappings in each layer.Because just the convolutional layers are used, the input pictures can be any size. The first convolutional layer has the same size as the picture, and the feature map sizes stay fixed for the subsequent layers. In general, each network layer has its own non-linear filter bank , whose complexity rises in proportion to the layer's location in the network .

B). Texture

To make a texture from a source picture, we first extract features of varying sizes in a homogenous manner. We compute a spatial summary statistic on the feature responses to derive a stationary description of the source picture.

To create a new texture based on a white noise image, we use gradient descent to find another image that fits the Gram-matrix representation of the original image. In this optimization, the mean-squared distance between the entries of the original picture's Gram matrix of the image being generated is minimized.

The Use of Simple Cellular Automata in Image Processing ^[1]Laura Diosan and team presents existing methods that support broad goal of identifying the rules of cellular automata for segmenting images automatically. The segmentation process involved in image processing is not affected by the number of Image features

Image processing using 3-state cellular automata^[2]Paul L. Rosin ^[2]old paper describes work on Intensity of Images and modified search mechanism in order to speed up the training stage. The paper presents the advantages of using 3 state representation with important conclusions on this issue.

Survey Paper on Training of Cellular Automata for image^[3]Swati Chauhan^[3]included portable automaton may be a finite automatic information that gives a computer model which is discrete under the complex behavior of image (computer and in real). This paper provides an investigation of how researchers can automatically train mobile data to find out the best rules for reaching the optimal solution during a large search space. used with modification techniques We found that many researchers have provided different-different methods to train cellular Automata this Survey Paper explains which method has beneficial approach in image manipulation to make it effective. Its security is enhanced mainly due to the factor that the matrix is broken into lower and upper triangular matrices. The system maintains four levels of security due to the finite state machine and the secret key.

Neural Cellular Automata Manifold ^[4]Alejandro Ruiz ^[4]with two authors introduces a new model type that encodes the picture space as mobile auto-data that may be used to generate the target models. It also goes over the Autoencoder architecture, which is used to train thousands of photos. The proposed system can be trained from start to finish and has consistent generalizability when compared to generated images.

A Survey on Cellular Automata ^[5]Niloy Ganguly and team in 2018 published paper this paper This study presents a survey of the mainstream literature on numerous approaches scholars have used to model data from automobiles. The research covers many forms of cellular data automation utilized for modelling as well as analytical approaches for predicting worldwide behavior from the behavior of the local AC in various global circumstances; the challenge is known because of inverse problem. Lastly, it discusses the many fields from it CA used.

Learning Graph Cellular Automata ^[6]Daniele Grattarola,Lorenzo Livi &Cesare Alippi ^[6]in 2021 discovered that the computing power of the automotive data is maintained through hundreds of thousands of repetitions in this work, resulting in highly consistent behavior in the environment it regulates. The system also displayed real-world characteristics such as a development stage, post-damage repeatability, noisy environment stability, and radio interference resistance. Input cancellation appears to be the case.

Towards self-organized control: using neural cellular automata to robustly control a cart-pole agent ^[7] Alexandre Variengien and research contributor discovered that the automobile data's computing power is maintained across hundreds of thousands of repeats in this work, resulting in extremely consistent behaviour regulates over much of steps. It also displayed real-world characteristics such as development stage, post-damage repeatability, noisy environment stability, and radio interference resistance. It appears that input cancellation is the case.

Cellular Automata as a Tool for Image Processing ^[8] Paul L. Rosin & Xianfang Sun ^[8]described automated usage of mobile data for picture processing is described in detail. We begin by considering the maximum samples which may present in a neighbor, giving permission of transformation invariance. These models correlate to potential rules, and many schemas are provided for learning a suitable for rules automatically from a data which is under training diagram that describes the rules for executing numerous activities. processing of images .

A Survey on Two Dimensional Cellular Automata and Its Application in Image Processing ^[9]Deepak Ranjan Nayak, Prashanta Kumar Patra & Amitav Mahapatra ^[9]In today's world, using parallel algorithms to solve image manipulation work which is a highly desired approach. The most common and basic parallel computing model is Automated Data Mobility (CA). As a result, CA has been used successfully in the field of image processing in recent years. This article surveys the existing literature on several methods used by different researchers to solve some important image processing problems using mobile autonomous cars. Rotation, zooming, translation, segmentation, edge detection, image compression, and denoising are among the important image processing tasks covered in the survey. Finally, the results of several methodologies' experiments are presented.

Training Cellular Automata for Image Edge Detection ^[10]Anand Prakash Shukla ^[10]introduced a new method to generate image thresholds using an algorithm called Otsu`s after applying automatic cellular data rules to training proposed. The proposed method was found to significantly reduce the time of it with zero loss of results. The outputs are confirmed qualitatively and quantitatively as well as compared it with several edge detection methods it is said to have good edge detection in those photographs. Furthermore, compared with standard edge detection methods, the introduced method does significantly good in corner identification .

Incorporating neural cellular automata has been shown to have a remarkable ability to learn the rules necessary to "grow" images, classify morphologies, segment images as well as perform general calculations. This model provides very efficient results in style creation in gray-scale or multi-color, as well as in image recreation, texture synthesis, and reinforcement learning.

The model provided here is the first step in developing embedding spaces to effectively capture not just static information but behavioral patterns, since machine learning techniques are already deemed crucial for the research of style metamorphosis. In addition, our studies develop programming for several agents that interact with the environment to accomplish the intended aggregate result. Given that our model can handle any type of vectorial input, we don't see any theoretical barriers to learning sophisticated programs and performing a variety of tasks using simply data and an estimated loss function.

Laura Diosan,Alina Enescu & Anca Andreica “The Use of Simple Cellular Automata in Image Processing” 2017 .
Paul L. Rosin “Image processing using 3-state cellular automata” 2010 .
Swati Chauhan “Survey Paper on Training of Cellular Automata for image” 2013 .
Francesc Noguer , Alejandro Ruiz, Armand Vilalta, “Neural Cellular Automata Manifold” 2020 .
Mark Sandler, Andrey Zhmoginov, Liangcheng Luo, Alexander Mordvintsev , Ettore Randazzo and Blaise Ag´uera y Arcas “Neural Cellular Automata Manifold” 2020 .
Niloy Ganguly, Biplab K Sikdar, Andreas Deutsch,Geoffrey Canright,P Pal Chaudhuri “A Survey on Cellular Automata” 2018.
Daniele Grattarola,Lorenzo Livi&Cesare Alippi “Learning Graph Cellular Automata” 2021 .
Alexandre Variengien, Stefano Nichele, Tom Glover1 and Sidney Pontes-Filho “Towards self-organized control: using neural cellular automata to robustly control a cart-pole agent” 2021 .
Paul L. Rosin & Xianfang Sun “Cellular Automata as a Tool for Image Processing” 2021.
Deepak Ranjan Nayak, Prashanta Kumar Patra & Amitav Mahapatra “A Survey on Two Dimensional Cellular Automata and Its Application in Image Processing” 2014 .
Anand Prakash Shukla “Training Cellular Automata for Image Edge Detection” 2016 .
Alexander Mordvintsev , Nicola Pezzotti , Ludwig Schubert , Chris Olah “Differentiable Image Parameterizations” 2018
Vayadande, Kuldeep, Ritesh Pokarne, Mahalaxmi Phaldesai, Tanushri Bhuruk, Tanmai Patil, and Prachi Kumar. “Simulation of conway’s game of life using cellular automata.”
Vayadande, Kuldeep, Ram Mandhana, Kaustubh Paralkar, Dhananjay Pawal, Siddhant Deshpande, and Vishal Sonkusale. "Pattern Matching in File System." International Journal of Computer Applications 975: 8887.
Vayadande, Kuldeep, Neha Bhavar, Sayee Chauhan, Sushrut Kulkarni, Abhijit Thorat, and Yash Annapure. Spell Checker Model for String Comparison in Automata. No. 7375. EasyChair, 2022.
VAYADANDE, KULDEEP. "Simulating Derivations of Context-Free Grammar." (2022).
Vayadande, Kuldeep, Neha Bhavar, Sayee Chauhan, Sushrut Kulkarni, Abhijit Thorat, and Yash Annapure. Spell Checker Model for String Comparison in Automata. No. 7375. EasyChair, 2022.
Varad Ingale, Kuldeep Vayadande, Vivek Verma, Abhishek Yeole, Sahil Zawar, Zoya Jamadar. Lexical analyzer using DFA, International Journal of Advance Research, Ideas and Innovations in Technology
Kuldeep Vayadande, Harshwardhan More,Omkar More, Shubham Muley,Atahrv Pathak, Vishwam Talanikar, “Pac Man: Game Development using PDA and OOP'',

Competing interests: The authors declare no competing interests.

ImageModificationUsingDeepNeuralCellularAutomata.pdf

Download PDF

Version 1

posted

You are reading this latest preprint version

Image Modification Using Deep Neural Cellular Automata

Status:

Version 1

Abstract

Figures

Introduction

Style, Clips Transformation

Literature Review

Advantages

Conclusion

Reference

Declarations

Supplementary Files

Status:

Version 1