FED-AT-VIDEO NETS - A Federated Capsule – Self Gated Learning Architecture for the  Multi-View Video Summarization Technique.

doi:10.21203/rs.3.rs-3232381/v1

Download PDF

Research Article

FED-AT-VIDEO NETS - A Federated Capsule – Self Gated Learning Architecture for the Multi-View Video Summarization Technique.

https://doi.org/10.21203/rs.3.rs-3232381/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Video analytics using the huge amount of data from the surveillance networks has become a core function for multiple applications such as object detection, human activity recognition, health care diagnosis and so on. Due to its massive nature, achieving efficient video summarization has become a vital challenge for constructing video analytics architecture. Moreover, these video contains private information, and security against different intruders has also added fuel to the existing challenges. In recent years, a number of architectures have been proposed to achieve better and secured multi-view summarization (MVS) techniques that can aid in better video analytics. Unfortunately, existing architecture needs brighter light of research to eradicate the aforementioned challenges. In this article, federated deep gated attention architecture (FDGAA) is proposed for attaining the secured MVS by organizing the computing and networking resources of cloud and edge cameras collectively. The proposed architecture is modeled as a three-tier framework which is precisely given as 1) Video collection unit (VDU) that collects the videos from the different views of the camera installed. 2) Distributed Training network(DTN) which consists of federated learning self-attention saliency Gated recurrent units(SAS-GRU) in which the training is collaboratively shared among the edges without sacrificing the privacy of video information. 3) Finally the extracted deep features are summarized in the cloud for further processing. Utilizing a variety of datasets and NVIDIA Nano Boards as edge nodes, substantial research is conducted to develop the Google Federated Tensorflow Libraries-based federated learning architecture. Performance has been compared with other MVS systems that are currently based on deep learning to demonstrate the superiority of the proposed framework. In comparison to other state-of-the-art MVS approaches, the experimental evaluation shows that the suggested model performs better.

Video analytics

Multi-view Summarization

Federated deep attention

Cloud

Edge camera

Tensorflow

With the explosive growth of the Internet, there arise striking enhancements in communication, networking architectures and gigantic volume of data. In many real-time applications, the fertile product of this enhanced scenario is video analytics on Internet of Things (IoT) platforms[1–2]. These merged platforms are evolving quickly and have taken the place of conventional sensing and analytics methods. The Internet of Things (IoT) is a set of intelligent wireless networks that connects a variety of heterogeneous gadgets to facilitate information exchange. These gadgets converse with one another and produce enormous amounts of data every day [3–5]. The video data generated by the multiple view cameras in IoT platforms find brighter applications such as health care, industry and even robotics. Since the number of cameras exponentially increases and it is now reflected as a major Big Data problem[6–10], efficient processing of these videos from multi-view cameras is considered one of the major challenges among researchers.

One such method for tackling the aforementioned problem is multi-view video summarising (MVS), which is frequently employed in internet-enabled surveillance systems installed at workplaces, financial institutions, manufacturing facilities, and city intersections for both private and public securities [11–14]. Multi-view video summary addresses the problem of distilling multi-view movies into instructive video summaries, which are frequently presented as dynamic video shots produced from multi-views, by taking into consideration content relationships within each view and across numerous views. For critical occurrences compared to less important ones, more specific information will be presented in the multi-view summary. The user will be able to accomplish this without needing to watch the complete multi-view film in order to comprehend the crucial details from the many points of view. Multi-view summarization will also be beneficial for the administration, analysis, and management of multi-view video content [15–17].

Current methodologies[18–20] used for multi-view video summarising techniques may provide duplicate summary results in the overlap of information with other video components. Also, multi-view videos are captured non-synchronously from different views of installation, these videos suffer from environmental dependent factors such as lightning, rain and other climatic conditions. Hence intelligent techniques need to be coined that can aid in an effective multi-video summarization. Currently, a succinct multi-view video summary that enables strong information extraction is produced using machine and deep learning techniques [21–23]. However, machine and deep learning algorithms needs can produce better performances, current circumstances of developing intelligent systems have become more and more complex due to privacy concerns and security flaws that may cause the worst catastrophes in the videos[24]. Since the sensitive data is in multiple videos, deployment of secure, reliable and computational free-learning-based MVS frameworks needs more progressive light to overcome privacy issues and handle the multiple videos[25, 26]. Motivated by this drawback, this paper proposes FED-AT-VIDEO nets, computationally less federated multiple video summarization architecture. The proposed architecture works on the principle of federated principles and implements the Deep Saliency Based Gated Attention learning algorithms for training. To the best of our knowledge, the FED-AT-VIDEO net is considered the first of its kind for light-weight secured multiple video summarization techniques which have the following features :

Multiple-Video Collection Unit: This unit collects multiple videos from heterogeneous sources and stores them as separate source labels.

Deep Capsule Gated Attention Based Learning Network: To overcome the problem of overlapping redundant features from multiple videos, the research article demonstrates the usage of Deep Capsule gated attention networks for extracting the deeper non-overlapping features which aid in effective video summarization.

Federated Learning Network: Federated learning framework for Deep Capsule Gated Attention networks has been proposed to handle sensitive data in the videos without sacrificing security and computational efficiency.

MV Summarization technique: With the use of cloud computing, video data may be handled more precisely and effectively, and the resulting summary may be analysed utilising the aforementioned federated learning method. This allows for the efficient analysis of just significant data.

The remaining portions of the articles are structured as follows: Section 2 lists the relevant works created by various authors. Section 3 describes the Proposed FED-AT-VIDEO nets. Section 4 describes the experimental setup, Comparative Analysis and Discussion. Section 5 Concludes the articles

A Parallel Deep Learning Framework for Video Surveillance that seeks to strike an appropriate equilibrium between precision and system efficiency or reliability was developed by Mayuri Karvande et al., 2021, with an emphasis on deep learning techniques and threading. For object detection, the Dynamic Selection Algorithm (MobileNet) alternates between two networks dependent on the queue's state to maintain system stability. When selecting a model for a particular application, one prefers to choose one with more accuracy. However, when it comes to applications, it is important to consider how rapidly the model delivers predictions. It is impossible to make forecasts with great accuracy and speed at the same time [27].

Using a powerful single static-camera multiple pedestrian recognition system for tracking developed by T. Barbu et al. in 2022, innovations in smart cities may be able to exploit this technology with effectiveness. The detection of moving persons is made possible by using a mix of cutting-edge computer vision and machine learning algorithms on each frame of the colour video series. These methods include “Aggregate Channel Features (ACF), Support Vector Machines (SVM), Histogram of Oriented Gradients (HOG), and Gaussian Mixture Models (GMM)”. The following phase involves using a DL-based multiscale analysis of the sub-pictures of the recognised pedestrians in a tracking strategy based on instance matching. In terms of precision, this strategy produces better results. However, the computational complexity is rather high. [28].

To solve the above-mentioned issues with recognizing anomalies for intelligent recordings, C. Huang et al. proposed the temporal-aware contrastive network (TAC-Net) in 2022. For capturing the high-level semantic features, TAC-Net, an unsupervised method, uses deep contrastive self-supervised learning. Additionally discussed is anomaly detection, which makes use of several self-supervised activities. The numerous task losses and contrastive similarities are combined throughout the phase of inference to determine the abnormal score. The effectiveness of TAC-Networks has been demonstrated through testing. The primary flaw with this system, though, is how rapidly it interprets data from real-time situations [29].

In order to identify the pet's disposition and health, Tsai et al. 2020 used KNN and provided a time-continuous condition of featured sections in addition to many emphasised elements. The core skills of the intelligent animal tracker were investigated on many levels using the KNN audio identification model, the multiple feature dependency detection approach, and the picture identification mechanism. The algorithm may therefore identify additional moods or emotions as long as they can be described by the distinctive properties of pets. The system can enhance its capacity to detect items by including an autonomous optimization element. It would not be proper for this configuration to be used in a real-time scenario, though [30].

Zahid et al., 2020 employed a bagging ensemble to empirically show the durability of a binary classifier. This technique has also demonstrated how feature acquisition dramatically improves an algorithm's precision by including Inceptionv3 elements in a 3-Layer Fully Connected (FC) neural network. Every research made use of the UCF-Anomaly Detection dataset, which contains real-world anomalous and normal video occurrences. This framework performs better in terms of AUC and RoC features. A large-scale data environment, however, is inappropriate [31].

Kyrkou et al., 2020 described the task as an imitation learning issue that required to be finished under supervision using deep learning in order to successfully face the active visual monitoring difficulty. The transition from visual data to camera movement was immediate. A DCNN is completely trained to act as the camera controller, learning the whole processing chain necessary to direct a camera to follow several objects and determine their density from a single image. The method can surpass previous strategies in terms of monitoring performance, both in terms of the number of targets viewed and in terms of monitoring time, while still attaining up to 25 FPS, according to experimental results. But it consumes a lot more energy [32].

In their sophisticated multi-camera framework, Sonavane et al., 2021 used the Resnet-34 model to perform recognition of faces and Tesseract-based optical character recognition (OCR) for detection of vehicle number plates. On a map that graphically represents the time and location of their exploration, the people and vehicles that the various cameras have identified are drawn out. Major businesses, such as places of employment, financial institutions, shopping centres, residential areas, etc., will profit from this security and monitoring platform. However, the primary drawback of this system is that as data volume increases, latency increases [33].

Kim et al. presented an accurate reinforcement learning-based monitoring system for an object of interest or deviant behaviour in 2022. Utilising either a CCTV controller or a DL framework for object identification, this kind of technology can identify details about the object like locations and dimensions. A PTZ camera management parameter is created and sent to the video monitoring system together with the information about the detected item that is mapped to the cameras that will be connected. The PTZ camera control value determines how the video monitoring system's PTZ management is executed. Using films filmed from several angles or locations aids in improving the precision of video analysis. This system requires greater power to handle data [34].

In 2019, Wang et al. developed a real-time multiple camera video-based person re-ID system using the NVIDIA Jetson TX2 platform. When stringent confidentiality and timely monitoring are required, this strategy may be used. This system uses human re-ID technologies and YOLOv3-tiny based lightweight approaches to cut computation by 46%, reduce model size by 39.9%, and speed up calculation by 21%. The system's ability to identify pedestrians with more accuracy has also been effectively enhanced. The main problem with this approach is the rising computational complexity [35].

A video retrieval system for retrieving and summarizing people from numerous security cameras was presented by Chien-Hao et al. in 2021. This system uses deep learning to accomplish human segmentation and attribute feature representation. The system for retrieving people comprises of appearance-based multi-CNN, pixel-by-pixel person segmentation, and a technique for summarizing videos. In multi-camera retrieval and surveillance applications, experiment results showed that the suggested system operates satisfactorily and offers retrieval performance. But this framework's fundamental flaw is that it processes information slowly in real-time situations [36]. Table 1 summarizes the literature survey.

Table 1: Quick summary of Deep Learning Framework for the Multiple Surveillance camera videos

Authors	Techniques incorporated	Accuracy	Computational complexity	Processing speed	Support for real time environment	Support for large scale environment	Memory consumption	Latency	Energy efficiency	Power consumption
Mayuri Karvande et al., 2021 [27]	Dynamic Selection Algorithm (MobileNet)	✔ Average	✘ High	✔ Average	✘ Not suitable	✘ Not suitable	✔ Average	✔ Average	✔ Average	✔ Average
T. Barbu et al., 2022 [28]	GMM, HOG, SVM, ACF	✔ Average	✘ High	✘ less	✘ Not suitable	✘ Not suitable	✘ High	✘ High	✘ less	✔ Average
C. Huang et al., 2022 [29]	TAC-Net	✔ Average	✘ High	✔ Average	✘ Not suitable	✘ Not suitable	✔ Average	✘ High	✔ Average	✘ High
Tsai et al., 2020 [30]	KNN	✔ Average	✘ High	✘ less	✘ Not suitable	✘ Not suitable	✔ Average	✔ Average	✔ Average	✘ High
Zahid et al., 2020 [31]	Fully Connected (FC) neural network	✔ Average	✔ Average	✘ less	✔ Suitable	✘ Not suitable	✘ High	✘ High	✘ less	✘ High
Kyrkou et al., 2020 [32]	DCNN	✔ Average	✘ High	✘ less	✘ Not suitable	✔ Suitable	✔ Average	✘ High	✘ less	✔ Average
Sonavane et al., 2021 [33]	Resnet-34 model	✔ Average	✘ High	✘ less	✘ Not suitable	✘ Not suitable	✔ Average	✘ High	✔ Average	✔ Average
Kim et al., 2022 [34]	Reinforcement learning	✔ Average	✘ High	✔ Average	✘ Not suitable	✘ Not suitable	✘ High	✘ High	✘ less	✘ High
Wang et al., 2019 [35]	DCNN	✔ Average	✘ High	✔ Average	✘ Not suitable	✘ Not suitable		✔ Average	✘ less	✘ High
Chien-Hao et al., 2021 [36]	Multi-CNN	✔ Average	✘ High	✘ less	✘ Not suitable	✘ Not suitable	✘ High	✘ High	✔ Average	✘ High

In this section, a proposed federated based FED-AT –VIDEO net is presented. The proposed framework consists of three important phases such as Video collection unit, federated training networks (Cloud/server model) and summarization methodology. In the first phase, videos from the multi-view camera are collected and stored in the database. In the second phase, important distant objects are extracted by the U-Net based saliency methods which are then used as the inputs to the proposed federated gated–attention networks to extract the secured and non-redundant features. Finally, the secured intelligent features are used for the summarization in the cloud-servers. The complete framework with its own functioning phases is demonstrated in Fig. 1.

3.1 Multi-View Video Collection Unit(MVVC) :

This module captures the multi view videos from the different video sources installed in the industrial environments. For an efficient construction of a video collection unit, raspberry pi model 4 is used as the main processor interfaced with cameras. Four Raspberry Pi (RPI) interfaced with cameras are used in this experimentation which collects the video data from different angles and feeds to the cloud server using the WIFI networks in which the process of segmentation, training and summarization is deployed. The proposed MVVC performs all the computations in real time with six frames per second(fps). Each video from the sources is stored in the server as tag-lines ID databases by differentiating each video frame by its source and time stamps.

3.2 Cloud Server Model :

In this phase, object saliency segmentation, federated learning process and finally summarization technique are deployed and a detailed description of each module is presented.

3.2.1 Object Saliency Segmentation Process :

Processed in this portion are the MVVC's outputs. The MVS considers object segmentation to be a crucial step that guarantees the highest level of summary quality. Since these camera generates multiple videos that contains humans and other uncertain activities, it is necessary to segment and detect the original objects that can aid in better summarization. From recent studies, many variants of Convolutional neural networks have been exploited for the segmentation and object detection. To obtain the better accuracy with less time of response, saliency based optimized tuned U-NETS(SBTU-nets) is proposed in this research. The optimization and fine tuning process with saliency extraction assist in enhancing the segmentation accuracy even if the targets are distant from each camera. A detailed description of the proposed model is explained in the preceding sections.

3.2.1.1 SBTU-nets :

Saliency Segmentation is the method which is used to segment sensitive information from the video frames. Also useful in segmenting important information from background images. Normally the saliency segmentation is processed by active contour methods [37]. Since this existing method introduces many ripples of complexity, this research incorporates the Fine-tuned U-NET for segmenting the saliency objects from each frame which is shown in Fig. 2. The 3 primary components of the U-Net network topology are “skip connection, up-sampling, and down-sampling”. U-net is an encoder-decoder structure in which the encoder structure uses up-sampling while the decoder structure uses down-sampling. Encoders are in charge of extracting features; in other words, the size of the picture is decreased by down sampling and convolution to extract a few superficial features. On the other hand, the decoder obtains some deep global features using up-sampling and convolution layers. In the deeper global region, there will be more global features (contextual information) whereas the shallow region contains more local detailed information. This kind of combination enhances the network performance after the fusion of multiple features. In the U-Net model, which successfully captures object attributes, the skip-connection approach and weighted cross entropy method are also utilised. Since the U-NET's hyper-parameters may be adjusted, it is chosen as the base model for saliency segmentation.

In SBUT-nets, hyper-parameters such as learning rate, epochs, input bias weights and momentum are fine-tuned to segment the distant objects from the multiple views of videos. For annotation and masked preparation, MVS datasets from office [38] are taken as references and labelling has been done by the one-hot encoding technique. For effective segmentation, the encoder is constructed with three convolutional layers, two kernels, and five pooling layers while the decoder is also designed in a similar fashion. The residual networks are used as skip-connection for accumulating more features that can aid in an effective segmentation. The fine-tuned hyper parameter that is used for constructing SBUT-Nets are listed in Table 2

Table 2

Fine-tuned Hyper-parameter used for constructing the SBUT-nets
Sl.No	Hyper parameters	Fine-tuned parameters
1	No of Epochs	1250
2	No of hidden layers	200
3	Learning rate	0.0014
4	Momentum	0.1
5	Batch size	50

3.3 Deep Non-Overlapping Feature Extraction:

This section demonstrates the importance of the non-overlapping feature extraction mechanism used in the proposed framework. To investigate the deeper feature extraction method, this research uses the capsule networks and Bi-gated attention networks (GAN) to exploit the spatio-temporal feature extraction. In previous studies [39], In order to solve the visual representation of pictures, Convolutional Neural Networks (CNN) have become more popular. These CNN, however, have serious flaws in their fundamental design, which negatively impact their performance in key tasks. CNN automatically extracts characteristics from the photos, and using these features, it can learn to recognize and recognize various visual things. Simple features like edges are extracted by early layers, and as the number of layers rises, the learning process gets more complicated. The final step is for CNN to predict the visual objects using all retrieved information. During this process. The spatial information that is recorded in the item is not taken into account by CNN; it only records the characteristics that are visible. The CNN design is seen to have a significant flaw in that it lacks the spatial information provided by the MVS. Given the necessity for more precision in the input raw data, CNN must make its own adjustments to obtain greater accuracy. In order to improve feature extraction, capsule networks are used in place of CNN's pooling layers to address the issues outlined above. A capsule network is suggested in this study paper for a more effective extraction of spatial features. The recently suggested Capsule Network [40] has been put up to alleviate this CNN network constraint. Neuronal clusters known as capsules store spatial data and the likelihood that an object will be present. Each entity in a picture has its own capsule in the network of capsules, and each capsule provides:

An entity's probability of existence
The instantiation conditions for entities

A low capsule layer, a high capsule layer, and a classification layer make up the three layers that make up the capsule network. A dynamic routing technique that has been optimized is utilized to update the parameters repeatedly while global parameter sharing is employed to decrease the buildup of mistakes. The calculation of the input vectors' input matrix multiplied by the weight matrix encodes the crucial spatial relationship between low-level and high-level convolutional features inside the picture.

$$Y\left(i.j\right)={W}_{i,j }U\left(i,j\right)*{S}_{j}$$

The present status of capsules and which upper level capsules receive their output are determined by adding the weighted input vectors.

$$S\left(j\right)={\sum }_{j}Y\left(i,j\right)*D\left(j\right)$$

The squash function is then utilized to apply non-linearity. The squashing function reduces a vector's directions while reducing its length to an upper limit of one and a lowest of zero.

$G\left(j\right)=squash\left(S\right(j)$ ) (3)

Capsule networks are able to gather data from several locations and use Eq. (1) to determine the link between the characteristics for efficient spatial information extraction. In the region of low and high capsules, respectively, convolution layers are used. The quantity of convolutional layers applied to the lower and main capsule areas is shown in Table 3. Eq. (2) is used to determine the output weights, which are then sent to the high capsule area. Eq. (3)'s squash function, on the other hand, compresses the vector's length to the value (0, 1) while preserving the vector's original orientation. The dot product between related capsules and outputs is included into the proposed model utilizing optimized dynamic routing in the next stage. The feature maps are formed using this procedure, which iteratively changes the network weights. Then, using layers that have been fully converted, the dimension of feature maps is decreased, and using layers that have been flattened, the dimension is lowered to create single-dimensional feature maps. Eq. (4) is the mathematical expression for the characteristics.

$$Z=F\left(G\left(j\right),P\right)$$

As discussed above, the activation function used in the network is LeakyRelu in the pre-convolution block connected with the capsule network which in turn increases the local features that aid for the better extraction of spatial features from the multi-view video frames. Table 3 gives the parameters used to model the Pre-convolutional Blocks.

Table 3

Parameters used for modelling the Pre-convolutional Blocks.
Sl.No	Convolutional layer’s count	Length of the stride	Layers count
1	Convol(2d)_Layer_1	1	3x3
2	Max_Pooling_layers_1	1	2x2
3	Convol(2d)_Layer_2	1	3x3
4	Max_Pooling_layers_2	1	2x2
5	Activation Function	LRelu	-----

3.4 Gated Recurrent Networks

Long short term memory (LSTM) is thought to have a particularly fascinating variety, called GRU which is shown in Fig. 3. This concept, tries to merge the forget gate and input vector as a single vector [41]. In addition to supporting extended memories, this network also supports long term patterns. Compared to the LSTM network, there is a significant reduction in complexity.

Following equations are coined by Chung to represent the characteristics of GRU

$${h}_{t}=\left(1-{x}_{t}\right) ⨀ {h}_{t-1}+{x}_{t} ⨀{ h}_{t}$$

$$\stackrel{\sim}{{h}_{t}}=g({W}_{h}{x}_{t}+ {U}_{h}({r}_{t} ⨀ {h}_{t-1})+ {b}_{h}$$

$${z}_{t}=\sigma ({W}_{h}{x}_{t}+{U}_{z}{h}_{t-1}+ {b}_{z})$$

$${r}_{t}=\sigma ({W}_{h}{x}_{t}+{U}_{r}{h}_{t-1}+ {b}_{r})$$

The overall GRU characteristic equation is represented by

$$P=GRU\left(\sum _{t=1}^{n}[{x}_{t,}{h}_{t,}{z}_{t,}{r}_{t}\left(W\left(t\right),B\left(t\right),\eta \left(tanh\right)\right)\right]$$

where $W\left(t\right)$ stands for weights, and B(t) stands for bias weights at the current instant. ${x}_{t,}$ stands for the input feature at the current state, ${z}_{t,}$stands for the output state, and ${h}_{t,}$ stands for the module's output at the current instant.

3.5 Self Attention Maps:

The attention map was first presented in 2014 to specify the correct terminology for sequence-to-sequence architecture [42]. In the majority of recent research, attention layers have been included to simulate redundant characteristics that can help with the accurate classification method. For each input sequence, the three vectors “Q, K, and V” are created as part of the self-attention mechanism, also referred to as the intra-attention mechanism. As a result, each layer's input sequences are turned into output sequences. To put it another way, it is a method that uses scaled dot functions to map the Query with the collection of key pairs. The dot product for self-attention is calculated using the following formula in mathematics.

$$F\left(K,Q\right)=(\left(K\right),{Q}^{T})/\left({V}_{K}\right)^0.5$$

3.6 Self-Attention (SA) Evoked Bi-Gated Recurrent Networks :

GRU networks are built as a BiGRU network, which consists of forward and backward GRU, to extract the temporal information from the raw sine waves. Eq. (13) provides information about the mathematical properties of the BiGRU network. The temporal characteristics that comprise the variety of information needed for PU identification are extracted by the BiGRU network. However, these elements include a wider range of data that could have an impact on the training period and cause a high degree of delay when identifying the main users. SA layers are incorporated between the BiGRU network and classification layer in order to reduce this detection overhead. Eq. (10), which generates attention features using the input features from the BiGRU network, is then applied to the softmax layer, which is subsequently sent to the feed forward classification layers.

$$P\left(F\right)=GRU\left(\sum _{t=1}^{n}[{x}_{t,}{h}_{t,}{z}_{t,}{r}_{t}\left(W\left(t\right),B\left(t\right),\eta \left(tanh\right)\right)\right] \left(11\right)$$

$$P\left(B\right)=GRU\left(\sum _{t=1}^{n}[{x}_{t,}{h}_{t,}{z}_{t,}{r}_{t}\left(W\left(t\right),B\left(t\right),\eta \left(tanh\right)\right)\right] \left(12\right)$$

Combining the Eqs. (11) and (12)

$$P\left(BiGRU\right)=P\left(F\right)+P\left(B\right) \left(13\right)$$

Integrated SA with BiGRU Feature Extraction is mentioned as follows

$$Y=Softmax \left(P\left(BiGRU), F\left(K,Q\right)\right) \right(14)$$

These features are then fed as the input to the training network which can differentiate the different objects in the videos that aid for video summarization. Table 4 illustrates the parameters used for constructing the GAN networks.

Table 4

Parameters used for constructing the Capsule-GAN frameworks.
Sl.No	Hyper parameters	Fine-tuned parameters
1	No of GRU cells	310
2	No of Epochs	700
3	Learning rate	0.0014
4	Training/Testing/Validation	70/20/10
5	Optimization Algorithm	ADAM

Since this deep spatio-temporal feature extraction consumes a lot of computational resources, the proposed framework incorporates federated training on the MV data to save time and resources for servers by maintaining the privacy of the MV data.

3.7 Federated Learning for the Proposed Feature Extraction:

According to a study of the literature, federated learning is a distributed method of machine learning in which a variety of clients train a single global network using local data under the control of a central server or cloud. The shared global model is the sole update that each client provides to the central server, in contrast to typical centralized learning systems, which sacrifice privacy and require costly computations. This improves training performance. In this study, the local model is trained on NVIDIA edge servers, and the local model and the global model are both finalized using servers and the cloud. The federated learning in the Algorithm-1 suggested framework is carried out using the following processes:

Selection of Edge Client: The server must inform all client nodes engaged in model training. Based on the types of data, model training, and data processing capabilities of the edge client server, its capacity is chosen.
Distribution of model: The initial weights are sent to the edge clients by the server after the client node has been chosen in order to enable distributed learning. The input bias weights (${\varvec{W}}^{\varvec{u}}$) of the classifiers are distributed among the edge client nodes in the framework that is being suggested.
Distributed learning: Each client updates the model on the local level by sending it to the central server after training it locally with its own data
Model Aggregation: The central server creates the new global model by averaging all the updated weights (${\varvec{W}}^{\varvec{l}}$) from the client nodes. To get the global model in this study, an ADAM-based network model was applied.
Model testing: The completed global model is tested using the datasets and its testing performance is determined on the central server.
Update of the Shared Model: The Client Edge Nodes' aggregated models are used by the Server to update the Shared Model. By using math, the revised weights are updated by the

$${\beta }={\varvec{W}}^{\varvec{u}}- {\varvec{W}}^{\varvec{l}}$$

Where, respectively ${\varvec{W}}^{\varvec{u}}$ and ${\varvec{W}}^{\varvec{l}}$ are regarded as distributed weights of the global model and the local model.

3.8 Federated Summarization Technique :

The final summary of the MV is generated in the cloud server station. Since the self-attention maps are used for generating the features and federated learning is used for the classification of different activities, these frames are grouped in accordance with the activities and a summary is generated based on predicted outputs from the learning models. Figure 4 illustrates the federated learning for MVS. Once the activity is predicted, these frames are clustered and identified with their own tag ID. Hence the video summarization is made complex-free by integrating the self-attention and a federated learning process.

This part conducts an experimental evaluation of the Federated supported MVS architecture from two angles, namely complexity and aim. Utilising the various assessment indicators listed in Table 5, the findings were produced. In order to establish the suggested framework's superiority, real-time office datasets are used for evaluation and comparison with existing cutting-edge intelligent MVS frameworks.

4.1 Experimental Settings

Using NVIDIA Jetson Nano boards as edge devices and well equipped laptops as cloud servers, the performance of the suggested architecture is assessed. “On a laptop with an Intel Xeon CPU, an NVIDIA Titan GPU, 16GB RAM, and a 3.5 GHz operating frequency, the entire model was constructed using open source TensorFlow version 2.1.0 with Keras as the backend”. In NVIDIA Jetson Nano, Google’s TensorFlow and Keras are used for training the local models. Figure 5 and Table 5 illustrate the hardware environment used for experimentation. As shown in Fig. 5, four independent cameras interfaced with Raspberry Pi 4 Model B + is used independently to capture the images from the environment.

Table 5

Specification Used for Experimental Environment
Sl.no	Hardware Environment	Specification
1	No of Edge Devices	03
2	No of Cloud Server	01
3	Edge Devices	NVIDIA JETSON NANO
4	Raspberry pi Model B + Boards	04
5	High End Cameras	04
6	Router	01

4.2 Dataset Description :

Real-time office datasets, which are used by the majority of MVS deployment and assessment approaches, are the datasets used for evaluation. The four steadily held cameras used to construct this dataset were not synced and were located in an office. In this dataset, the quality of the movies that were recorded from various viewpoints is affected by a variety of lighting situations. The datasets are formulated based on the videos captured by the experimental setup (as mentioned in Section 4.1) installed on the different floors of the offices. Nearly 45 days of videos are collected in the different scenarios of the environment and stored in the cloud server station. Table 6 illustrates the different datasets collected and Fig. 6 shows the sample datasets used for training and validation.

Table 6

Collection of Datasets used for Experimentation
Floor Details	No of cameras	No of videos	Labelled as
Floor-1	04	1728	Office-0
Floor-2	04	1634	Office-1
Floor-3	04	1562	Office-2
Floor-4	04	1620	Office-3
	Total	6544

4.3 Performance Evaluation Metrics :

Performance indicators including “accuracy, specificity, recall, accuracy, and F1-score” are used to assess the proposed design. These measurements show how well the model is able to distinguish between people who recognize things and those who don't. The formulas used to calculate the performance measures are listed in Table 7.

Table 7

Formulae for performance evaluation
Formulae	Numerical expressions
Accuracy	$\frac{TP+TN}{TP+TN+FP+FN}$
Sensitivity or recall	$\frac{\text{T}\text{P}}{\text{T}\text{P}+\text{F}\text{N}}$x100
Specificity	$\frac{TN}{TN+FP}$
Precision	$\frac{TN}{TP+FP}$
F1-Score	$2.\frac{Precison*Recall}{Precision+Recall}$

TP is True Positive Values, TN is True Negative Values, FP is False Positive and FN is False negative values.

High precision, accuracy, and recall are requirements for the proposed MVS. In this research study, the early halting method is employed to address the issue of model overfitting and to enhance the generalization problem. This technique may be used to halt the planned network training when the validation performance doesn't increase for N consecutive periods. Each dataset is divided into several scenarios for testing, training, and validation, accordingly.

Figures 7 to Fig. 9 show the stability of the proposed federated framework to handle the increasing datasets from 70–90%. Consequently, it is clear that the performance of the model is stable even though the datasets have grown.

4.4 Comparative Analysis and Discussion :

Comparative study is done between the various deep learning algorithms such as TAC-NETS [43], RESNET-34 [44], MULTI-CNN [45], CNN-LSTM [46], CNN-BILSTM [47], CNN-GRU [48] (derived from similar publications) utilized for building the MVS approaches in order to demonstrate the superiority of the proposed framework.

The performance metrics of several MVS frameworks built on deep learning in managing multi-view datasets are shown in Figs. 10 to 14 making it clear that the suggested model's accuracy is determined to be high, whereas TAC-NETS has provided the least performance. The suggested model outperformed CNN-BiLSTM and CNN-GRU, though only marginally. Obviously, the federated learning framework with capsule and gated recurrent network has produced the best performance and outperformed the other DL based MVS framework.

4.5 Time Complexity Analysis :

Table 8 outlines the time-consuming nature of producing summaries using various deep learning techniques. In Table 8, federated learning based MVS techniques have also been analysed. From Table 8, the federated learning based framework has produced the edge over the other algorithms. Furthermore, capsule and self-attention GRU networks have produced a considered the little complexity than the other algorithms implemented in federated scenarios. Moreover, the time complexity of the proposed federated framework has been compared with the other existing models in which FED-AT-VIDEO nets have produced lesser complexity than the others which is evident from Table 9.

Table 8

Time Complexity Analysis
Type of Algorithm	Type of training	Complexity time(secs)	Communication time(secs)
TAC-NETS	Local Training	3483.56	N/A
RESNETS-34		4783.56	N/A
Multi-CNN		4356.5	N/A
CNN-LSTM		3890.34	N/A
CNN-BILSTM		4089.56	N/A
CNN-GRU		3200.56	N/A
PROPOSED MODEL	Federated Training	2673.56	345.78
Type of Algorithm	Type of training	Complexity time(secs)	Communication time(secs)
TAC-NETS	Federated Training	3208.6	678.31
RESNETS-34		4532.67	734.90
Multi-CNN		4002.5	778.50
CNN-LSTM		3521.89	754.90
CNN-BILSTM		3783.78	832.90
CNN-GRU		3002.56	673.9
PROPOSED MODEL	Federated Training	2673.56	345.78

Table 9

Comparative Analysis of Different Algorithms in terms of Time Complexity
Type of Algorithm	Complexity time(secs)	Communication time(secs)
FedAvg [49]	3208.6	678.31
FEVA-Nets [50]	4532.67	734.90
Fed-Res [51]	4002.5	778.50
Proposed Model	2673.56	345.78

4.6 Subjective Evaluation Analysis

This section discusses about the subjective analysis of the proposed framework. Six individuals with a background in Mater of Engineering (Computer Vision) were chosen to participate in the subjective analysis, which measured the quantity of information contained in the final summary produced by the suggested framework. The participants were asked to submit the results after 3 days of summary analyses. The parameters such as enjoyability, informativeness and usefulness are used to indicate the performance of the model during the analysis by the participants. A scale of 1–10 is used to analyse the user experiences in which 0 stands for poor performance and 10 represents excellent performance. Table 10 presents the statistical data results presented by the 6 participants.

Table 10

Statistical Analysis of the Subjective Analysis from different user
Parameters	User-1	User-2	User-3	User-4	User-5	User-6
Enjoyability	8.6	8.7	8.5	8.6	8.8	8.5
In formativeness	9.2	9.0	8.8	9.0	9.1	8.9
Usefulness	9.0	9.0	8.9	8.8	9.0	8.9

In today’s world of technology, surveillance networks are installed everywhere to monitor human behaviours and even to prevent criminal activities in society. These networks have now become a pivotal role in day-to-day life that can generate a huge amount of videos which are then analysed to overcome the above said challenges. To overcome this above challenge, this research article proposed the most efficient federated learning framework which integrates the capsule and self–attention mechanism. The ultimate aim of the proposed model is to generate the non-overlapping redundant summarization with less time complexity. The proposed framework amalgamates the Saliency Tuned U-NETS (for Segmentation), capsule and self-attention gated networks are used for the better feature extraction and activity recognition. Furthermore, federated learning has been introduced to handle larger datasets with lesser time complexity. The experimentation has been carried out using real time multi-view videos collected from Raspberry Pi boards with federated learning implemented in the NVIDIA JETSON Nano edge boards. The evaluation is carried out using the different performance metrics and compared with the other deep learning based MVS frameworks. Experimental results demonstrate that the proposed framework has shown its dominance over the other learning models in terms of performance, stability, user experience and time complexity.

Since the current framework has not included any appropriate mechanism for human behaviour detection, the proposed framework needs a greater probability of design attraction in human behavioural detection. In future, this integration in the framework will reduce human crimes in society.

Ethical Approval: Not Applicable

Competing interests: The author(s) declare(s) that they have no competing interests.

Authors' contributions

VIJAY ANAND KADASWAMY as a researcher has made a substantial contribution to the concept or design of the article, analysis, interpretation of data collection, development of the algorithm, experiment, results, and manuscript preparation.

Dr BHUVANESWARI BALACHANDER guided and reviewed the result and manuscript preparation.

Funding : Not applicable

Availability of data and materials: The data that support the findings of this study are available from the corresponding author upon reasonable request.

A. Gluhak, S. Krco, M. Nati, D. Pfisterer, N. Mitton, and T. Razafindralambo, “A survey on facilities for experimental Internet of Things research,” IEEE Commun. Mag., vol. 49, no. 11, pp. 58–67, Nov. 2011.
L. Atzori, A. Iera, and G. Morabito, “The Internet of Things: A survey,” Comput. Netw., vol. 54, pp. 2787–2805, 2010.
R. Khan, S. U. Khan, R. Zaheer, and S. Khan, “Future Internet: The Internet of Things architecture, possible applications and key challenges,” in Proc. 10th Int. Conf. Frontiers Inf. Technol., 2012, pp. 257–260.
J. Gubbi, R. Buyya, S. Marusic, and M. Palaniswami, “Internet of Things (IoT): A vision, architectural elements, and future directions,” Future Gener. Comput. Syst., vol. 29, pp. 1645–1660, 2013.
R. V. Kulkarni, A. Forster, and G. K. Venayagamoorthy, “Computational intelligence in wireless sensor networks: A survey,” IEEE Commun. Surv. Tut., vol. 13, no. 1, pp. 68–96, 1Q 2011.
D. E. O’Leary, “‘Big data’, the ‘Internet of Things’ and the ‘Internet of Signs’,” Intell. Syst. Accounting, Finance Manage., vol. 20, pp. 53–65, 2013
M. Thibaud, H. Chi, W. Zhou, and S. Piramuthu, “Internet of Things (IoT) in high-risk environment, health and safety (EHS) industries: A comprehensive review,” Decis. Support Syst., vol. 108, pp. 79–95, 2018.
J. A. Guerrero-ibanez, S. Zeadally, and J. Contreras-Castillo, “Integration challenges of intelligent transportation systems with connected vehicle, cloud computing, and Internet of Things technologies,” IEEE Wireless Commun., vol. 22, no. 6, pp. 122–128, Dec. 2015.
T. N. Pham, M.-F. Tsai, D. B. Nguyen, C.-R. Dow, and D.-J. Deng, “A cloud-based smart-parking system based on Internet-of-Things technologies,” IEEE Access, vol. 3, pp. 1581–1591, 2015.
T. Wang et al., “Generative neural networks for anomaly detection in crowded scenes,” IEEE Trans. Inf. Forensics Security, vol. 14, no. 5, pp. 1390–1399, May 2019
K. M. Mahmoud, M. A. Ismail, and N. M. Ghanem, “VSCAN: An enhanced video summarization using density-based spatial clustering,” in Proc. Int. Conf. Image Anal. Process., 2013, pp. 733–742.
B. Xu, X. Wang, and Y.-G. Jiang, “Fast summarization of user-generated videos: Exploiting semantic, emotional, and quality clues,” IEEE MultiMedia, vol. 23, no. 3, pp. 23–33, Jul./Sep. 2016.
M. Fei, W. Jiang, and W. Mao, “Memorable and rich video summarization,” J. Vis. Commun. Image Representation, vol. 42, pp. 207–217, 2017.
A. S. Murugan, K. S. Devi, A. Sivaranjani, and P. Srinivasan, “A study on various methods used for video summarization and moving object detection for video surveillance applications,” Multimedia Tools Appl., vol. 77, pp. 23273–23290, 2018.
M. Fei, W. Jiang, and W. Mao, “Memorable and rich video summarization,” J. Vis. Commun. Image Representation, vol. 42, pp. 207–217, 2017.
A. S. Murugan, K. S. Devi, A. Sivaranjani, and P. Srinivasan, “A study on various methods used for video summarization and moving object detection for video surveillance applications,” Multimedia Tools Appl., vol. 77, pp. 23273–23290, 2018.
F. Murtaza, M. H. Yousaf, and S. A. Velastin, “Multi-view human action recognition using 2D motion templates based on MHIs and their HOG description,” IET Comput. Vis., vol. 10, pp. 758–767, 2016
Y. Li and B. Merialdo, “Multi-video summarization based on VideoMMR,” in Proc. 11th Int. Workshop Image Anal. Multimedia Interactive Serv., 2010, pp. 1–4.
S. H. Ou et al., “Communication-efficient multi-view keyframe extraction in distributed video sensors,” in Proc.IEEE Vis. Commun. Image Process. Conf., 2014, pp.13–16.
A. Mahapatra, P. K. Sa, and B. Majhi, “A multi-view video synopsis framework,” in Proc. IEEE Int. Conf. Image Process., 2015, pp. 1260– 1264.
R. Panda, A. Dasy, and A. K. Roy-Chowdhury, “Video summarization in a multi-view camera network,” in Proc. 23rd Int. Conf. Pattern Recognit., 2016, pp. 2971–2976.
Y. Jia et al., “Caffe: Convolutional architecture for fast feature embedding,” in Proc. 22nd ACM Int. Conf. Multimedia, 2014, pp. 675–678.
R. Panda, A. Das, and A. K. Roy-Chowdhury, “Embedded sparse coding for summarizing multi-view videos,” in Proc. IEEE Int. Conf. Image Process., 2016, pp. 191–195
A. Triastcyn and B. Faltings, “Federated learning with bayesian differential privacy,” in Proc. IEEE Big Data, 2019, pp. 2587–2596.
J. Zhang, B. Chen, S. Yu, and H. Deng, “PEFL: A privacy-enhanced federated learning scheme for big data analytics,” in Proc. IEEE Global Commun. Conf., 2019, pp. 1–6.
S. Sharma, C. Xing, Y. Liu, and Y. Kang, “Secure and efficient federated transfer learning,” in Proc. IEEE Big Data, 2019, pp. 2569–2576
Mayuri Karvande,Apoorv Katkar,Nikhil Koli,Amit Joshi,”Parallel Deep Learning Frame work for Video Surveillance System” In book: Recent Trends in Intensive Computing, December 2021, DOI:10.3233/APC210191
T. Barbu, "Multiple Pedestrian Tracking Framework using Deep Learning-based Multiscale Image Analysis for Stationary-camera Video Surveillance," 2022 IEEE International Smart Cities Conference (ISC2), Pafos, Cyprus, 2022, pp. 1-7, doi: 10.1109/ISC255366.2022.9922217.
C. Huang, Z. Wu, J. Wen, Y. Xu, Q. Jiang and Y. Wang, "Abnormal Event Detection Using Deep Contrastive Learning for Intelligent Video Surveillance System," in IEEE Transactions on Industrial Informatics, vol. 18, no. 8, pp. 5171-5179, Aug. 2022, doi: 10.1109/TII.2021.3122801.
Tsai, Ming-Fong, Pei-Ching Lin, Zi-Hao Huang, and Cheng-Hsun Lin. 2020. "Multiple Feature Dependency Detection for Deep Learning Technology—Smart Pet Surveillance System Implementation" Electronics 9, no. 9: 1387. https://doi.org/10.3390/electronics9091387.
Y. Zahid, M. A. Tahir and M. N. Durrani, "Ensemble Learning Using Bagging And Inception-V3 For Anomaly Detection In Surveillance Videos," 2020 IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates, 2020, pp. 588-592, doi: 10.1109/ICIP40778.2020.9190673.
C. Kyrkou, "Imitation-Based Active Camera Control with Deep Convolutional Neural Network," 2020 IEEE 4th International Conference on Image Processing, Applications and Systems (IPAS), Genova, Italy, 2020, pp. 168-173, doi: 10.1109/IPAS50080.2020.9334958.
C. Sonavane, P. Kulkarni, O. Podey and P. Rewane, "Smart Surveillance and Tracking System using Resnet and Tesseract-OCR," 2021 IEEE Pune Section International Conference (PuneCon), Pune, India, 2021, pp. 1-6, doi: 10.1109/PuneCon52575.2021.9686493.
D. Kim and C. M. Yang, "Reinforcement Learning-based Multiple Camera Collaboration Control Scheme," 2022 Thirteenth International Conference on Ubiquitous and Future Networks (ICUFN), Barcelona, Spain, 2022, pp. 414-416, doi: 10.1109/ICUFN55119.2022.9829605.
C. -Y. Wang, P. -Y. Chen, M. -C. Chen, J. -W. Hsieh and H. -Y. M. Liao, "Real-Time Video-Based Person Re-Identification Surveillance with Light-Weight Deep Convolutional Networks," 2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Taipei, Taiwan, 2019, pp. 1-8, doi: 10.1109/AVSS.2019.8909855.
Chien-Hao Tseng, Chia-Chien Hsieh, Dah-Jing Jwo, Jyh-Horng Wu, Ruey-Kai Sheu, Lun-Chi Chen, "Person Retrieval in Video Surveillance Using Deep Learning–Based Instance Segmentation", Journal of Sensors, vol. 2021, Article ID 9566628, 12 pages, 2021. https://doi.org/10.1155/2021/9566628
H. Zia, A. Niaz and K. N. Choi, "Active Contour Model for Image Segmentation," 2022 Asia Conference on Advanced Robotics, Automation, and Control Engineering (ARACE), Qingdao, China, 2022, pp. 13-17, doi: 10.1109/ARACE56528.2022.00011.
Z. Tian, R. Wang, Z. Wang and R. Wang, "HQP-MVS:High-Quality Plane Priors Assisted Multi-View Stereo for Low-Textured Areas," ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Rhodes Island, Greece, 2023, pp. 1-5, doi: 10.1109/ICASSP49357.2023.10096871.
M. Junaid et al., "Multi-Feature View-Based Shallow Convolutional Neural Network for Road Segmentation," in IEEE Access, vol. 8, pp. 36612-36623, 2020, doi: 10.1109/ACCESS.2020.2968965.
F. Kınlı, B. Özcan and F. Kıraç, "Fashion Image Retrieval with Capsule Networks," 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Korea (South), 2019, pp. 3109-3112, doi: 10.1109/ICCVW.2019.00376.
J. C. Heck and F. M. Salem, "Simplified minimal gated unit variations for recurrent neural networks," 2017 IEEE 60th International Midwest Symposium on Circuits and Systems (MWSCAS), Boston, MA, USA, 2017, pp. 1593-1596, doi: 10.1109/MWSCAS.2017.8053242.
R. R. A. Pramono, Y. -T. Chen and W. -H. Fang, "Hierarchical Self-Attention Network for Action Localization in Videos," 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), 2019, pp. 61-70, doi: 10.1109/ICCV.2019.00015.
Luchuan Song, Bin Liu, Guojun Yin, Xiaoyi Dong, Yufei Zhang, and Jia-Xuan Bai. 2021. TACR-Net: Editing on Deep Video and Voice Portraits. In Proceedings of the 29th ACM International Conference on Multimedia (MM '21). Association for Computing Machinery, New York, NY, USA, 478–486. https://doi.org/10.1145/3474085.3475196
C. Sonavane, P. Kulkarni, O. Podey and P. Rewane, "Smart Surveillance and Tracking System using Resnet and Tesseract-OCR," 2021 IEEE Pune Section International Conference (PuneCon), Pune, India, 2021, pp. 1-6, doi: 10.1109/PuneCon52575.2021.9686493.
X. Li, S. Wang, C. Zhu, L. Song, R. Xie and W. Zhang, "Viewport Prediction for Panoramic Video with Multi-CNN," 2019 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), Jeju, Korea (South), 2019, pp. 1-6, doi: 10.1109/BMSB47279.2019.8971933.
W. Iqrar, M. Z. Abidien, W. Hameed and A. Shahzad, "CNN-LSTM Based Smart Real-time Video Surveillance System," 2022 14th International Conference on Mathematics, Actuarial Science, Computer Science and Statistics (MACS), Karachi, Pakistan, 2022, pp. 1-5, doi: 10.1109/MACS56771.2022.10023018.
Z. Li, S. Zhang, J. Yin, M. Du, Z. Zhang and Q. Liu, "Fighting Against Piracy:An Approach to Detect Pirated Video Websites Enhanced by Third-party Services," 2022 IEEE Symposium on Computers and Communications (ISCC), Rhodes, Greece, 2022, pp. 1-7, doi: 10.1109/ISCC55528.2022.9912777.
N. -W. Kwong, Y. -L. Chan, S. -H. Tsang and D. P. -K. Lun, "Quality Feature Learning via Multi-Channel CNN and GRU for No-Reference Video Quality Assessment," in IEEE Access, vol. 11, pp. 28060-28075, 2023, doi: 10.1109/ACCESS.2023.3259101.
S. Xing, Z. Ning, J. Zhou, X. Liao, J. Xu and W. Zou, "N-FedAvg: Novel Federated Average Algorithm Based on FedAvg," 2022 14th International Conference on Communication Software and Networks (ICCSN), Chongqing, China, 2022, pp. 187-196, doi: 10.1109/ICCSN55126.2022.9817607.
C. Hu, R. Lu and D. Wang, "FEVA: A Federated Video Analytics Architecture for Networked Smart Cameras," in IEEE Network, vol. 35, no. 6, pp. 163-170, November/December 2021, doi: 10.1109/MNET.001.2100261.
T. Liu, Q. Meng, J. -J. Huang, A. Vlontzos, D. Rueckert and B. Kainz, "Video Summarization Through Reinforcement Learning With a 3D Spatio-Temporal U-Net," in IEEE Transactions on Image Processing, vol. 31, pp. 1573-1586, 2022, doi: 10.1109/TIP.2022.3143699.

No competing interests reported.

Download PDF

Editorial decision: Major revision
13 Aug, 2023
Reviews received at journal
12 Aug, 2023
Reviewers agreed at journal
07 Aug, 2023
Reviewers invited by journal
07 Aug, 2023
Submission checks completed at journal
05 Aug, 2023
Editor assigned by journal
05 Aug, 2023
First submitted to journal
03 Aug, 2023

You are reading this latest preprint version

FED-AT-VIDEO NETS - A Federated Capsule – Self Gated Learning Architecture for the Multi-View Video Summarization Technique.

Status:

Version 1

Abstract

Figures

1 Introduction

2 Related Works

3 Proposed FED-AT-VIDEO NETS.