Fuzzy Rule Based Video Watermarking in DWT-SVD Domain

This paper addresses the issues in video copyright using DWT and SVD. The prevailing algorithms countermeasure various attacks and they do not contemplate on the redundancy of frames in the video. Proposed methodology focuses on the identification of non-redundant frames by introducing a fuzzy model for reducing the processing time. The frequently changed scenes are identified by scene change detection algorithm. The key frames are effectively identified from each scene by fuzzy rules using entropy, absolute mean difference and absolute difference of frame variance of the video frames. DWT is applied to the key frames. The watermark image is divided into number of blocks based on the number of key frames selected in the scene. The order of embedding the watermark block in each scene is different. The SVD is applied to the key frames and watermark. In the embedding process, the singular values of key frame are added to the Principal Component (PC) of the watermark bock. The experimental results show that the proposed methodology is resilient to image processing, frame based attacks and also resolves the false positive problem as well as improves the robustness and imperceptibility of video and watermark.


Introduction
Due to high speed computer networks and the rapid development of the Internet and the World Wide Web the usage of multimedia data has increased. The way in which digital content is transmitted across the web results in unlimited duplicate copy. Maintaining the ownership of data becomes difficult. Digital watermarking is a technique used to protect copy right. A watermark is a digital data embedded in multimedia objects like image, audio and video. Video watermarking is used to provide authentication for the video data by embedding digital data into video sequence. Image watermarking techniques can be applied for video watermarking, but due to the redundancy of data it exhibits some additional properties. A video may be subjected to attacks such as frame dropping, frame averaging, frame swapping, etc. therefore the watermark method should be robust against all these attacks [1][2][3].
Digital watermarks and their techniques can be subdivided into various categories; it can be classified according to the application, source type (image watermarks, video watermarks, audio watermarks, and text watermarks), human perception, and technique used [1][2][3][4][5][6]. The quality of the digital watermarking technique can be measured on the basis of certain properties such as fidelity, robustness, fragility and complexity [4][5][6].

State of the art
In recent times, numerous techniques are devised for protecting video data. Each method has its merits and limitations in terms of quality, imperceptibility and robustness. Some of the techniques and its merits and demerits are discussed as follows.
Majid masoumi et al [7], proposed a technique for video ownership verification based on spatio-temporal data hiding in frequency domain. In this paper, the 3D DWT was performed in the video frame. The CDMA with two random number sequences were used. The binary image was used as a watermark, and then it is embedded into the high and middle frequency sub bands of wavelet. The motion frames are considered as a key frame. This method survives the frame based attacks and addition of noise. Amlan Karmakar et al [8] suggested a watermarking method for video data using DCT. The Zernike moments of the squared watermark block was embedded into middle of the luminance component. This method withstands the rotation, collusion and video processing attacks. Tanima et al [9] proposed a watermarking scheme for video coding. The process of embedding and extraction was done in compressed domain. The watermark was embedded into intra predicted blocks of the video. This algorithm withstands attacks such as filtering, addition of noise, compression.
Anjaneyulu Sake et al [10] proposed a video watermarking based on bi-orthogonal wavelet transform and optimization techniques. Artificial Bee colony algorithm was chosen for the selection of frames for embedding and extraction. Agilandeeswari et al [11] proposed a video watermarking method based on BAM neural networks and fuzzy system. In this algorithm DWT was applied into the video frames. The weighted matrix was embedded into the middle frequency of all available frames in the video. The embedding strength α was different for each frame, controlled by fuzzy system. Ejima et al [ 12] proposed a method, based on wavelet packets. Here the same watermark was inserted into all the available frames in the video. This method does not withstand the following attacks namely frame averaging, swapping and filtering. Dewan [13 ], proposed an algorithm based on DWT. In this method the 3D blocks of the video are taken for embedding process. All the frames in the video are taken for watermarking process. This algorithm shows resistant for certain image and video processing attacks except filtering attack. Radu O. Preda et al [14] proposed a method for video copyright. Here the binary image was embedded into the detail coefficients of the middle frequency band in all available frames in the video. This algorithm does not withstand the video processing attacks. Osama S [15] proposed a method for video watermarking using DWT and SVD. The watermark with error correction code was embedded into the middle and high frequency subbands of entire frames in the video. This method withstands certain kinds of image processing attacks not for video processing attacks. Divjot Kaur Thind etal [16] proposed a watermarking technique for video based on DWT-SVD. The watermark was embedded in the HH sub band. Here all the frames are embedded with the same watermark. This algorithm was resistant for image processing attacks. It does not withstand for video processing attacks. Lama raja b et al [17] proposed a video watermarking technique based on SCHUR with DWT. The binary image was used as a watermark. It was embedded into the middle sub band of the randomly selected frames. This algorithm was only resistant for the attacks such as addition of noise and frame swapping and averaging. Ponni et al [31] proposed watermarking algorithm based on non-redundant frames and key frames. In this technique non-redundant frames and key frames were identified by mathematical computation hence the large video like movie, does not processed efficiently. This method was resistant against image and video processing attacks. Li et al [33] proposed a dual watermarking technique using contourlet and SIFT transform. This technique was resists for image processing and geometrical attacks.
Based on the above survey, the following challenges are identified  The watermark was embedded into the whole video, thereby increasing the processing time.  The whole watermark has been inserted into the video frames, which makes it easy for the attacker to notice the watermark by frame dropping attack  The existing methods only withstand basic image processing attacks not for geometrical and video processing attacks.  SVD based algorithms does not withstand for false positive attacks.  The redundancy of frames in the video is not deliberated.

Contributions
The aim of the proposed algorithm is to resolve the above mentioned challenges by the following contributions.
 Devised a new mechanism for identifying the non-redundant frames by implementing the fuzzy inference system for reducing the processing time.  Proposed a fuzzy rule based key frame identification algorithm for identify the suitable key frames for watermark embedding for improving the authentication.  Proposed a pseudo random number based watermark block processing for Segmenting the watermark into number of blocks for avoiding the occurrence of frame dropping attack.  Embedded the principal component of the watermark block to singular values of video frames for resolving the false positive attack. The paper is structured as follows: Section 2 explains the prefixes of the proposed watermarking system. Section 3 describes various experimental results and its discussions. Section 4 deals with the conclusion.

Proposed Methodology
Proposed methodology attempts to provide a solution for video security by engaging the PCA with DWT. The PCA technique is used to avoid the false positive attack. The video is preprocessed into number of frames and it is grouped into different scenes based on the absolute difference of frame variance (ADFV). In each scene the nonredundant frames are identified by proposed fuzzy based non-redundant frame identification algorithm (Fuzzy-NRFI).  Figure.1 Work flow of proposed system

No
The redundant and non-redundant frames [32] are separately stored into the redundant and non-redundant databases (RFDB and NRFDB. From the NRFDB, the appropriate key frames are identified by the proposed fuzzy based key frame selection algorithm (Fuzzy-KFS). The DWT is applied into the key frames, it decompose the frames into four subbands such as LL, HL, LH, HH. The middle band LH is selected for embedding process because it is stable compared to other subbands [30]. For improving the authentication level of the watermark data, the watermark is divided into number of blocks and different order of the blocks are embedded into the various scenes. The purpose of resolving the false positive attack, the principal component (PC) of the watermark block is used in the watermark embedding. The PC of the watermark block is embedded in to singular values of the key frames with suitable scaling factor. The process flow of proposed methodology is represented in Figure 1.

Scene change detection
The scene change detection technique is employed for detecting abrupt scene changes in the video. It is done by using Absolute Difference of Frame Variance (ADFV) method [18,19,32]. Frame variance is calculated by using the expressions as shown in Eq 1.
(1) FV n -Frame variance of n th frame The difference between the ADFV of consecutive frame is compared with the predefined threshold value. The scene change is controlled by threshold, which is calculated by using Otsu's method.

Fuzzy Inference System (FIS)
The process of fuzzy inference system is  All input values are fuzzified into membership functions.  The fuzzy output functions are computed by executing appropriate fuzzy rules.  The fuzzy output functions are defuzzified to produce crisp value for output. The fuzzy rules are if-then statements, it can make decision efficiently. In the proposed system uses the Mamdani fuzzy inference system. Gaussian membership function is adapted for processing because it is smooth and non-zero at all points. The defuzziification process is done by using centroid method. Here fuzzy linguistic terms are used for removing the redundant frames in a scene for reducing the processing time. The linguistic variables are SSIM (Structural Similarity Index), AMD (Absolute Mean Difference) and rank. The fuzzy membership functions of those attributes are low, medium and high. The fuzzy attributes and its ranges are represented in the Table 1. SSIM(x, y) = (2µ a µ b + k1)(2σ ab + k 1 ) ( µa 2 + µb 2 + k 1 )(σa 2 + σb 2 + k 2 ) SSIM metric is used to measure the perceived changes in structural information between the neighboring frames in a scene. It is calculated by Low 0.0-0.7 Medium 0.7-0.9 High 0.9-1.0

Rank
Rank is used to decide the range of nonredundant frames in the scene.
The fuzzy system is implemented by the MATLAB simulator. The details of fuzzy inference system such as linguistic variables, FIS type, and defuzzification method are represented in Figure 2.

Proposed algorithm for Identification of non-redundant frames using Fuzzy System
Initialize Set N= No of frames in Video Set Fuzzy Inference System: Mumdani system Set RFDB = Redundant frame Database. Set NRFDB = Non-Redundant frame Database.
For each frame 5.
Set Fuzzy Linguistic variable, Membership function and ranges 6.
Set Select the suitable Fuzzy rules: Rule 1. If (AMD is Low) and (SSIM is High) then (RANK is High)

Rule 2. If (AMD is Low) and (SSIM is Medium) then (RANK is Medium) Rule 3. If (AMD is Low) and (SSIM is Low) then (RANK is Medium) Rule 4. If (AMD is Medium) and (SSIM is High) then (RANK is High) Rule 5. If (AMD is Medium) and (SSIM is Medium) then (RANK is Medium) Rule 6. If (AMD is Medium) and (SSIM is Low) then (RANK is Low)
Rule 7. If (AMD is High) and (SSIM is High) then (RANK is Medium) Rule 8. If (AMD is High) and (SSIM is Medium) then (RANK is Low) Rule 9. If (AMD is High) and (SSIM is Low) then (RANK is Low) 10.

11.
END FOR

Key frame Selection:
Key frame selection is the process for detecting the suitable frames for embedding the watermark. It is identified by proposed Fuzzy based Key frame selection (Fuzzy-KFS) algorithm. The key frames should be non-redundant frames, which are selected from the non-redundant frame database (NRFDB). The input attributes of the FIS system are entropy, AMD and ADFV. The membership functions of those attributes are low, medium and high. The ranges of the membership functions are represented in Table 2. Based on the key frame identification algorithm, the key frames are selected in each scene for embedding the watermark block. The procedure is described in section 2.3.1.
Fuzzy inference system for key frame selection is shown in Figure.3.

Rank
Rank is used to decide the range of key frames in the scene.

IF (ENTROPY is Low) and (AMD is High) and (ADFV is High) then (RANK is High) END FOR END FOR
The key frames are identified in each scene based on the fuzzy rule from the non-redundant database. The fuzzy linguistic variables are Entropy, AMD and ADFV. The fuzzy rules are formulated by using linguistic variables and evaluated by membership function then the resulting inference is derived based on the rule. The high value of entropy, low value of AMD and ADFV are preferred for selecting the key frames, because the high entropy frame holds the maximum energy level and low ADFV and AMD frames maintain the maximum structural difference. Finally, the defuzzification process is performed to map the result to the crisp value. In total, 27 rules are formulated to predict the key frames in each scene.

Watermark embedding and extraction process:
It is the process of embedding the watermark into source video.The scene change detection algorithm is used to identify the various scenes in the video. In each scene, the non-redundant frames are identified by a proposed fuzzy based non-redundant frame identification algorithm. The redundant and non-redundant frames are stored separately in the database, namely RFDB and NRFDB. The key f rames are identified from the NRFDB by applying the proposed fuzzy-based keyframe selection algorithm (Fuzzy-KFS). The binary image is used as a watermark, divided into number of blocks equal to the number of key frames in a scene. These blocks are rearranged based on the pseudorandom number generator. In each scene, the watermarks are rearranged and embedded in the key frames. The DWT is applied to the key frames, the middle level subband LH is selected for watermarking process. Then the SVD is applied to the key frames and watermark to perform the embedding process. The singular values of the key frames are embedded to the principal component of the watermark block with a suitable scaling factor. Here, 0.47 is selected as a scaling value based on the survey [31]. The watermark extraction is the reverse process of embedding. The workflow is shown in Figure. 4. The process is described in the section 2.4.4.

DWT (Discrete Wavelet Transform)
The DWT is used to capture the frequency and location information about frames. It is applied into the key frames, which decompose it into LL, HH, HL and LH sub bands. The middle frequency band LH is more stable compared to other sub bands [15,16]. Hence it is selected for watermark embedding process.

SVD (Singular Value Decomposition)
SVD is a powerful mathematical tool used to analyze the matrix. Which decompose the matrix into three components such as U, S and V. U and V are the unitary matrices and S is the diagonal matric [17,18]. The SVD is applied into both key frames and watermark. The singular values of key frames and principal component (PC) of watermark are used in the watermark embedding process. The PC is calculated by using equation 3.
PC= U*S

Watermark embedding algorithm
The process flow of watermark embedding algorithm is shown in figure1.
Step 1 The Input video V and secret image W is selected for processing.
Step 2 The Video is preprocessed into number of frames and segmented into different scenes using absolute difference of frame variance.
Step 3 The non-redundant frames are identified by using Fuzzy inference system.
Step 4 In each scene (s 1 ,s 2 …s n ),calculate the entropy of individual frames and AMD and ADFV between adjacent frames for formulating the fuzzy system.
Step 5 Select the key frames (K x ) of each scene based on fuzzy inference system.
Available frames ((AF) = Total Number of frames-Number of redundant frames Number of keyframe =AF/ Number of scenes in video.
Step 6 Apply DWT to key frames, select LH sub band for embedding process.
Step 7 SVD is applied, to extract the singular values of the key frames and PC of the watermark.
Step 8 The watermark image (W) is processed into number of equivalent blocks based on the number of key frames in the scene.
Step 9 The watermark blocks are reordered based on the random number generator (R k ). The number of reorder is based on the availability of number of key frames in a scene.
Step 10 The principal component of the image block is embedded into the singular values of the center region (E r ) in key frames. The size of the embedding region is equivalent to the size of the watermark block W f =S F +k*W pc; S F : Singular values of key frame ; W pc : Principal component of watermark block; K=0.47 Step 11 Combine all the frames in a scene including redundant frames, discorded frames and watermarked frames to make watermarked video (V w ).

Watermark Extraction algorithm
The watermark extraction is the reverse process of embedding. The watermark is extracted from the watermarked video by using the various keys such as keyframe , location of the embedding region and watermark block reordering key (R k ). The process flow of watermark extraction is described in the following algorithm.
Step 1 The watermarked video V w is preprocessed and segmented into different scenes. Step 2 Identify the non-redundant frames in a scene from NRFDB.
Step 3 Select the key frames in each scene based on the keys used in the key frame selection algorithm.
Step 4 Identify the embedding region (Er) of the Individual Key frames in a scene and apply the same key pair used in the embedding algorithm.
Step 5 Extract principal component of the watermark blocks from the watermark embedding region in the key frames.
W pc =W f -S Fi /k Step 6 Reorder the extracted watermark block based on the pseudo random number as a key(R k ) used in the embedding algorithm. W b = (b 1 , b2…., b n ).
Step 7 Combine all extracted block to obtain the watermark image.

Results and discussion:
Fuzzy rule based video watermarking in DWT-SVD domain is proposed. The chosen input video sequences (Suzie, News and Vipmen) in AVI format with frame rate of 30 frames /sec are available in the standard library. It is shown in figure 8. The output of scene change detection in Suzie video is shown in Figure.5. The non-redundant frame identification algorithm is implemented by using fuzzy logic tool box available in the MATLAB. The result of the algorithm is represented in the rule viewer and surface viewer, is represented in Figure.6a and 6b. From Figure.6, we inferred that the median value of SSIM and AMD provides the rank as 9. Based on the rank, the redundant frames and non-redundant frames are segregated and maintained in the separate database. Figure.7 shows that the rule viewer representation of selection of key frames.
Where, P = L*K L,K = Size of the frame in pixels W : Original watermark

Imperceptibility assessment:
The imperceptibility of the proposed system is assessed by conducting various experiments using MATLAB simulator. Different kinds of video and image processing attacks such as blur (10%), brighten (10%), compression(10%), addition of Gaussian and poison noise with 10% variance and zero mean , filtering (window size : 3×3), frame dropping (25%) , frame swapping (25%) and frame averaging(25%) were applied on the watermarked video. The imperceptibility of the watermarked video and watermark is measured by metrics PSNR. The PSNR value of the attacked watermarked video is shown in the Table 3a and 3b. After applying the various attacks, the watermarked video are considerably degraded in terms of quality and also little bit of data are vanished but the watermark is extracted with reasonable PSNR. The extracted watermark PSNR value was compared with the original watermark. The extracted watermark is to be identical to original. Hence the proposed method is robust against these kinds of attacks. For the purpose of evaluating the proposed approach, various input video and watermarks are tested, which are shown in Table 4a. The resultant PSNR values of different watermarks embedded into various video are presented in the Table 4b. The PSNR value of the watermark extracted from videos subjected to attack is closer to the PSNR value of the extracted watermark without attack. Hence the proposed watermarking scheme is robust against the common attacks such as blurring, brighten, adding Gaussian noise and salt & pepper noise, Poisson noise, frame averaging and frame swapping and dropping.

Robustness Assessment:
The proposed method is evaluated in terms of robustness by using the metrics such as NCC, BER, SR and SSIM. Various kinds of image and video processing attacks such as brighten, blur , addition of noise, frame dropping, swapping and averaging are performed in the watermarked video. After the attack, the watermark is extracted and evaluates the robustness based on the quality metrics.

Computational Time:
The computational complexity of the proposed methodology is measured using various input videos having varying number of frames. Table 6 shows number of redundant frames and key frames, computational time for identifying non-redundant frames and key frames, watermark embedding time with NRFI-KFS algorithm and without proposed algorithms. The table shows that nearly up to 38% of redundant frames are available in the video, which is varying based on the total frames in the video, which are excluded from the watermarking process. Hence the overall watermarking embedding time is gets reduced. In table 6 shows that the numbers of key frames are varied based on the number of non-redundant frames present in the video, nearly 25% frames are selected as a key frame in each scene. The watermarking is only done in the key frames identified by the proposed key frame selection algorithm, which improves the authentication of the watermark and also the embedding time is drastically reduced. The fuzzy logic is adapted in the proposed method for identifying the redundant frames and selecting the key frames. The redundant frames are identified using parameters such as SSIM and AMD. Both metrics are used to identify the structurally similar frames in the scene. The fuzzy rules are evaluated on the basis of the abovementioned metrics in a possible way, and finally the non-redundant and redundant database is formulated separately. Then the key frames are identified by the FIS system from the non-redundant database. The evaluation of the FIS is based on the metrics Entropy, AMD and ADFV. Various possible combinations of fuzzy rules are evaluated by the mumdani system and eventually generate the number of key frames for embedding watermark. The fuzzy system is suitable for improving the accuracy for identifying the non-redundant frame and selection of key frames. Hence the processing time is reduced compared to the conventional method which is inferred from Table 5.

Performance comparisons with existing approaches
The proposed methodology is compared with the existing algorithms proposed by various researchers for the purpose of evaluating the performance of the proposed algorithm. The PSNR comparison is represented in Figure.9.
The NCC values between the extracted watermark and the original watermark obtained after applying various attacks in the watermarked video, compared with various existing methods are shown in Table 6. The SR and BER comparison is shown in Figure.10a and 10b.  The proposed algorithm is compared with various existing schemes in order to analyze efficiency. Ejima et al [12] proposed Wavelet based watermarking for digital images and Video. In this algorithm the NCC value was lesser for filtering and frame averaging. Da-Wen [13] proposed a blind video watermarking algorithm based on 3D wavelet transform. In this methodology all kind of attacks were performed, but the correlation values are lesser than the proposed methodology. Sadik [8] proposed robust video watermarking based on 3D-DWT Domain. In this method NCC values for addition of noise and filtering were low. Cruz-Ramos [26] proposed a blind video watermarking scheme robust to frame attacks. This algorithm was not resistant for filtering and compression attacks. Alam et al [27] proposed a technique on video watermarking using 2D DWT and 2-level SVD. In this algorithm only addition of noise and filtering attacks were applied but video processing attacks were not performed. Divjot Kaur Thind [16] proposed video watermarking based on DWT and SVD. This algorithm was robust only for addition of noise and frame averaging attacks but was not resistant with other attacks. Lama Raja b et al [17] proposed DWT-SCHUR based video watermarking. Here some kind of video processing attacks were applied, but it does not produce good correlation value for frame dropping attack. These results are represented in Table 6. The NCC values of various attacks for exiting algorithms are compared with proposed work, it is found that the proposed method gives better results for image and video processing attacks.  [18] proposed robust waveletbased video watermarking scheme for copyright protection using the human visual system. In this algorithm all kinds of attacks were applied into the watermarked video, but it does not produce the minimum acceptable value for filtering attack. Divjot Kaur Thind [16] proposed video watermarking based on DWT and SVD. In this method the PSNR value for watermarked video was good, but after performing various attacks the quality drastically changed. Osama S. Faragallah [15] proposed singular value decomposition in the discrete wavelet transform based video watermarking. Here all kind of attacks were applied and produced reasonable PSNR values for all attacks except filtering. Radu O. Preda [14] proposed robust digital watermarking scheme based on multi-resolution wavelet decomposition. In this algorithm some kinds of attacks were applied, it does not withstand for filtering attack. These existing algorithms do not support the filtering and some of the video processing attacks. The SR and BER metrics result comparison of proposed with existing methodology for various attacks are shown in Figure 10.

Conclusion
In this paper Fuzzy rule based video watermarking in DWT-SVD domain is proposed. The aim of the proposed approach is exclusion of redundant frames from the watermarking process for reducing the overall processing time which is effectively achieved by proposed NRFI algorithm. The authentication and accuracy is improved by proposed KFS algorithm, which is implemented by fuzzy inference system. The PCA technique is employed for avoiding the false positive problem. The principal component of the image block is embedded into the singular values of the keyframe in the video. The pseudo random number generator is tied up with the watermark block processing for avoiding the frame dropping attack. Generally fuzzy based algorithms are not sensitive in accordance with environment. Compare to other conventional system, which needs less computation power and time. The proposed methodology survives the certain kinds of geometric attacks by incorporating the properties of DWT and SVD in the watermarking process. This method also withstands certain image and video processing attacks.

Funding
No funding was received for conducting this study.

Conflicts of interest
All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript.

Availability of data and material
Data sharing not applicable to this article as no datasets were generated or analysed during the current study.

Authors' contributions
Dr.Ramakrishnan S has proposed the idea and supervised the overall work as well as significantly contributed in writing the paper.
Ms.Ponni(a)sathya S has implemented the idea with the guidance of Dr.S.Ramakrishnan using MATLAB and did Literature review related to the proposed method     Rule viewer of key frame selection Figure 8 Original watermark, Watermarked Video and Extracted watermark Figure 9 PSNR Variations of watermarked video between the proposed approach and existing approach Figure 10 a. SR Variations of watermarked video between the proposed approach and existing approach b PSNR Variations of watermarked video between the proposed approach and existing approach