DOI: https://doi.org/10.21203/rs.3.rs-907883/v1
India has its unique and rich cultural heritage. One such uniqueness in India is ancient paintings. Especially in South India, Tanjore paintings are very popular. These paintings are made with vibrant colours, gold, silver and precious stones. These paintings are the memorabilia of the great Chola kingdom. These paintings can be seen in great Brahadeeshwara Temple walls till now. Damages to these paintings happen due to varying environmental conditions throughout the year. Hence, preserving these heritages could be an additional source in National Cultural Museum and cultural libraries. This paper focuses towards the restoration of such ancient painting images that can be digitized and archived for the future use of aesthete.
India is famous for its culture and civilization. Indian ancestors excelled in the field of Architecture, Art, Medicine, Astrology, etc., that are recognized and admired worldwide. They passed on the information about the heritage to their future generation in the form of sculptures, paintings and inscriptions. Paintings are one such art form which depicts the ancient history and culture. Chittannavasal and Tanjore paintings are very prominent in India which attracts people around the world. Tanjore painting play a significant role in Indian paintings which are made of vibrant colours, gold and precious stones. But due to climatic changes, lack of maintenance and rituals these paintings are degraded. Though many efforts are carried out to save these paintings from further damage, it is very challenging to restore the paintings from the existing degradation. However, with developing technology in the field of image processing has made this challenge achievable. These painting can be restored by formulating a degradation model and then developing an algorithm to restore the degraded portions. This paper deals with the various methodologies adopted for restoration of damaged images. This section describes the various restoration techniques adapted for paintings.
Marwa Jmal et al (2017) developed an image restoration with nonuniform illumination enhancement technique. Initially the authors performed contrast adjustment, then illumination is enhanced on the application of a modified homomorphic filter in the frequency domain. Optimal parameters were computed using golden section search algorithm to produce the enhanced image. At last, a color restoration function is applied to prevail over the problem of color violation. Their results yielded, local contrast improvement, detail enhancement, and preserving the originality of the image. The technique is applied on the collected dataset of cultural heritage images.
Jan Blazek et al (2009) have proposed a technology by combining the fresco art image of narrow-band ultraviolet, and the broad-band ultraviolet wavelength spectra. In addition, they used fusion of the old black and white photograph. They gathered all available information in such a way to view the details in one fused image using PCA transform. Then they performed the chemical analysis on the image spots using the spectroscopy and structure based neighborhood algorithm for the better visualization of the image.
Yuan Zeng and Yi Gong (2018) applied nearest neighbouring method for restoration of damaged ancient Chinese paintings that have tears, flakes and cracks. The damage detection was obtained by estimating a mask initially. Followed by the masking, the damaged part is constructed using Inpainting algorithm. The authors also discussed the application of deep learning algorithm as future research. Nikolaos Karianakis and Petros Maragos (2013) presented a computer vision system for robust restoration of prehistoric Tehran wall paintings. The authors applied an image stitching algorithm for image restoration. An area of relevant semantics, geometry and color in a different spot of the wall paintings was selected and stitched into the damaged area. Their key focus was the identification of damaged or missing area in the painting performed using morphological algorithm in addition with edge information. Ioannis Giakoumis et al (2006) presented a methodology for both detection and elimination of cracks on digitized paintings. Initially thresholding of the morphological top-hat transform was performed for crack identification. Then, median radial basis function is used to remove the misidentified cracks by region growing technique. Finally, crack filling using order statistics filters or controlled anisotropic diffusion is performed. The author claimed that their methodology was well suited for digitized paintings affected from cracks. Similar methodology was implemented by Shrinivas D Desai et al (2013) and they were able to achieved true positive rate of 98.3%.
M. Barni et al (2000) presented computer-guided and virtual artwork restoration techniques. These technique aids the restorer with virtual cleaning software to identify the best suitable cleaning procedure with a small patch of the paintings. Thus with initial study, it could be extended to the painting upon successful implementation with the small patch. Song Wei (2014) developed a novel framework based on hierarchy for the restoration of Chinese paintings. The framework involves 3 phases such as layering phase, hierarchy restoration phase and synthesis phase. In layering phase the painting was split into foreground and background layers. In hierarchy restoration phase, various image restoration algorithms were applied to these layers. In synthesis phase the restored image from the foreground and background were combined to get the complete restored painting. Ayman M. T. Ahmed (2009) introduced two different methodologies for color image restoration. The first technique involves blending of the standard deviation- weighted gray world and the Combined Gray World and Retinex (CGWR). The second technique was based on alteration of the Multi Scale Retinex (MSR) theory. In these techniques, the effect of neighboring pixels on the human eye is replicated for modifying the algorithms. In addition, the modified MSR is applied on CGWR technique to improve the performance of the basic algorithm. Their experimental results depicted the comparison between these two techniques with the basic traditional technique. Ioana Cortea et al (2020) presented analytical characterization of Romanian Monastery paintings. X-Ray Fluorescence (XRF) and Fourier Transform Infrared Spectroscopy (FTIR) were applied for the analysis. The data from FTIR supported XRF result to provide material characterization. The authors were able to identify many mineral pigments and the evidence of organic binders from the paintings.
From the literature studies it is evident that restoration of images plays a vital role in preserving the cultural heritage. This paper focuses on restoration of ancient Tanjore painting using image processing techniques in two ways one with segmentation process and the other with patch based inpainting technique.
In the proposed work, the process of ancient painting restoration is performed in two different ways as shown in Fig. 1. In the first process the image to be restored is segmented into various blocks. Each block of an image is identified with the Region of Interest (ROI) and the average of neighbouring colours of a block is filled in the ROI. The second process is use of inpainting techniques. This technique uses a binary mask; the masked region depicts the damaged part of the painting. The masked region is filled by interpolation technique.
The images were collected from Tanjore Brahadeeshwar Temple and data bases were created. As a first step image pre-processing steps are implemented for the removal of noise. Various filters such as Weiner, Median and Gaussian filter have been implemented. The Peak Signal to Noise Ratio (PSNR) is estimated and best suit filter is selected for pre – processing steps. The filtration is carried out with intentionally added noise and no additive noise. Figure 2 shows the PSNR values of median, Gaussian and Weiner filter applied on 2 images with Gaussian noise added to it. From the figure it is inferred that Weiner filter performs better than other two filters. Figure 3 shows the PSNR values of different filters for images with salt and pepper noise added to it. It is observed that Median filter performs better than the other two filters. Figure 4 depicts the PSNR value of different filters for images with additive speckle nose. From the results it is observed that wiener filter performs better for speckle noise than the other two filters. Figure 5 shows the performance of various filters on the original image without any additive noise. Weiner filter performs better for the images without any noise added to it. Hence from the results it is concluded that Wiener filter is considered for removal of noise in the painting Images. The Mean Square Error (MSE) values by applying various filters for images without and without additive noise are given in Table 1. It is observed that the MSE is very less for Weiner filter except for Salt and Pepper noise.
Authors are required to adhere to this Microsoft Word template in preparing their manuscripts for submission. It will speed up the review and typesetting process.
| Type of Noise | Filter | MSE | |
| Image 1 | Image 2 | ||
| Gaussian | Median | 0.10 | 0.03 |
| Gaussian | 0.12 | 0.04 | |
| Wiener | 0.09 | 0.03 | |
| Salt and Pepper | Median | 0.009 | 0.002 |
| Gaussian | 0.1 | 0.04 | |
| Wiener | 0.2 | 0.07 | |
| Speckle | Median | 0.09 | 0.04 |
| Gaussian | 0.09 | 0.04 | |
| Wiener | 0.08 | 0.03 | |
| Without Noise | Median | 0.007 | 0.002 |
| Gaussian | 0.02 | 0.005 | |
| Wiener | 0.005 | 0.001 | |
Figure 6 shows the performance of various filters for Gaussian noise. Figures 7 and 8 shows the performance of various filters for Salt and pepper noise and speckle noise respectively. From the figure it is observed that for salt and pepper noise median filter provided better results. Apart from that, Weiner filter is better for other noises and also without additive noise. Hence in the research, Weiner filter is applied for pre-processing of painting images which is effective in eliminating any noise that might have occurred during acquisition of images.
The input images are segmented in to various blocks such as 4, 8 and 16. Each block of a colour image is converted in to binary image and region of interest (ROI) is identified. The average of RGB colour component of the corresponding block is filled in the ROI. The following images shown in Fig. 9(a) and 9(b) are the original and restored painting images respectively.
From Fig. 9(a) and (b) it is inferred that the images have been filled with colours to avoid the damage, however the filled portions are not the exact match of colours and few portions are not identified properly. Hence the outcome of segmentation techniques is not productive. Initially the image was segmented in to four parts and the restored image was analyzed. The ROI is not identified properly using four blocks of segmentation, and hence filling was very poor. When the same has been analyzed with sixteen blocks, the restoration is better than 4 blocks however, the colours are not properly averaged. Hence a compromised no of blocks for segmentation is taken as 8. Figure 9 (b) shows the restored images using segmentation of eight blocks. The number blocks fixed in this work is based on heuristic approach. To achieve better restoration of ancient images a semi-automated process with in-painting technique is implemented.
Inpainting technique is implemented to restore painting [19]. In this process the region of interest is selected manually. Hence the process is semi-automated. A patch based method is adopted for damage identification using a binary mask (M) which is created for the damaged part of the painting according to Eq. (1).
\(\text{M}\left(\text{p}\right)=\left\{\begin{array}{c}1 for p\in D \\ 0 for p \in U\end{array}\right.\) 1
Where, p is the pixel index, D is the damage area of the painting and U is the undamaged area of the painting. Initially the image is decomposed for its texture and structural features. Once the features are extracted the interpolation of images is done for both extracted images. The flow of the work has been given in Fig. 10 and the original and restored images through this technique are given in Fig. 11.
This research focuses on the restoration of Tanjore painting images based on segmentation and Inpainting techniques. From the results, it is concluded that in painting techniques outperforms segmentation process for the restoration of ancient painting images. The restoration performed using segmentation technique by averaging RGB components was not able to fill the colour properly. The ROI of the segmented block is filled with the average colour of the corresponding segment. However, the region of interest may have a dominant colour of the neighbouring block. Hence segmentation of blocks should be optimized in such a way that appropriate colour should be chosen for filling. At the same time the number of blocks could be varied dynamically based on the colours used in images. Images having less number of colours are restored efficiently than the images with a large number of colour combinations. It is evident that in Tanjore paintings vibrant and number of colours are used. Hence segmentation technique is not suitable for painting images. With Inpainting technique, automatic selection of ROI is not appropriate and hence manual selection of ROI is applied. Inpainting performs better for restoration of ancient painting images. Further research can be carried out with automatic selection of ROI without compromising on the quality of restored image.
Declaration
Availability of data and materials:
The data was collected through capturing the degraded images using a digital camera by the Authors of this paper at Bhahadeeshwara Temple, Tanjore, India
Competing Interest:
The authors declare that they have no conflicts of interest to report regarding the present study.
Funding Statement:
This work is funded by Indian Council of Social Science Research (ICSSR) under IMPRESS scheme (IMPRESS/P1415 /2018-19/ICSSR).
Authors’ contribution:
S.Poornapushpakala – Image acquisition and preprocessing, preparation of the manuscript
S.Barani – Developed the restoration algorithm, preparation of manuscript
M.Subramoniam – Image acquisition and processing
T.Vijayashree – Supported in documentation of literature reviews
Acknowledgement:
The authors thank ICSSR and Sathyabama Institute of Science and Technology for extending their support.