Near-infrared (NIR) conveys sharp images with dramatic viewpoint in photography. Certainly sky and water always appear in black and vegetation somehow very bright, for some reason the visibility turn out different. In vegetation, this is due to the cellular structure of the leaves; water absorbs near-infrared radiation. Clouds and sky are composed of droplets that throw inward light according to Mie’s law (the angle of scattering is independent of the wavelength) [1]. Blue is most scattered compared to near-infrared due to very small particles scattered light but scattering angle is proportional to the inverse power of wave length. Fig. 1 shows the range of visible and near-infrared spectrum.
In other research work the use of data image in the far-infrared (4-12m), using special detector that is dedicated for use in the far-infrared band is discussed. In this work, we focus instead on the visible-RGB (400 -700) and near-infrared (700-1100nm) that can be captured using an ordinary digital camera that provides high resolution images.
The power spectra and the joint wavelet statistics of coefficient distributions of datasets of indoor and outdoor show strong correlation between object boundaries in IR and visible images, leading to high potential for vision applications using a combined statistical model, as discussed in [15].
A method to obtain simultaneously the near-infrared and red bands from a standard RGB camera is proposed in [16]. Results show that NDVI bands can be acquired with high resolution cameras, which allow more research in crop monitoring applications.
The geo-referenced NDVI vegetation Index produced by flying certain area of interest with visible and RGB+NIR full spectrum camera [19]. The images from visible are RGB+NIR processed inside the Dronemapper which results of having two orthomosaic which can generate pure NIR ortho.
The design and evaluation of a Parrot drone multispectral imaging system based on identical multispectral image camera, is presented in [22]. The method of second order was used to align the multi-color and Near-infrared image to attain sub-pixel accuracy.
Raspberry pi A+ board is thought to be a solution that can be used with visible and NIR imaging systems [20]. The board can be used for storage and communication. It can exploit the correlation and de-correlation of visible and NIR image frequencies and intensities to enhance the visual quality of images/videos and to extract more accurate information about the scene. Fig. 2 shows the Raspberry pi interface with camera.
The schematic shown below in Fig.3 presents a simplified overview of the multispectral imaging system. Two Raspberry Pi boards are adopted, with multispectral image cameras (RGB and NIR), which can be robust and reliable for deployment on the Unmanned aerial vehicle (UAV) [22].
Raspberry pi A+ and B+ modules are similar but present lightly different features. An overview of the modules and components is highlighted in Fig. 4. The two boards have common features such as the GPIO input/output interface and many more. The A+ module has two USB ports that enable it to be used as master’s part and B+ module for slaves since has only one USB port. The Camera RGB (visible) interface with A+ module while B+ with slave’s part through CSI (Camera serial interface). One of the vital features of Pi is low power consumptions.
The CSI bus connects to BCM2835 processor on the Pi, where higher Bandwidth link carry pixel data from camera to the processor. Fig. 5 shows two prototypes of an NIR and RGB cameras (source: www.google.com).
A 3D of RGB & NIR camera
The two boards are very small in size, at about 25mm x 24mm x 9mm. They only weight over 3gram that make them perfect for real time applications where size and weight are vitally considered. The short ribbon cable used to connect the cameras to the pi via CSI. These CSI acts as a bus that connects the cameras to BCM2835 processor on the pi, where higher bandwidth link carry pixel data from camera to the processor. This bus travels laterally ribbon cable that assigns the camera board to the Raspberry pi. RGB cameras alone are useful to vegetation studies but for detailed analysis, NIR is preferable and is more essential. In the next section, we present the system design methodology.
System Design Methodology
With the increase of advanced technology demand, multispectral imaging system has been used nowadays in various applications such as environmental observation, and for security/defense purposes. Traditional spectral imaging techniques involve time-sequential scanning of either a spatial or a spectral dimension combined with snapshot imaging of the other two dimensions [1]. These methods are exemplified by a push-broom scan of a one-dimensional spectral imager across the required field of view [2], the use of tunable spectral filtering [3] or imaging Fourier-transform spectrometry [4]. The applications of these traditional time sequential techniques are restricted to the applications where an extended recording time is acceptable, for example in microscopy, remote sensing, and biomedical imaging [5, 6, 7, and 8].
Ground Coverage and Pixel Size
The ground coverage of the camera was determined by this formula;
𝑮𝑳 = 𝑳/𝑭𝑯 = 𝟏.𝟖𝑯 (𝟏)
where GL is the length of the ground coverage (m), GW is the width of the ground coverage (m), L is the length of the sensor (36 mm), W is the width of the sensor (24 mm), F is the focal length of the lens (20 mm), and H is the flight height above ground level (AGL) (m).
The pixel size is determined by the formula below, where P is the ground pixel size (m);
𝑷 = 𝟏.𝟖/𝟓𝟔𝟏𝟔𝑯 = 𝟑.𝟐𝟎𝟓 𝑿𝟏𝟎−𝟒𝑯 (𝟐)
Image Alignment
Misalignment of images obtained from multispectral cameras is one intrinsic problematic allied with any multi-camera imaging system. Therefore, cameras were mounted close to each other. The mechanical design system ensure that image capturing is at maximum overlapping, though it is not adequate for data extraction and image analysis directly. A second-order polynomial transformation as given below was introduced to accurately align the individual images [22].
𝒙𝟎 = 𝒂𝟏+𝒂𝟐𝒙𝟏+𝒂𝟑𝒚𝟏+𝒂𝟒𝒙𝟏𝟐+𝒂𝟓𝒙𝟏𝒚𝟏+𝒂𝟔𝒚𝟏𝟐 (3)
𝒚𝟎 = 𝒃𝟏+𝒃𝟐𝒙𝟏+𝒃𝟑𝒚𝟏+𝒃𝟒𝒙𝟏𝟐+𝒃𝟓𝒙𝟏𝒚𝟏+𝒃𝟔𝒚𝟏𝟐 (4)
However, the method of Intensity-Based Automatic Image registration was used to align images, through its applicability to Matlab available in the Image Processing Toolbox, Fig. 6. This method is iterative, thus requires a pair of images, a metric, an optimizer and a transformation type. Below is the architecture of this alignment process.
The above process starts with a transform type specified by the user. Internally, a transformation matrix is determined so to determine specific image transformation for the moving image, through bilinear interpolation. The metric then compares the fixed image with the transformed moving image to obtain a metric value which monitored by the optimizer. A stop condition is reach when the optimizer detects a maximum number of iterations. The metric and optimizer work closely in this propose, thus it defines image similarity metric and the optimizer expresses a method for minimizing/maximizing it. Fig. 7 presents the overall system bloc diagram.
Software Component Design
The software experiment for the project was done on a PC that runs on the Raspi operating system, “Raspbian”. The OS runs on the working PC, thus upon request for a password (“pi”) and username (‘raspberry’). After logging in using the given password and username, the configuration window appears where the user can enable the camera and other OS requirements. By entering “startx” on the command window, the raspberry pi Graphical User Interface (GUI) window appears and that allows the user to interact with electronic devices through graphical icons and visual indicators. For this work, python3 was used to implement the system module that will capture image every five seconds and store them in the micro SD card attached to the Raspi board. Fig. 8 depicts a snapshot of the Python code used for capturing NIR images assuming a single camera configuration.
In the following section, we discuss the system implementation and prototyping.
Prototyping
The system is multi spectral imaging system was implemented in MATLAB/Simulink and the system was tested for accuracy considering different environmental situations and scenario.
Below we show the experimental set-up for the proposed system in Fig. 10.
System Integration
Depicted in Fig. 11 below is the detailed process of how the hardware and software components are integrated together to one functional system. The system total weight is 191.8 grams.
Computer Simulation
MatLab/Simulink was used as simulation tool for the purpose of image alignment and measuring system accuracy. The system was used to measure the system accuracy considering various environmental conditions and scenarios. Fig. 14 shows the System architecture of phase correlation and spatial referencing method.
The system estimates how tight the alignment between the moving image and fixed image is. The smaller the RMS value, the better the alignment. Further improvement can be done to obtain perfect alignment by adjusting the number of iterations and the initial radius of the optimizer, either increasing or decreasing. The output should then give an expected value of RMS error to determine that accuracy of image alignment.
Python programming was used for communication between master and slave processors (PIs).
In the next section, we presents the experimental results.