21. NMAT background
NMAT (version 1.0) is a simple GUI interface primarily used to easily and rapidly access and obtain cell data stored in the NeuroMorpho.Org database. The NMAT program, together with its source code is freely available for download at GitHub (https://github.com/izaletel/NeuroMorpho-Access-Tools). Users can choose to download Windows or Linux compatible versions of the program. The source code is also available for advanced users who wish to modify the functionally of the program according to their own needs. The core of NMAT was built in Python and utilizes the NeuroMorpho.Org API to access the database. NMAT has a GUI built in Qt environment (PySide6, version 6.2.3, https://pypi.org/project/PySide6/). The entire NMAT program is lightweight, consists of one executable file and has no dependencies. It possesses a simple, friendly and intuitive interface and thus does not require any extensive accompanying instructional or technical documentation. Additionally, NMAT does not need to be installed on the computer and can very easily be transferred from one device to another, which allows easy access to the database from any computer.
2.2 Overview of NMAT interface
2.2.1 Querying and acquiring cell data
The current version of NMAT consists of a single 3-tab window, which allow users to obtain cell data and acquire cell image reconstructions (Figure 1). Each tab consists of different combo boxes and a text box, which gives user feedback information about the tasks that the user has given the program through its combo box.
The first tab called “Generate CSV” is the main component of the NMAT program and is used to generate a .csv file of cell reconstruction data and morphometry based on the user’s query input. Note that .csv format was chosen because it can be easily imported into a spreadsheet, Python or R program for subsequent analysis. The NeuroMorpho.Org database allows the users to explore the database using multiple search criteria. Here we have decided to simplify the query process and thus have opted for three main criteria that the user can query: 1) Brain Region, 2) Species, and 3) Cell Type. These three combo boxes contain the identical values/options found in the NeuroMorpho.Org database. After selecting the appropriate values and pressing the “Get CSV” button, the program will make an API connection to the NeuroMorpho.Org server, find and then download the cells based on the search criteria. The program will first show the available number of cells and begin with a 10 second countdown during which the user can decide to continue with the generation of the .csv file or cancel the current query and acquisition. After the 10 second countdown the program will automatically continue with the query after which the NMAT will create a joined Pandas Data Frame with both cell reconstruction data and cell morphometry data. The user can follow the cells that are being queried in the text box and get feedback about the remaining time need to generate the .csv file by looking at the progress bar at the top of the window. At the end of process, the text box will show the structure of the newly created Data Frame (No. rows x No. columns) together with the time that was needed to generate the .csv file. By using the buttons “Open CSV file” or “Open file location” the user can open the .csv file or access its folder file location. If no cells are available for the given combination the program will return code 404 with the statement “Resource not found or does not exist.”, as it is the default code given by NeuroMorpho.Org server.
The “Generate CSV” tab has an underlying Python script that consists of two main parts: finding the cell reconstruction data and ID numbers of queried cells, and then fetching the morphometric data of queried cells using their ID number. This results in the creation of 3 .pkl files (“pickled” date files) which are needed to store the information about cell reconstruction and morphometric data and that will appear in the /output folder of NMAT after the first search. These are generated/overwritten for each new search that the user requests from the program. The last part of the script is used to save the generated Pandas Data Frame in the form of a .csv file. By default, each generated .csv file will be saved in the following format “NM_BrainRegion_Species_CellType”, which will correspond to the values queried by the user in the beginning of the search selection. The user also has an option to manually choose and enter the name of the .csv file that is going to be generated.
2.2.2 Getting images of downloaded cells
The NMAT also provides the ability to download an image (.png) file of each reconstructed cell. This is done in the second tab named “Get Images”. After opening this tab, the user will be shown all of the .csv files that were generated earlier and just by simply selecting the appropriate .csv file and pressing the button “Get Images”, the program will begin downloading the .png reconstructions. The files will be saved in the /output/images folder, where a new folder will be created with a name identical to the .csv file used to download image reconstructions. It is also possible to select multiple .csv files at once and download images files from more than one.csv file at the same time. This will lead to the creation of multiple new /output/images folders, with the same names as the selected .csv files.
The “Get Images” option can also work if the user modifies the previously downloaded .csv files. If the user does not need all of the image reconstructions contained in the .csv file, the user can simply delete the rows which contain those cells, save the .csv file and then select it from the “Get Images” tab of the NMAT program. The only requirement for this option to work is to not modify the “Png URL” column in the downloaded .csv file.
2.3 Multithreading architecture and additional notes
Through the use of the NMAT program it is possible to simultaneously download cell reconstruction and morphometric data and download image reconstructions based on previously generated .csv files. Each function uses multiple threads. Thus, all of the tabs work independently and allow users to perform multiple actions at the same time.
Due to its multithread-based architecture, NMAT is capable of downloading a large amount of morphometric and reconstruction data in a short time period, which represent the main advantage of the program. The multithreading capabilities were implemented through the use of QThreadPool threads for GUI threading, and concurrent.futures.ThreadPoolExecutor threads for the worker threading. GUI threading was implemented in order to allow the uninterrupted use of the program when an operation is running. Current operations are the acquisition of neuron morphometry (Acquisition) and the download of neuron images (Image). Upon starting an operation (clicking a button on the related tab) a QThread is created. The subsequent QThread activates worker threads to complete its function. Worker threading is implemented in order to speed up operations. Current maximum for threading is set to 4 threads, with the possibility of changing this number in further versions. Worker threads split up the acquiring of morphometry, and the download of images in order to speed up the requests for data.
The entire code for accessing the NeuroMorpho.Org API server and the NMAT program code are available at the NMAT project GitHub page and can be freely downloaded and modified by other users to fit their specific needs.