Image genetics reveal the connection between microscopic genetics and macroscopic imaging and then detect diseases’ biomarkers. In this research, we make full use of the prior knowledge that has significant reference value for exploring the correlation of brain and genetics to explore more biologically substantial biomarkers. In this paper, we propose Joint-Connectivity-Based Sparse Nonnegative Matrix Factorization (JCB-SNMF). The algorithm simultaneously projects structural Magnetic Resonance Imaging (sMRI), Single Nucleotide Polymorphism sites (SNPs), and gene expression data onto a common feature space, where heterogeneous variables with large coefficients in the same projection direction form a common module. In addition, the connectivity information of each region of the brain and genetic data are added as prior knowledge to identify regions of interest (ROI), SNP, and gene-related risks related to Alzheimer's disease (AD) patients. GraphNet regularization increases the anti-noise performance of the algorithm and the biological interpretability of the results. The simulation results show that compared with other NMF-based algorithms (JNMF, JSNMNMF), JCB-SNMF has better anti-noise performance and can identify and predict biomarkers closely related to AD from significant modules. By constructing a protein-protein interaction network (PPI), we identified SF3B1, RPS20, and RBM14 as potential biomarkers of AD. We also find some significant SNP-ROI and Gene-ROI pairs. Among them, two SNPs of rs4472239 and rs11918049 and three genes of KLHL8, ZC3H11A, and OSGEPL1 may have effects on the gray matter volume of multiple brain regions. This model provides a new way to further integrate multimodal impact genetic data to identify complex disease association patterns.