Understanding the structure of the protein-ligand complex is crucial to drug development. However, existing virtual structure measurement methods are mainly docking and its derived methods combined with deep learning, which have restricted performance and efficiency due to their sampling and scoring methodology. Here we show the complex structure can be directly predicted using our proposed LigPose based on geometric deep learning in an end-to-end manner. By representing the ligand and the protein as a complete graph, LigPose optimizes the 3-D structure of the complexes with their atom coordinates in the Euclidean space. LigPose achieved state-of-the-art performance on two major tasks in drug development, i.e., complex structure prediction and affinity estimation, indicating a promising paradigm of predicting the protein-ligand complex structures in drug development, with improved capacity far beyond popular docking tools.