In Mobile Edge Computing (MEC), the service coverage limitation of base stations leads to dynamic changes in the connections between mobile devices and them, directly impacting the decision-making process of agents. The relationship between mobile devices and base stations is abstracted into an MEC structure graph in current research, however, traditional deep reinforcement learning faces challenges in effectively capturing the complex relationships between nodes within dynamic graph structures. To address this issue, this paper proposes a hierarchical mechanism known as Graph Neural Reinforcement Learning (M-GNRL) under multiple constraints to facilitate optimal result. Specifically, when tasks necessitate offloading for execution, M-GNRL algorithm recommends potential offloading nodes based on node representations. Selecting nodes with higher recommended probabilities to reconstruct into a deep reinforcement learning environment, aiming to extract crucial nodes to reduce the state space size of the policy. This strategy incorporates edge features from graph neural networks into the deep reinforcement learning architecture, after training, which can assist the agent in making optimal decisions in dynamic environments. To reduce the time required for mapping MEC scenarios into graph structures, this paper proposes an updating algorithm to acquire graph information that varies with anchor node. Comparative experiments demonstrate that the M-GNRL algorithm outperforms other baseline algorithms in terms of system cost and convergence performance.