Structural control over the magnetic properties of molecular nanomagnets (MNM) is key to bottom up design strategies for novel molecule-based magnetic materials, for e.g. data storage, magnetic cooling or spintronics. Herein, we switch the magnetism of materials made of the single-molecule magnet [CoxZn(1 x)(piv)2(2 NH2 Py)2], by structural changes induced by gradual replacement of complexes containing paramagnetic Co(II) with analogues containing diamagnetic Zn(II). Combining advanced structural and magnetic characterization techniques, we show that a change of the ratio of the two ions induces a change of the Co(II) coordination structure, changed magnetic relaxation times and a switch from easy-plane to easy-axis type magnetic anisotropy. Complementary quantum chemical calculations reveal magneto structural correlations, rationalizing the structural origin of the changed magnetic properties upon magnetic dilution. This approach provides an important new lever to understand, predict and tune the magnetic properties of a large range of MNM based magnetic materials.