The present study experimentally investigated the shear behaviors of artificial rock samples containing single or double joints under constant normal load (CNL) and constant normal stiffness (CNS) boundary conditions. The influence of surface roughness represented by joint roughness coefficient (JRC) on the evolutions of shear stress, normal stress, and normal displacement as well as the failure mode are analyzed and discussed. The results show that the peak shear stress and peak normal displacement are increased by 104% and 45.1% for single-joint specimens, which are increased by 50.1% and 182.8% for double-joint specimens when JRC ranges from 3.21 to 12.16 under CNL boundary conditions. Under a CNS boundary condition, the shear stress for single-joint and double-joint specimens increase slowly with the increase in shear displacement, and no obvious peak value is observed. However, the shear stress of double-joint specimen is lower than that of single-joint specimen due to that the former has a more complex joint structure. The surface damage degree of the upper joint is larger than that of the lower joint, and the surface failure of the lower joint lags behind that of the upper joint.