Particles in a crowded environment exhibit slow anomalous diffusion. In crystals, constituent particles do not diffuse, but skyrmions and half-skyrmions, spatially localised quasiparticles, exhibit normal diffusion in dilute conditions. However, the cooperative dynamics and diffusion of (half-)skyrmions in liquid, hexatic, and crystalline phases are elusive. Here we show that the fusion and fission of half-skyrmions, not their diffusion, are responsible for the primary structural relaxation in these condensed phases. The diffusion, which contributes to the secondary structural relaxation, is suppressed by cages formed by surrounding half-skyrmions, whereas enhanced by Mermin-Wagner fluctuation characteristic to two-dimensional systems, leading to subdiffusive motion. Large displacement of half-skyrmions is locally induced by fusion-fission and bond-breaking between adjacent half-skyrmions. Furthermore, the motion of half-skyrmions couples with transverse and longitudinal sound wave excitation differently. Our result provides a qualitative difference in dynamical properties between skyrmion gaseous and condensed phases, suggesting the efficient manipulation of high-density (half-)skyrmions.