For rock specimen in uniaxial compression, the energy transformations from elastic strain energy in both the rock and the loading system to plastic strain work in the rock can be identified with the changes in these energy components, whose rates are also useful indicators for distinguishing stable and unstable rock failure. In this study, the influences of the loading system stiffness (LSS), the rock stiffness and the rock brittleness on rock failure modes are examined. The observed energy transformations during rock failure in numerical models are interpreted from an energy perspective. The results show that unstable rock failure tends to occur in rock with large brittleness and small stiffness under a soft loading system. A low LSS and rock stiffness will increase the magnitude of stored elastic strain energy before rock failure, while a brittle rock requires less elastic strain energy to be converted plastic strain work than a ductile rock during its failure. This energy-based approach is useful for investigating potential unstable rock failures that could ultimately be applied to analyze complex mine-scale rockburst cases.