It is commonly known that the dephasing in open quantum systems is due to the establishment of bipartite correlations with ambient environments, which are typically difficult to be fully characterized. Recently, a new approach of average over disordered Hamiltonian ensemble is developed and shown to be capable of describing the nonclassicality of incoherent dynamics based on inferring the nonclassical nature of the correlations. Here we further extend the approach of Hamiltonian ensemble in the canonical form to the realm of structural disorder. Under the variable separation of the probability distribution within the Hamiltonian ensemble, the geometrical structure is easily visualized and can be characterized according to the degree of symmetry. We demonstrate four degrees and investigate the effects of different types of symmetry breaking on the incoherent dynamics. We show that these effects are easily understood from the emergences of additional terms in the master equations, leading to rather general master equations and, consequently, going beyond the previous frameworks of pure dephasing or isotropic depolarization.