In variational quantum algorithms, it is important to balance conflicting requirements of expressibility and trainability of a parameterized quantum circuit (PQC). However, appropriate PQC designs are not necessarily trivial. Here, we propose an algorithm for optimizing the PQC structure, where single-qubit gates are sequentially replaced by the optimal ones via diagonalization of a matrix whose elements are evaluated on slightly modified circuits. This replacement leads to a better approximation of target states with limited circuit depth. Furthermore, we clarify the existence of a barren plateau in the sequential optimization in terms of the spectrum concentration of the matrix, which defines the cost landscape with respect to changes in the target gate. Then, we rigorously show the concentration is no faster than polynomials in the number of qubits when an $n$-qubit PQC depth is $O(\log{n})$ using local observables. Finally, numerical experiments are provided to show the convergence of our method which is faster than gradient-based optimizers. Our results provide evidences for sequential optimizers as better alternatives to optimize PQCs on near-term quantum devices.