This paper proposes a sliding-mode-based unified control framework to solve the synchronization problem of memristor chaotic systems. Both finite- and fixed-time synchronization of the memristor chaotic systems can be obtained in the uniform framework. According to the Lyapunov stability and finite-time stability theories, we demonstrate that the trajectories of error system reach the presented sliding-mode surface and converge to the origin along the surface in a finite/fixed time. Moreover, an image encryption algorithm is developed based on the presented control framework. Finally, the numerical simulations and the statistical performance analyses are discussed to illustrate the correctness of synchronization results, the effectiveness of the proposed encryption algorithm, and its potential applications in the scope of secure communication.