Traditional iterative-based reconstruction algorithms for compressed color imaging often suffer from long reconstruction time and low reconstruction accuracy at extreme low-rate subsampling. This paper proposes a model-driven deep learning framework for compressed color imaging. In the training step, extract the image blocks at the same position of the R, G, and B channel images as the ground truth, then singular value decomposition is performed on the measurement matrix to obtain the optimized measurement matrix and low-dimensional measurements, afterwards the ground-truth and optimized measurements are utilized to construct a large amount of training data pairs to train an ‘end-to-end’ deep unfolding model for compressed color imaging. In the test step, the single pretrained model is used to reconstruct high-quality images from optimized low-dimensional compressed measurements for each channel and synthesize a color image. Numerical experiments demonstrate that our proposed unified framework can achieve high accuracy and real-time reconstruction for the color image at extremely low subsampling rate.