Comprehensive research is conducted on the design and control of the unmanned systems for electric vehicles. The environmental risk prediction and avoidance system is divided into the prediction part and the avoidance part. The prediction part is divided into environmental perception, environmental risk assessment, and risk prediction. In the avoidance part, the conservative driving strategy based on speed restriction is adopted according to the results of risk prediction. Additionally, the core function is achieved through the target detection technology based on deep learning algorithm and the data conclusion based on deep learning method. Moreover, the location of bounding box is further optimized to improve the accuracy of SSD target detection method based on solving the problem of unbalanced sample categories. Software such as MATLAB and Carsim are applied in the system. From the comparison results of the simulations of unmanned vehicles with or without a system, it that the system can provide effective safety guarantee for unmanned driving.