With the development of full digitalization, the amount of time-series data generated by sensors is ever increasing; thus, time-series outlier detection has become crucial. Moreover, in practice, discovering and flagging anomalies is very time-consuming and expensive. To solve this problem, unsupervised anomaly detection methods have often been used in the past, wherein the model is trained with normal data to learn its behavioral patterns. Generative adversarial networks (GAN) can simulate complex and high-dimensional distributions of data and can be used to learn the behavioral patterns of normal data for unsupervised anomaly detection. However, due to the problem of convergence, GAN are difficult to train. Thus, USAD utilize an autoencoder (AE) to undertake the task of the generator and discriminator and enhance the stability during adversarial training by using the AE to alleviate the problem of non-convergence encountered in GAN. Therefore, in this study, we used the USAD’s generative adversarial training architecture combined with convolutional AEs to improve the model’s feature extraction capabilities. In addition, to reduce false-positive outcomes caused by the prominent sharp points in the reconstructed data, we used the exponential weighted moving average method to smooth the reconstruction error, thereby improving the anomaly detection accuracy of the model. Finally, we experimented with real-world time-series data (ECG and 2D gesture) and verified that our approach could improve the accuracy.