Machine learning (ML) models, if trained to datasets of high-fidelity quantum simulations, produce accurate and efficient interatomic potentials. Active learning (AL) is a powerful tool to iteratively generate diverse datasets. In this approach, the ML model provides an uncertainty estimate along with its prediction for each new atomic configuration. If the uncertainty estimate passes a certain threshold, then the configuration is included in the dataset. A key challenge in this process is locating structures for which the model lacks underlying training data. Here, we develop a strategy to more rapidly discover configurations that meaningfully augment the training dataset. The approach, uncertainty driven dynamics for active learning (UDD-AL), modifies the potential energy surface used in molecular dynamics simulations to favor regions of configuration space for which there is large model uncertainty. Performance of UDD-AL is demonstrated for two challenging AL tasks: sampling the conformational space of glycine and sampling the promotion of proton transfer in acetylacetone. The method is shown to efficiently explore chemically relevant configuration space, which may be inaccessible using regular dynamical sampling at target temperature conditions.