Cell type-specific gene expression patterns are outputs of transcriptional gene regulatory networks (GRNs) that connect transcription factors and signaling proteins to target genes. These networks reconfigure during dynamic processes such as cell fate specification to drive diverse cellular states. Single-cell transcriptomic technologies, such as single cell RNA-sequencing (scRNA-seq) and single cell Assay for Transposase-Accessible Chromatin using sequencing (scATAC-seq), can examine the transcriptional state of individual cells, allowing the study of cell-type specific gene regulation at unprecedented detail. However, current approaches to infer cell type-specific gene regulatory networks from these datasets are limited in their ability to integrate scRNA-seq and scATAC-seq measurements and to model network dynamics on a cell lineage. To address this challenge, we have developed single-cell Multi-Task Network Inference (scMTNI), a multi-task learning framework to infer the gene regulatory network for each cell type on a lineage from scRNA-seq and scATAC-seq data. Using simulated, published and newly collected single cell omic datasets, we show that scMTNI is able to accurately infer gene regulatory networks and captures meaningful network dynamics that identify GRN components associated with cell type transitions. Application of our method to mouse cellular reprogramming identified key regulators associated with cell populations that reprogram versus those that are stalled. Taken together, scMTNI is a powerful framework to infer cell type-specific gene regulatory networks and their dynamics from scRNA-seq and scATAC-seq datasets.