Spatial transcriptomics enable us to dissect tissue heterogeneity and map out inter-cellular communications. Optimal integration of transcriptomics data and associated spatial information is essential towards fully exploiting the data. We present SEDR, an unsupervised spatially embedded deep representation of both transcript and spatial information. The SEDR pipeline uses a deep autoencoder to construct a low-dimensional latent representation of gene expression, which is then simultaneously embedded with the corresponding spatial information through a variational graph autoencoder. We applied SEDR on human dorsolateral prefrontal cortex data and achieved better clustering accuracy, and correctly retraced the prenatal cortex development order with trajectory analysis. We also found the SEDR representation to be eminently suited for batch integration. Applying SEDR to human breast cancer data, we discerned heterogeneous sub-regions within a visually homogenous tumor region, identifying a tumor core with pro-inflammatory microenvironment and an outer ring region enriched with tumor associated macrophages which drives an immune suppressive microenvironment.