We present ABACUS-R, a method based on deep learning for designing amino acid sequences that autonomously fold into a given target backbone. This method predicts the sidechain type of a central residue from its 3-D local environment by using an encoder-decoder network trained with a multi-task learning strategy. The environmental features encoded by the network include the types but not the conformations of the sidechains of surrounding residues. This eliminates the needs for reconstructing and optimizing sidechain structures, and drastically simplifies the sequence design process. Thus iteratively applying the encoder-decoder to different central residues is able to produce self-consistent overall sequences for a target backbone. Extensive results of wet experiments, including five structures solved by X-ray crystallography, show that ABACUS-R outperforms state-of-the-art energy function-based de novo sequence design methods by significant margins in success rate and design precision. ABACUS-R constitutes a robust tool for wide applications in protein design and protein engineering.