3D layout extraction of a single image is an important aspect of artificial intelligence (AI) applications, e.g., 3D TV, object localization, and scene understanding. The pixel-level 3D layout of a single image can be extracted by using the prior information of image scene geometry. State-of-the-art Lou’s method introduces predefined templates corresponding to 3D scene geometry which use as prior information to extract the pixel-level 3D layout from a single image. 3D scene geometries called as Stages, are rough models of indoor and outdoor scenes. We introduce a new method of 3D layout extraction by using template-based segmentation as prior information and further template-based-segmentation is used as weighting map in random walk method to enhance the edges between two regions. Next, for each sub-region, multiple seeds are initialized by random walk method and their output segments are combined to generate 3D layout of a single image. Proposed algorithm is evaluated on the indoor and outdoor images dataset. The experimental results show that the proposed method has improved the accuracy by 5.31% versus the state-of-the-art methods. Additionally, this algorithm is extended for road scene extraction where instead of using template-based segmentation, the magnitude value of Gaussian derivative uses as weighting map. It is evaluated on KITTI road scene dataset and quantitatively compared to previous efforts on a publically available, provides superior results.