This paper presents a method to estimate the covariances of the inputs in a factor-graph formulation for localization under non-line-of-sight conditions. A general solution, based on covariance estimation and M estimators in linear regression problems, is presented that is shown to give unbiased estimators of multiple variances and are robust against outliers. An iteratively re-weighted least squares (IRLS) algorithm is proposed to jointly compute the proposed variance estimators and the state estimates for the non-linear factor graph optimization. The efficacy of the method is illustrated in a simulation study using a robot localization problem under various process and measurement models and measurement outlier scenarios. A case study involving a Global Positioning System (GPS) based localization in an urban environment and data containing multipath problems demonstrates the application of the proposed technique.