Heart rate variability (HRV) analysis is often used to estimate human health and fitness status. More specifically, a range of parameters that express the variability in beat-to-beat intervals are calculated from electrocardiogram beat detections. Since beat detection may yield erroneous interval data, these errors travel through the processing chain and may result in misleading parameter values that can lead to incorrect conclusions. In this study, we utilized Monte Carlo simulation on real data, Kolmogorov-Smirnov tests and Bland-Altman analysis to carry out extensive analysis of the noise sensitivity of different HRV parameters. The used noise models consider Gaussian and student-t distributed noise. As a result we observed that commonly used HRV parameters (e.g. pNN50 and LF/HF ratio) are especially sensitive to noise and that all parameters show biases to some extent. We conclude that researchers should be careful when reporting different HRV parameters, consider the distributions in addition to mean values, and consider reference data if applicable. The analysis of HRV parameter sensitivity to noise and resulting biases presented in this work generalizes over a wide population and can serve as a reference and thus provide a basis for the decision about which HRV parameters to choose under similar conditions.