A Bayesian-Inspired Framework for Parameter Estimation and Error Quantification
- 1 Dual Analytics, Geistthal-Södingberg, Austria
- 2 Chair of Mathematics and Statistics, Montanuniversität Leoben, Leoben, Austria
- 3 Materials Center Leoben Forschung GmbH, Leoben, Austria
- 4 Institute of Microwave and Photonic Engineering, Graz University of Technology, Graz, Austria
- 5 Chair of Materials Science, Technical University Munich, Garching, Germany
- 6 Chair of Mechanics, Montanuniversität Leoben, Leoben, Austria
Abstract
This work introduces a novel Bayesian inspired regression method for the simultaneous estimation of model parameters and data uncertainties. The key mathematical result of this framework is an extended least squares objective function. The conventional sum of squared residuals is expanded by adding the logarithms of the computed standard deviations. This approach is particularly useful in cases with strongly varying or parameter-dependent uncertainties. Through five examples, we demonstrate that our extended least squares analysis robustly estimates data uncertainties and quantifies model parameter correlations via the Hessian matrix of the objective function. Finally, in a sixth example from materials science, we applied the method to model the evolution of the dislocation density in martensite during annealing of chromium stainless steel using a Boltzmann function. The approach successfully estimates the values of the parameters, their uncertainties and their correlations. A key outcome of our uncertainty quantification is the derivation of a credible interval for the simulated dislocation densities.
- Gagin, A. and Levin, I. (2015) Accounting for Unknown Systematic Errors in Rietveld Refinements: A Bayesian Statistics Approach. Journal of Applied Crystallography , 48, 1201-1211. https://doi.org/10.1107/s1600576715011322
- Box, G.E.P. (1979) Robustness in the Strategy of Scientific Model Building. In: Launer, R.L. and Wilkinson, G.N., Eds., Robustness in Statistics , Elsevier, 201-236. https://doi.org/10.1016/b978-0-12-438150-6.50018-2
- Dumouchel, W. and O’Brien, F. (1989) Integrating a Robust Option into a Multiple Regression Computing Environment. In: Buja, A. and Tukey, P.A., Eds., Computer Science and Statistics : Proceedings of the 21 st Symposium on the Interface , Springer-Verlag, American Statistical Association, 297-302.
- Holland, P.W. and Welsch, R.E. (1977) Robust Regression Using Iteratively Reweighted Least-Squares. Communications in Statistics - Theory and Methods , 6, 813-827. https://doi.org/10.1080/03610927708827533
- Huber, P.J. (1981) Robust Statistics. Wiley. https://doi.org/10.1002/0471725250
- Street, J.O., Carroll, R.J. and Ruppert, D. (1988) A Note on Computing Robust Regression Estimates via Iteratively Reweighted Least Squares. The American Statistician , 42, 152-154. https://doi.org/10.1080/00031305.1988.10475548
- Singh, K. and Upadhyaya, S. (2012) Outlier Detection: Applications and Techniques. International Journal of Computer Science Issues , 9, 307-323.
- Hodge, V. and Austin, J. (2004) A Survey of Outlier Detection Methodologies. Artificial Intelligence Review , 22, 85-126. https://doi.org/10.1023/b:aire.0000045502.10941.a9
- Green, P.J. (1984) Iteratively Reweighted Least Squares for Maximum Likelihood Estimation, and Some Robust and Resistant Alternatives. Journal of the Royal Statistical Society Series B : Statistical Methodology , 46, 149-170. https://doi.org/10.1111/j.2517-6161.1984.tb01288.x
- Kutner, M.H., Nachtsheim, C.J., Neter, J. and Li, W. (2005) Applied Linear Statistical Models. 5th Edition, McGraw-Hill/Irwin.
- Sivia, D. and Skilling, J. (2006) Data Analysis: A Bayesian Tutorial. Oxford University Press.
- MacKay, D.J.C. (2005) Information Theory, Inference, and Learning Algorithms. Cambridge University Press.
- Bolstad, W.M. and Curran, J.M. (2016) Introduction to Bayesian Statistics. 3rd Edition, Wiley. https://doi.org/10.1002/9781118593165