There exist many iterative methods for computing the maximum likelihood estimator but most of them suffer from one or several drawbacks such as the need to inverse a Hessian matrix and the need to find good initial approximations of the parameters that are unknown in practice. In this paper, we present an estimation method without matrix inversion based on a linear approximation of the likelihood equations in a neighborhood of the constrained maximum likelihood estimator. We obtain closed-form approximations of solutions and standard errors. Then, we propose an iterative algorithm which cycles through the components of the vector parameter and updates one component at a time. The initial solution, which is necessary to start the iterative procedure, is automated. The proposed algorithm is compared to some of the best iterative optimization algorithms available on R and MATLAB software through a simulation study and applied to the statistical analysis of a road safety measure.
KeywordsConstrained Maximum LikelihoodPartial Linear ApproximationSchur’s ComplementIterative AlgorithmsRoad Safety MeasureMultinomial Model
Hauer, E. (1997) Observational Before-After Studies in Road Safety Estimating the Effect of Highway Traffic Engineering Measures on Road Safety. Pergamon, Oxford.
N’Guessan, A., Essai, A. and Langrand, C. (2001) Estimation multidimensionnelle des controles et de l’effet moyen d’une mesure de sécurité routière. Revue de Statistique Appliquée, 49, 85-102.
Lord, D. and Mannering, F. (2010) The Statistical Analysis of Crash-Frequency Data: A Review and Assessment of Methodological Alternatives. Transport Research Part A, 44, 291-305.
Aitchison, J. and Silvey, S.D. (1958) Maximum Likelihood Estimation of Parameters Subject to Restraints. The Annals of Mathematical Statistics, 29, 813-828. https://doi.org/10.1214/aoms/1177706538
Crowder, M. (1984) On the Constrained Maximum Likelihood Estimation with Non-I.I.D Observations. Annals of the Institute of Statistical Mathematics, 36, 239-249. https://doi.org/10.1007/BF02481968
Haber, M. and Brown, M.B. (1986) Maximum Likelihood Methods for Log-Linear Models When Expected Frequencies Are Subject to Linear Constraints. Journal of the American Statistical Association, 81, 477-482. https://doi.org/10.1080/01621459.1986.10478293
Lange, K. (2010) Numerical Analysis for Statisticians. 2nd Edition, Springer, Berlin. https://doi.org/10.1007/978-1-4419-5945-4
Li, Z.F., Osborne, M.R. and Prvan, T. (2003) Numerical Algorithms for Constrained Maximum Likelihood Estimation. ANZIAM Journal, 45, 91-114. https://doi.org/10.1017/S1446181100013171
Slivapulle, M.J. and Sen, P.K. (2005) Constrained Statistical Inference. Wiley, Hoboken.
Wang, Y. (2007) Maximum Likelihood Computation Based on the Fisher Scoring and Gauss-Newton Quadratic Approximations. Computational Statistics & Data Analysis, 51, 3776-3787. https://doi.org/10.1016/j.csda.2006.12.037
Mkhadri, A., N’Guessan, A. and Hafidi, B. (2010) An MM Algorithm for Constrained Estimation in a Road Safety Measure Modeling. Communications in Statistics-Simulation and Computation, 39, 1057-1071. https://doi.org/10.1080/03610911003778119
El Barmi, H. and Dykstra, R.L. (1998) Maximum Likelihood Estimates via Duality for Log-Convex Models When Cell Probabilities Are Subject to Convex Constraints. The Annals of Statistics, 26, 1878-1893. https://doi.org/10.1214/aos/1024691361
Matthews, G.B. and Crowther, N.A.S. (1995) A Maximum Likelihood Estimation Procedure When Modelling in Terms of Constraints. South African Statistical Journal, 29, 29-51.
Liu, C. (2000) Estimation of Discrete Distributions with a Class of Simplex Constraints. Journal of the American Statistical Association, 95, 109-120. https://doi.org/10.1080/01621459.2000.10473907
Ouellette, D.V. (1981) Schur Complement and Statistics. Linear Algebra and Its Applications, 36, 187-295. https://doi.org/10.1016/0024-3795(81)90232-9
Zhang, F. (2005) The Schur Complement and Its Applications. Springer, Berlin. https://doi.org/10.1007/b105056
N’Guessan, A. and Langrand, C. (2005) A Covariance Components Estimation Procedure When Modelling a Road Safety Measure in Terms of Linear Constraints. Statistics, 39, 303-314. https://doi.org/10.1080/02331880500108544
N’Guessan, A., Essai, A. and N’zi, M. (2006) An Estimation Method of the Average Effect and the Different Accident Risks When Modelling a Road Safety Measure: A Simulation Study. Computational Statistics & Data Analysis, 51, 1260-1277. https://doi.org/10.1016/j.csda.2005.09.002
N’Guessan, A. (2010) Analytical Existence of Solutions to a System of Non-Linear Equations with Application. Journal of Computational and Applied Mathematics, 234, 297-304. https://doi.org/10.1016/j.cam.2009.12.026
N’Guessan, A. and Geraldo, I.C. (2015) A Cyclic Algorithm for Maximum Likelihood Estimation Using Schur Complement. Numerical Linear Algebra with Applications, 22, 1161-1179. https://doi.org/10.1002/nla.1999
Nelder, J.A. and Mead, R. (1965) A Simplex Algorithm for Function Minimization. Computer Journal, 7, 308-313. https://doi.org/10.1093/comjnl/7.4.308
Broyden, C.G. (1970) The Convergence of a Class of Double-Rank Minimization Algorithms. Journal of the Institute of Mathematics and Its Applications, 6, 76-90. https://doi.org/10.1093/imamat/6.1.76
Fletcher, R. (1970) A New Approach to Variable Metric Algorithms. The Computer Journal, 13, 317-322. https://doi.org/10.1093/comjnl/13.3.317
Goldfarb, D. (1970) A Family of Variable Metric Updates Derived by Variational Means. Mathematics of Computation, 24, 23-26. https://doi.org/10.1090/S0025-5718-1970-0258249-6
Shanno, D.F. (1970) Conditioning of Quasi-Newton Methods for Function Minimization. Mathematics of Computation, 24, 647-656. https://doi.org/10.1090/S0025-5718-1970-0274029-X
Waltz, R.A., Morales, J.L., Nocedal, J. and Orban, D. (2006) An Interior Algorithm for Nonlinear Optimization That Combines Line Search and Trust Region Steps. Mathematical Programming, 107, 391-408. https://doi.org/10.1007/s10107-004-0560-5
Levenberg, K. (1944) A Method for the Solution of Certain Problems in Least Squares. Quarterly of Applied Mathematics, 2, 164-168. https://doi.org/10.1090/qam/10666
Marquardt, D. (1963) An Algorithm for Least-Squares Estimation of Nonlinear Parameters. SIAM Journal on Applied Mathematics, 11, 431-441. https://doi.org/10.1137/0111030
Byrd, R.H., Schnabel, R. and Shultz, G.A. (1988) Approximate Solution of the Trust Region Problem by Minimization over Two-Dimensional Subspaces. Mathematical Programming, 40, 247-263. https://doi.org/10.1007/BF01580735
Varadhan, R. (2011) Alabama: Constrained Nonlinear Optimization. R Package Version 9-1.
N’Guessan, A. and Truffier, M. (2008) Impact d’un aménagement de sécurité routière sur la gravité des accidents de la route. Journal de la Société Francaise de Statistiques, 149, 23-41.
Geraldo, I.C., N’Guessan, A. and Gneyou, K.E. (2015) A Note on the Strong Consistency of a Constrained Maximum Likelihood Estimator Used in Crash Data Modelling. Comptes Rendus Mathematique, 353, 1147-1152. https://doi.org/10.1016/j.crma.2015.09.025