We review two important fields of application for the method of Variable Projections (VARPRO): Seismic Prospecting and Medical Imaging. We cover the period since 2002 when a first review was published. Variable Projections is a method for the solution of nonlinear least squares problems where the model is a linear combination of nonlinear functions. Its success is based on the fact that the reduced functional, where the linear parameters are eliminated converges always faster than if the same method of solution is applied to the original problem. More importantly, many of these problems are very hard to solve in their original format and VARPRO has shown its value in many different applications.
KeywordsMedical ImagingGeophysicsVariable ProjectionsNonlinear Least Squares
Golub, G.H. and Pereyra, V. (1973) The Differentiation of Pseudo-Inverses and Nonlinear Least Squares Problems Whose Variables Separate. SIAM Journal on Numerical Analysis, 10, 413-432. https://doi.org/10.1137/0710036
Wold, H. and Lyttkens, E. (1969) Nonlinear Iterative Partial Least Squares (NIPALS) Estimation Procedures. Bulletin of the International Statistical Institute, 43, 29-51.
Pereyra, V. and Scherer, G. (2019) Variable Projections for Separable Nonlinear Least Squares Problems since 2002. https://doi.org/10.1155/2019/4861708 http://vpereyra.com/wp-content/uploads/2019/08/Surveypaper2019.pdf
Golub, G.H. and Pereyra, V. (2003) Separable Nnonlinear Least Squares: The Variable Projection Method and Its Applications. Inverse Problems, 19, R1-R26. https://doi.org/10.1088/0266-5611/19/2/201
Pereyra, V., Keller, H.B. and Lee, W.H.K. (1980) Computational Methods for Inverse Problems in Geophysics: Inversion of Travel Time Observations. Physics of the Earth and Planetary Interiors, 21, 120-125. https://doi.org/10.1016/0031-9201(80)90063-1
Pereyra, V. (1996) 3D Block Travel Time Inversion. SEG 66 Anual Meeting, Extended Abstracts. https://doi.org/10.1190/1.1826565
Pereyra, V. and Kaelin, B. (2008) Fast Wave Propagation by Model Order Reduction. Electronic Transactions on Numerical Analysis, 30, 406-419.
Rickett, J. (2013) The Variable Projection Method for Waveform Inversion with an Unknown Source Function. Geophysical Prospecting, 61, 874-881. https://doi.org/10.1111/1365-2478.12008
Symes, W.W. and Kern, M. (1994) Inversion of Reflection Seismograms by Differential Semblance Analysis: Algorithm Structure and Synthetic Examples. Geophysical Prospecting, 42, 565-614. https://doi.org/10.1111/j.1365-2478.1994.tb00231.x
Barnier, G.B., Biondi, E.B. and Biondi, B.B. (2018) Tomographic Full Waveform Inversion via Iterative Least-Squares Migration by Variable Projection Method. 1st EAGE/SBGf Workshop on Least-Squares Migration, Rio de Janeiro, 27-28 November 2018, 1-5. https://doi.org/10.3997/2214-4609.201803066
Barnier, G.B., Biondi, E.B. and Biondi, B.B. (2018) Full Waveform Inversion by Model Extension. SEG Technical Program Expanded Abstracts. https://doi.org/10.1190/segam2018-2998613.1
Yarman, C.E., Kumar, R. and Rickett, J. (2018) A Model-Based Data-Driven Dictionary Learning for Seismic Data Representation. Geophysical Prospecting, 66, 98-123. https://doi.org/10.1111/1365-2478.12533
Mullen, K.M. and van Stokkum, I.H.M. (2007) TIMP: An R Package for Modeling Multi-Way Spectroscopic Measurements. Journal of Statistic Software, 18, 1-46. https://doi.org/10.18637/jss.v018.i03
Skjelvareid, M.H., Heia, K., Olsen, S.H. and Kristian, S. (2017) Detection of Blood in Fish Muscle by Constrained Spectral Unmixing of Hyper-Spectral Images. Journal of Food Engineering, 212, 252-261. https://doi.org/10.1016/j.jfoodeng.2017.05.029
Westwick, D.T. and Perreault, E.J. (2011) Closed-Loop Identification: Application to the Estimation of Limb Impedance in a Compliant Environment. IEEE Transactions on Biomedical Engineering, 58, 521-530. https://doi.org/10.1109/TBME.2010.2096424
Laptenok, S.P., Borst, J.B., Mullen, K.M., van Stokkum, I.H.M., Visser, A.J.W. and van Anerongen, H. (2010) Global Analysis of Forster Resonance Energy Transfer in Live Cells Measured by Fluorescence Lifetime Imaging Microscopy Exploiting the Rise Time of Acceptor Fluorescence. Physical Chemistry Chemical Physics, 12, 7593-7602. https://doi.org/10.1039/b919700a
Sollmann, N., Weidlich, D., Cervantes, B., Klupp, E., Ganter, C., Kooijman, H., Rummeny, E.J., Zimmer, C., Kirschke, J.S. and Karampinos, D.C. (2019) High Isotropic Resolution T2 Mapping of the Lumbosacral Plexus with T2-Prepared 3D Turbo Spin Echo. Clinical Neuroradiology, 29, 223-230. https://doi.org/10.1007/s00062-017-0658-9
Kadrmas, D.J. and Oktay, M.B. (2013) Generalized Separable Parameter Space Techniques for Fitting 1K-5K Serial Compartment Models. Medical Physics, 40, Article ID: 072502. https://doi.org/10.1118/1.4810937
Riabkov, D.Y. and Di Bella, E.V.R. (2004) Blind Identification of the Kinetic Parameters in Three-Compartment Models. Physics in Medicine and Biology, 49, 639-664. https://doi.org/10.1088/0031-9155/49/5/001
Riabkov, D.Y. and Di Bella, E.V.R. (2002) Estimation of Kinetic Parameters without Input Functions: Analysis of Three Methods for Multichannel Blind Identification. IEEE Transactions on Biomedical Engineering, 49, 1318-1327. https://doi.org/10.1109/TBME.2002.804588
Wu, F.X. and Mu, L. (2009) Separable Parameter Estimation Method for Nonlinear Biological Systems. 3rd International Conference on Bioinformatics and Biomedical Engineering, Beijing, 11-16 June 2009, 1-4. https://doi.org/10.1109/ICBBE.2009.5163379
Fisz, J.J. (2007) Fluorescence Polarization Spectroscopy at Combined High-Aperture Excitation and Detection: Application to One-Photon-Excitation Fluorescence Microscopy. The Journal of Physical Chemistry A, 111, 8606-8621. https://doi.org/10.1021/jp072113b
Mullen, K.M. and Van Stokkum, I.H.M. (2009) The Variable Projection Algorithm in Time-Resolved Spectroscopy, Microscopy and Mass Spectrometry Applications. Numerical Algorithms, 51, 319-340. https://doi.org/10.1007/s11075-008-9235-2
Mullen, K.M., Vengris, M. and van Stokkum, H.M. (2007) Algorithms for Separable Nonlinear Least Squares with Application to Modeling Time-Resolved Spectra. Journal of Global Optimization, 38, 201-213. https://doi.org/10.1007/s10898-006-9071-7
Jiang, T., Luo, A., Li, X. and Kruggel, F. (2003) A Comparative Study of Global Optimization Approaches to MEG Source Localization. International Journal of Computer Mathematics, 80, 305-324. https://doi.org/10.1080/0020716022000009255
Volpato, A., Bolzonello, L., Meneghin, E. and Collini, E. (2016) Global Analysis of Coherence and Population Dynamics in 2D Electronic Spectroscopy. Optics Express, 24, 24773-24785. https://doi.org/10.1364/OE.24.024773
van Leeuwen, T., Maretzke, S. and Batenburg, K.J. (2018) Automatic Alignment for Three-Dimensional Tomographic Reconstruction. Inverse Problems, 34, Article ID: 024004. https://doi.org/10.1088/1361-6420/aaa0f8
Bronnikov, A.V. (1999) Numerical Solution of the Identification Problem for the Attenuated Radon Transform. Inverse Problems, 15, 1315-1324. https://doi.org/10.1088/0266-5611/15/5/314
Gourion, D. and Noll, D. (2002) The Inverse Problem of Emission Tomography. Inverse Problems, 18, 1435-1460. https://doi.org/10.1088/0266-5611/18/5/315
Chung, J. and Nguyen, L. (2016) Motion Estimation and Correction in Photo-Acoustic Tomographic Reconstruction. SIAM Journal on Imaging Sciences, 10, 216-224.
Sheng, Q., Wang, K., Matthews, T.P., Xia, J., Zhu, L., Wang, L.V. and Anastasio, M.A. (2015) A Constrained Variable Projection Reconstruction Method for Photo-Acoustic Computed Tomography without Accurate Knowledge of Transducer Responses. IEEE Transactions on Medical Imaging, 34, 2443-2458. https://doi.org/10.1109/TMI.2015.2437356
Kargar, S., Borisch, E.A., Froemming, A.T., Kawashima, A., Mynderse, L.A., Stinson, E.G., Trzasko, J.D. and Riederer, S.J. (2018) Robust and Efficient Pharmacokinetic Parameter Non-Linear Least Squares Estimation for Dynamic Contrast Enhanced MRI of the Prostate. Magnetic Resonance Imaging, 48, 50-61. https://doi.org/10.1016/j.mri.2017.12.021
van Stokkum, I.H.M., Larsen, D.S. and van Grondelle, R. (2004) Global and Target Analysis of Time-Resolved Spectra. Biochimica et Biophysica Acta, 1657, 82-104. https://doi.org/10.1016/j.bbabio.2004.04.011
Dorlhiac, G.F., Fare, C. and van Thor, J.J. (2017) PyLDM—An Open Source Package for Lifetime Density Analysis of Time-Resolved Spectroscopic Data. PLoS Computational Biology, 13, e1005528. https://doi.org/10.1371/journal.pcbi.1005528
Ning, Q., Ma, C., Lam, F. and Liang, Z.P. (2017) Spectral Quantification for High-Resolution MR Spectroscopic Imaging with Spatiospectral Constraints. IEEE Transactions on Biomedical Engineering, 64, 1178-1186. https://doi.org/10.1109/TBME.2016.2594583
Poullet, J.B., Sima, D.M., Simonetti, A.W., De Neuter, B., Vanhamme, L., Lemmerling, P. and Van Huffel, S. (2007) An Automated Quantitation of Short Echo Time MRS Spectra in an Open Source Software Environment: AQSES. NMR in Biomedicine, 20, 493-504. https://doi.org/10.1002/nbm.1112
Sima, D.M. and Van Huffel, S. (2007) Separable Nonlinear Least Squares Fitting with Linear Bound Constraints and Its Application in Magnetic Resonance Spectroscopy Data Quantification. Journal of Computational and Applied Mathematics, 203, 264-278. https://doi.org/10.1016/j.cam.2006.03.025
Sima, D.M., Garcia, M.I.O., Poullet, J., Suvichakorn, A., Antoine, J.-P., Van Huffel, S. and van Ormondt, D. (2009) Line-Shape Estimation for Magnetic Resonance Spectroscopy (MRS) Signals: Self-Deconvolution Revisited. Measurement Science and Technology, 20, Article ID: 104031. https://doi.org/10.1088/0957-0233/20/10/104031
Barral, J.K., Gudmundson, E., Stikov, N., Etezadi-Amoli, M., Stoica, P. and Nishimura, D.G. (2010) A Robust Methodology for in Vivo T1 Mapping. Magnetic Resonance in Medicine, 64, 1057-1067. https://doi.org/10.1002/mrm.22497
Haldar, J.P., Anderson, J. and Sun, S.W. (2007) Maximum Likelihood Estimation of T1 Relaxation Parameters Using VARPRO. Proceedings of the 15th Annual Meeting of ISMRM, Berlin, 41.
Trzasko, J.D., Mostardi, P.M., Riederer, S.J. and Manduca, A. (2013) Estimating T1 from Multichannel Variable Flip Angle SPGR Sequences. Magnetic Resonance in Medicine, 69, 1787-1794. https://doi.org/10.1002/mrm.24401
Fuchs, A., Boesiger, P., Schulte, R.F. and Henning, A. (2014) ProFit Revisited. Magnetic Resonance in Medicine, 71, 458-468. https://doi.org/10.1002/mrm.24703
Schulte, F. and Boesiger, P. (2006) ProFit: Two-Dimensional Prior-Knowledge Fitting of J-Resolved Spectra. NMR in Biomedicine, 19, 255-263. https://doi.org/10.1002/nbm.1026
Stinson, E.G., Trzasko, J.D., Fletcher, J.G. and Riederer, S.J. (2017) Dual Echo Dixon Imaging with a Constrained Phase Signal Model and Graph Cuts Reconstruction. Magnetic Resonance in Medicine, 78, 2203-2215. https://doi.org/10.1002/mrm.26620
Damon, B.M., Gregory, C.D., Hall, K.L., Stark, H.J., Gulani, V. and Dawson, M.J. (2002) Intracellular Acidification and Volume Increases Explain R2 Decreases in Exercising Muscle. Magnetic Resonance in Medicine, 47, 14-23. https://doi.org/10.1002/mrm.10043
Farooq, H., Xu, J., Nam, J.W., Keefe, D.F., Yacoub, E., Georgiou, T. and Lenglet, C. (2016) Microstructure Imaging of Crossing (MIX) White Matter Fibers from Diffusion MRI. Scientific Reports, 6, Article No. 38927. https://doi.org/10.1038/srep38927
Zhao, B., Setsompop, K., Ye, H., Cauley, S.F. and Wald, L.L. (2016) Maximum Likelihood Reconstruction for Magnetic Resonance Fingerprinting. IEEE Transactions on Medical Imaging, 35, 1812-1823. https://doi.org/10.1109/TMI.2016.2531640
Wilson, M. (2018) Robust Retrospective Frequency and Phase Correction for Single-Voxel MR Spectroscopy. Magnetic Resonance in Medicine, 81, 2878-2886. https://doi.org/10.1002/mrm.27605
Zhou, Y. (2018) Estimation and Inference in Problems from Imaging and Biophysics. Ph. D. Thesis, University of Delaware, Newark.
Sollmann, N., Mathonia, N., Weidlich, D., Bonfert, M., Schroeder, S.A., Badura, K.A., Renner, T., Trepte-Freisleder, F., Ganter, C., Krieg, S.M., Zimmer, C., Rummeny, E.J., Karampinos, D.C., Baum, T., Landgraf Mirjam, N. and Heinen, F. (2019) Quantitative Magnetic Resonance Imaging, of the Upper Trapezius Muscles—Assessment of Myofascial Trigger Points in Patients with Migraine. The Journal of Headache and Pain, 20, 8. https://doi.org/10.1186/s10194-019-0960-9
Farooq, H., Xu, J., Yacoub, E., Georgiou, T. and Lenglet, C. (2015) Brain Microstructure Mapping from Diffusion MRI Using Least Squares Variable Separation. CDMRI (MICCAI). https://doi.org/10.1007/978-3-319-28588-7_5
Ma, C., Lam, F., Ning, Q., Johnson, C.L. and Liang, Z.P. (2017) High-Resolution 1H-MRSI of the Brain Using Short-TE SPICE. Magnetic Resonance in Medicine, 77, 467-479. https://doi.org/10.1002/mrm.26130
Poullet, J.B. (2008) Quantification and Classification of Magnetic Resonance Spectroscopic Data for Brain Tumor Diagnosis. Ph. D. Thesis, Katholic University of Leuven, Leuven.
Ruhe, A. and Wedin, P.A. (1980) Algorithms for Separable Nonlinear Least Squares. SIAM Review, 22, 318-337. https://doi.org/10.1137/1022057