Although compressive measurements save data storage and bandwidth usage, they are difficult to be used directly for target tracking and classification without pixel reconstruction. This is because the Gaussian random matrix destroys the target location information in the original video frames. This paper summarizes our research effort on target tracking and classification directly in the compressive measurement domain. We focus on one particular type of compressive measurement using pixel subsampling. That is, original pixels in video frames are randomly subsampled. Even in such a special compressive sensing setting, conventional trackers do not work in a satisfactory manner. We propose a deep learning approach that integrates YOLO (You Only Look Once) and ResNet (residual network) for multiple target tracking and classification. YOLO is used for multiple target tracking and ResNet is for target classification. Extensive experiments using short wave infrared (SWIR), mid-wave infrared (MWIR), and long-wave infrared (LWIR) videos demonstrated the efficacy of the proposed approach even though the training data are very scarce.
Li, X., Kwan, C., Mei, G. and Li, B. (2006) A Generic Approach to Object Matching and Tracking. In: Campilho, A. and Kamel, M.S., Eds., Image Analysis and Recognition. ICIAR 2006. Lecture Notes in Computer Science, Springer, Berlin, Heidelberg, 839-849. https://doi.org/10.1007/11867586_76
Zhou, J. and Kwan, C. (2018) Tracking of Multiple Pixel Targets Using Multiple Cameras. In: Huang, T., Lv, J., Sun, C. and Tuzikov, A., Eds., Advances in Neural Networks. Lecture Notes in Computer Science, Springer, Cham, 484-493. https://doi.org/10.1007/978-3-319-92537-0_56
Zhou, J. and Kwan, C. (2018) Anomaly Detection in Low Quality Traffic Monitoring Videos Using Optical Flow. Proceedings of SPIE 10649, Pattern Recognition and Tracking XXIX, 106490F.
Kwan, C., Zhou, J., Wang, Z. and Li, B. (2018) Efficient Anomaly Detection Algorithms for Summarizing Low Quality Videos. Proceedings of SPIE 10649, Pattern Recognition and Tracking XXIX, 1064906. https://doi.org/10.1117/12.2303764
Kwan, C., Chou, B. and Kwan, L. M. (2018) A Comparative Study of Conventional and Deep Learning Target Tracking Algorithms for Low Quality Videos. In: Huang, T., Lv, J., Sun, C. and Tuzikov, A., Eds., Advances in Neural Networks. Lecture Notes in Computer Science, Springer, Cham, 521-531. https://doi.org/10.1007/978-3-319-92537-0_60
Kwan, C., Yin, J. and Zhou, J. (2018) The Development of a Video Browsing and Video Summary Review Tool. Proceedings of SPIE 10649, Pattern Recognition and Tracking XXIX, 1064907. https://doi.org/10.1117/12.2303654
Zhao, Z., Chen, H., Chen, G., Kwan, C. and Li, X.R. (2006) IMM-LMMSE Filtering Algorithm for Ballistic Target Tracking with Unknown Ballistic Coefficient. Proceedings of SPIE, Volume 6236, Signal and Data Processing of Small Targets. https://doi.org/10.1117/12.665760
Zhao, Z., Chen, H., Chen, G., Kwan, C. and Li, X.R. (2006) Comparison of Several Ballistic Target Tracking Filters. Proceedings of American Control Conference, Minneapolis, MN, 14-16 June 2006, 2197-2202.
Candes, E.J. and Wakin, M.B. (2008) An Introduction to Compressive Sampling. IEEE Signal Processing Magazine, 25, 21-30. https://doi.org/10.1109/MSP.2007.914731
Kwan, C., Chou, B., Echavarren, A., Budavari, B., Li, J. and Tran, T. (2018) Compressive Vehicle Tracking Using Deep Learning. IEEE Ubiquitous Computing, Electronics & Mobile Communication Conference, New York City, 8-10 November 2018, 51-56. https://doi.org/10.1109/UEMCON.2018.8796778
Tropp, J.A. (2004) Greed Is Good: Algorithmic Results for Sparse Approximation. IEEE Transactions on Information Theory, 50, 2231-2242. https://doi.org/10.1109/TIT.2004.834793
Yang, J. and Zhang, Y. (2011) Alternating Direction Algorithms for L1-Problems in Compressive Sensing. SIAM Journal on Scientific Computing, 33, 250-278. https://doi.org/10.1137/090777761
Dao, M., Kwan, C., Koperski, K. and Marchisio, G. (2017) A Joint Sparsity Approach to Tunnel Activity Monitoring Using High Resolution Satellite Images. 2017 IEEE 8th Annual Ubiquitous Computing, Electronics and Mobile Communication Conference, New York, 19-21 October 2017, 322-328. https://doi.org/10.1109/UEMCON.2017.8249061
Zhou, J., Ayhan, B., Kwan, C. and Tran, T. (2018) ATR Performance Improvement Using Images with Corrupted or Missing Pixels. Proceedings of SPIE 10649, Pattern Recognition and Tracking XXIX, 106490E.
Applied Research LLC (2017) Phase 1 Final Report.
Kwan, C., Chou, B., Yang, J. and Tran, T. (2019) Target Tracking and Classification Directly in Compressive Measurement for Low Quality Videos. Pattern Recognition and Tracking XXX (Conference SI120). https://doi.org/10.1117/12.2518496
Kwan, C., Gribben, D. and Tran, T. (2019) Multiple Human Objects Tracking and Classification Directly in Compressive Measurement Domain for Long Range Infrared Videos. IEEE Ubiquitous Computing, Electronics & Mobile Communication Conference, New York City.
Kwan, C., Chou, B., Yang, J., and T. Tran, T. (2019) Deep Learning Based Target Tracking and Classification Directly in Compressive Measurement for Low Quality Videos. Signal & Image Processing: An International Journal (SIPIJ).
Kwan, C., Gribben, D. and Tran, T. (2019) Tracking and Classification of Multiple Human Objects Directly in Compressive Measurement Domain for Low Quality Optical Videos. IEEE Ubiquitous Computing, Electronics & Mobile Communication Conference, New York City.
Kwan, C., Chou, B., Yang, J., Rangamani, A., Tran, T., Zhang, J. and Etienne-Cummings, R. (2019) Target Tracking and Classification Directly Using Compressive Sensing Camera for SWIR Videos. Signal, Image, and Video Processing, 13, 1629-1637. https://doi.org/10.1007/s11760-019-01506-4
Kwan, C., Chou, B., Yang, J., Rangamani, A., Tran, T., Zhang, J. and Etienne-Cummings, R. (2019) Target Tracking and Classification Using Compressive Measurements of MWIR and LWIR Coded Aperture Cameras. Journal Signal and Information Processing, 10, 73-95. https://doi.org/10.4236/jsip.2019.103006
Kwan, C., Chou, B., Yang, J., Rangamani, A., Tran, T., Zhang, J. and Etienne-Cummings, R., (2019) Deep Learning based Target Tracking and Classification for Low Quality Videos Using Coded Aperture Camera. Sensors, 19, 3702. https://doi.org/10.3390/s19173702
Yang, M.H., Zhang, K. and Zhang, L. (2012) Real-Time Compressive Tracking. In European Conference on Computer Vision.
He, K., Zhang, X., Ren, S. and Sun, J. (2016) Deep Residual Learning for Image Recognition. 2016 Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, 27-30 June 2016, 770-778. https://doi.org/10.1109/CVPR.2016.90
Redmon, J. and Farhadi, A. (2018) YOLOv3: An Incremental Improvement.
Ren S., He, K., Girshick, R. and Sun, J. (2015) Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. In: Advances in Neural Information Processing Systems, 1-9.
Kwan, C., Chou, B., Yang, J., Budavari, B., and Tran, T. (2019) Compressive Object Tracking and Classification Using Deep Learning for Infrared Videos. Pattern Recognition and Tracking XXX (Conference SI120). https://doi.org/10.1117/12.2518490
Bertinetto, L., Valmadre, J., Golodetz, S., Miksik, O. and Torr, P. (2016) Staple: Complementary Learners for Real-Time Tracking. 2016 Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, 27-30 June 2016, 1401-1409. https://doi.org/10.1109/CVPR.2016.156
Stauffer, C. and Grimson, W.E.L. (1999) Adaptive Background Mixture Models for Real-Time Tracking, Computer Vision and Pattern Recognition. IEEE Computer Society Conference, 2, 2246-2252.