To reduce the computation cost of a combined probabilistic graphical model and a deep neural network in semantic segmentation, the local region condition random field (LRCRF) model is investigated which selectively applies the condition random field (CRF) to the most active region in the image. The full convolutional network structure is optimized with the ResNet-18 structure and dilated convolution to expand the receptive field. The tracking networks are also improved based on SiameseFC by considering the frame relations in consecutive-frame traffic scene maps. Moreover, the segmentation results of the greyscale input data sets are more stable and effective than using the RGB images for deep neural network feature extraction. The experimental results show that the proposed method takes advantage of the image features directly and achieves good real-time performance and high segmentation accuracy.
KeywordsImage SegmentationLocal Region Condition Random Field ModelDeep Neural NetworkConsecutive Shooting Traffic Scene
Gupta, S., Arbeláez, P., Girshick, R. and Malik, J. (2015) Indoor Scene Understanding with RGB-D Images: Bottom-Up Segmentation, Object Detection and Semantic Segmentation. International Journal of Computer Vision, 112, 133-149. https://doi.org/10.1007/s11263-014-0777-6
Wang, P., Shen, X.H., Lin, Z., Cohen, S. and Yuille, A. (2015) Towards Unified Depth and Semantic Prediction from a Single Image. 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, 7-12 June 2015, 2800-2809. https://doi.org/10.1109/CVPR.2015.7298897
Liu, Z., Li, X., Luo, P., Loy, C.C. and Tang, X. (2015) Semantic Image Segmentation via Deep Parsing Network. 2015 IEEE International Conference on Computer Vision, Santiago, 7-13 December 2015, 1377-1385. https://doi.org/10.1109/ICCV.2015.162
Krizhevsky, A., Sutskever, I. and Hinton, G.E. (2012) ImageNet Classification with Deep Convolutional Neural Networks. 2012 International Conference on Neural Information Processing Systems, Lake Tahoe, 3-6 December 2012, 1097-1105.
Long, J., Shelhamer, E. and Darrell, T. (2014) Fully Convolutional Networks for Semantic Segmentation. IEEE Transactions on Pattern Analysis & Machine Intelligence, 39, 640-651. https://doi.org/10.1109/TPAMI.2016.2572683
Simonyan, K. and Zisserman, A. (2015) Very Deep Convolutional Networks for Large-Scale Image Recognition. International Conference on Learning Representations, San Diego, 7-9 May 2015, 1-14.
Badrinarayanan, V., Kendall, A. and Cipolla, R. (2017) SegNet: A Deep Convolutional Encoder-Decoder Architecture for Scene Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39, 2481-2495. https://doi.org/10.1109/TPAMI.2016.2644615
Noh, H., Hong, S. and Han, B. (2015) Learning Deconvolution Network for Semantic Segmentation, 2015 IEEE International Conference on Computer Vision, Santiago, 7-13 December 2015, 1520-1528. https://doi.org/10.1109/ICCV.2015.178
Tu, Z.W. (2008) Auto-Context and Its Application to High-Level Visiontasks. 2008 IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, 23-28 June 2008, 1-8. https://doi.org/10.1109/CVPR.2008.4587436
Ladicky, L., Russell, C., Kohli, P. and Torr, P.H. (2009) Associative Hierarchical CRFs for Object Class Image Segmentation. 12th IEEE International Conference on Computer Vision, Kyoto, 29 September-2 October 2009, 739-746. https://doi.org/10.1109/ICCV.2009.5459248
Mottaghi, R., Chen, X.J,, Liu, X.B., Cho, N.G. and Lee, S.W., et al. (2014) The Role of Context for Object Detection and Semantic Segmentation in the Wild. 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, 23-28 June 2014, 891-898. https://doi.org/10.1109/CVPR.2014.119
Vemulapalli, R., Tuzel, O., Liu, M. and Chellappa, R. (2016) Gaussian Conditional Random Field Network for Semantic Segmentation. 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, 27-30 June 2016, 3224-3233. https://doi.org/10.1109/CVPR.2016.351
Isobe, S. and Arai, S. (2017) Deep Convolutional Encoder-Decoder Network with Model Uncertainty for Semantic Segmentation. 2017 IEEE International Conference on Innovations in Intelligent Systems and Applications, Gdynia, 3-5 July 2017, 365-370. https://doi.org/10.1109/INISTA.2017.8001187
Ronneberger, O., Fischer, P. and Brox, T. (2015) U-Net: Convolutional Networks for Biomedical Image Segmentation. 2015 Medical Image Computing and Computer-Assisted Intervention, Munich, 5-9 October 2015, 234-241. https://doi.org/10.1007/978-3-319-24574-4_28
Zhu, S.P., Xia, X., Zhang, Q.R. and Belloulata, K. (2007) An Image Segmentation Algorithm in Image Processing Based on Threshold Segmentation. 3rd International IEEE Conference on Signal-Image Technologies and Internet-Based System, Shanghai, 16-18 December 2007, 673-678. https://doi.org/10.1109/SITIS.2007.116
Brejl, M. and Sonka, M. (1998) Edge-Based Image Segmentation: Machine Learning from Examples. 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence, Anchorage, 4-9 May 1998, 814-819. https://doi.org/10.1109/IJCNN.1998.685872
Karoui, I., Fablet, R., Boucher, J. M. and Augustin, J.M. (2006) Region-Based Image Segmentation Using Texture Statistics and Level-Set Methods. 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings, Toulouse, 14-19 May 2006, II. https://doi.org/10.1109/ICASSP.2006.1660437
Zhang, Q. and Hu, Y.L. (2012) Image Segmentation Algorithm Based on Spectral Clustering Algorithm. Journal of Shenyang Ligong University, 33, 6.
Chen, L.C., Papandreou, G., Kokkinos, I., et al. (2018) DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. IEEE Transactions on Pattern Analysis & Machine Intelligence, 40, 834-848. https://doi.org/10.1109/TPAMI.2017.2699184
Zheng, S., Jayasumana, S., Romera-Paredes, B., et al. (2015) Conditional Random Fields as Recurrent Neural Networks. 2015 IEEE International Conference on Computer Vision, Santiago, 7-13 December 2015, 1529-1537. https://doi.org/10.1109/ICCV.2015.179
He, K.M., Zhang, X.Y., Ren, S.Q. and Sun, J. (2016) Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, 27-30 June 2016, 770-778. https://doi.org/10.1109/CVPR.2016.90
Yu, F. and Koltun, V. (2016) Multi-Scale Context Aggregation by Dilated Convolutions. International Conference on Learning Representations, Caribe Hilton, 2-4 May 2016, 1-4.
Bertinetto, L., Valmadre, J., Henriques, J.F., et al. (2016) Fully-Convolutional Siamese Networks for Object Tracking. 2016 European Conference on Computer Vision, Amsterdam, 8-10, 15-16 October 2016, 850-865. https://doi.org/10.1007/978-3-319-48881-3_56
Brostow, G.J., Julien, F. and Roberto, C. (2009) Semantic Object Classes in Video: A High-Definition Ground Truth Database. Pattern Recognition Letters, 30, 88-97. https://doi.org/10.1016/j.patrec.2008.04.005
Manthira Moorthi, S., Gambhir, R.K., Misra, I. and Ramakrishnan, R. (2011) Adaptive Stochastic Gradient Descent Optimization in Multi Temporal Satellite Image Registration. 2011 IEEE Recent Advances in Intelligent Computational Systems, Trivandrum, 22-24 September 2011, 373-377. https://doi.org/10.1109/RAICS.2011.6069337
Glorot, X. and Bengio, Y. (2010) Understanding the Difficulty of Training Deep Feedforward Neural Network. Journal of Machine Learning Research, 9, 249-256.
Pont-Tuset, J. and Marques, F. (2016) Supervised Evaluation of Image Segmentation and Object Proposal Techniques. IEEE Transactions on Pattern Analysis & Machine Intelligence, 38, 1465-1478. https://doi.org/10.1109/TPAMI.2015.2481406
Rubinstein, R.Y. and Kroese, D.P. (2004) The Cross-Entropy Method. Springer, New York, 92-92. https://doi.org/10.1007/978-1-4757-4321-0
Liu, T.J., Liu, H.H., Pei, S.C., et al. (2018) A High-Definition Diversity-Scene Database for Image Quality Assessment. IEEE Access, 6, 45427-45438. https://doi.org/10.1109/ACCESS.2018.2864514
Pont-Tuset, J. and Marques, F. (2013) Measures and Meta-Measures for the Supervised Evaluation of Image Segmentation. 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, 23-28 June 2013, 2131-2138. https://doi.org/10.1109/CVPR.2013.277
Wang, J.Y. and Yuille, A. (2015) Semantic Part Segmentation Using Compositional Model Combining Shape and Appearance. 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, 7-12 June 2015, 1788-1797. https://doi.org/10.1109/CVPR.2015.7298788
Wang, P., Shen, X., Lin, Z., Cohen, S., Price, B. and Yuille, A. (2015) Joint Object and Part Segmentation Using Deep Learned Potentials. 2015 IEEE International Conference on Computer Vision, Santiago, 7-13 December 2015, 1573-1581. https://doi.org/10.1109/ICCV.2015.184