Creating a Dataset to Boost Civil Engineering Deep Learning Research and Application — Oak Academic Publishing
Research ArticleOpen AccessGoogle Scholar indexed
Creating a Dataset to Boost Civil Engineering Deep Learning Research and Application
Graduate Transportation Associate, Tennessee Department of Transportation, Nashville, Tennessee, USA
,
University of Tennessee at Chattanooga, Chattanooga, Tennessee, USA
,
Transportation Engineer, Maryland Department of Transportation, Baltimore, Maryland, USA
,
UC Foundation Professor and Department Head, Department of Civil & Chemical Engineering, The University of Tennessee at Chattanooga, Chattanooga, Tennessee, USA
,
Department of Civil & Chemical Engineering, The University of Tennessee at Chattanooga, Chattanooga, Tennessee, USA
,
Department of Civil & Chemical Engineering, The University of Tennessee at Chattanooga, Chattanooga, Tennessee, USA
1 Graduate Transportation Associate, Tennessee Department of Transportation, Nashville, Tennessee, USA
2 University of Tennessee at Chattanooga, Chattanooga, Tennessee, USA
3 Transportation Engineer, Maryland Department of Transportation, Baltimore, Maryland, USA
4 UC Foundation Professor and Department Head, Department of Civil & Chemical Engineering, The University of Tennessee at Chattanooga, Chattanooga, Tennessee, USA
5 Department of Civil & Chemical Engineering, The University of Tennessee at Chattanooga, Chattanooga, Tennessee, USA
6 Department of Civil & Chemical Engineering, The University of Tennessee at Chattanooga, Chattanooga, Tennessee, USA
With cutting edge deep learning breakthrough, numerous innovations in many fields including civil engineering are stimulated. However, a fundamental issue that civil engineering research community currently facing is lack of a publicly available, free, quality-controlled and human-annotated large dataset that supports and drives civil engineering deep learning research and applications on such as intelligent transportation including connected vehicle, structural health monitoring, and bridge inspection. This paper is a general discussion about demanding needs and construction of a long-anticipated dataset for researchers and engineers in civil engineering and beyond for providing critical training, testing and benchmarking data. The establishment of such a free dataset will remove a major hurdle and boost deep learning research in civil engineering and we hope this work will urge researchers, engineers, government agencies and even computer scientists to work together to start building such datasets. A framework has been developed for the proposed database. Also, some pilot study databases were developed for concrete crack detection, pavement crack detection using normal and infrared thermography, as well as pedestrian and bicyclist detection. A convolution neural network model called Faster RCNN was deployed to check the detection accuracy and a 98% detection accuracy of the proposed datasets was obtained.
Sato, K., Young, C. and Patterson, D. (2017) An In-Depth Look at Google’s First Tensor Processing Unit (TPU). Google Cloud Big Data and Machine Learning Blog, 12.
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K. and Fei-Fei, L. (2009) ImageNet: A Large-Scale Hierarchical Image Database. 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, 20-25 June 2009, 248-255. https://doi.org/10.1109/CVPR.2009.5206848
Chollet, F. (2017) Deep Learning with Python. Manning Publications Co., New York.
Lin, Y.Z., Nie, Z.H. and Ma, H.W. (2017) Structural Damage Detection with Automatic Feature-Extraction through Deep Learning. Computer-Aided Civil and Infrastructure Engineering, 32, 1025-1046. https://doi.org/10.1111/mice.12313
Gulgec, N.S., Takáč, M. and Pakzad, S.N. (2017) Structural Damage Detection Using Convolutional Neural Networks. In: Model Validation and Uncertainty Quantification, Volume 3, Springer, New York, 331-337. https://doi.org/10.1007/978-3-319-54858-6_33
Cha, Y.J., Choi, W. and Büyüköztürk, O. (2017) Deep Learning-Based Crack Damage Detection Using Convolutional Neural Networks. Computer-Aided Civil and Infrastructure Engineering, 32, 361-378. https://doi.org/10.1111/mice.12263
Qurishee, M.A. (2019) Low-Cost Deep Learning UAV and Raspberry Pi Solution to Real Time Pavement Condition Assessment.
Cha, Y.J., Choi, W., Suh, G., Mahmoudkhani, S. and Büyüköztürk, O. (2017) Autonomous Structural Visual Inspection Using Region-Based Deep Learning for Detecting Multiple Damage Types. Computer-Aided Civil and Infrastructure Engineering, 33, 731-747. https://doi.org/10.1111/mice.12334
Maeda, H., Sekimoto, Y., Seto, T., Kashiyama, T. and Omata, H. (2018) Road Damage Detection Using Deep Neural Networks with Images Captured Through a Smartphone. arXiv Preprint arXiv:1801.09454.
Makantasis, K., Protopapadakis, E., Doulamis, A., Doulamis, N. and Loupos, C. (2015) Deep Convolutional Neural Networks for Efficient Vision Based Tunnel Inspection. 2015 IEEE International Conference on Intelligent Computer Communication and Processing (ICCP), Cluj-Napoca, Romania, 3-5 September 2015, 335-342. https://doi.org/10.1109/ICCP.2015.7312681
Varghese, A., Gubbi, J., Sharma, H. and Balamuralidhar, P. (2017) Power Infrastructure Monitoring and Damage Detection Using Drone Captured Images. 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, 14-19 May 2017, 1681-1687. https://doi.org/10.1109/IJCNN.2017.7966053
Wang, F., Kerekes, J.P., Xu, Z. and Wang, Y. (2018) Residential Roof Condition Assessment System Using Deep Learning. Journal of Applied Remote Sensing, 12, Article ID: 016040. https://doi.org/10.1117/1.JRS.12.016040
Russakovsky, O., et al. (2015) ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision, 115, 211-252. https://doi.org/10.1007/s11263-015-0816-y
Krizhevsky, A., Sutskever, I. and Hinton, G.E. (2012) Imagenet Classification with Deep Convolutional Neural Networks. In: Advances in Neural Information Processing Systems, Springer, New York, 1097-1105.
Lin, T.-Y., et al. (2014) Microsoft Coco: Common Objects in Context. In: European Conference on Computer Vision, Springer, New York, 740-755. https://doi.org/10.1007/978-3-319-10602-1_48
Krizhevsky, A., Nair, V. and Hinton, G. (2014) The CIFAR-10 Dataset. https://www.cs.toronto.edu/~kriz/cifar.html
Torralba, A., Fergus, R. and Freeman, W.T. (2008) 80 Million Tiny Images: A Large Data Set for Nonparametric Object and Scene Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30, 1958-1970. https://doi.org/10.1109/TPAMI.2008.128
Y. LeCun, C. Cortes, and C. Burges (2010) MNIST Handwritten Digit Database. AT & T Labs. Volume 2. http://yann.lecun.com/exdb/mnist
Knight, W. (2018) The White House Promises to Release Government Data to Fuel the AI Boom. https://www.technologyreview.com/s/611331/the-white-house-promises-to-release-government-data-to-fuel-the-ai-boom/
Sun, C., Shrivastava, A., Singh, S. and Gupta, A. (2017) Revisiting Unreasonable Effectiveness of Data in Deep Learning Era. 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22-29 October 2017, 843-852. https://doi.org/10.1109/ICCV.2017.97
Downey, A.S. and Olson, S. (2013) Sharing Clinical Research Data: Workshop Summary. National Academies Press, Washington DC.
Chollet, F. (2016) Building Powerful Image Classification Models Using Very Little Data. Volume 13.
Yosinski, J., Clune, J., Bengio, Y. and Lipson, H. (2014) How Transferable Are Features in Deep Neural Networks? In: Advances in Neural Information Processing Systems, Springer, New York, 3320-3328.
Chourabi, H., et al. (2012) Understanding Smart Cities: An Integrative Framework. 2012 45th Hawaii International Conference on System Science (HICSS), Maui, HI, 4-7 January 2012, 2289-2297. https://doi.org/10.1109/HICSS.2012.615
Figueiredo, L., Jesus, I., Machado, J.T., Ferreira, J.R. and De Carvalho, J.M. (2001) Towards the Development of Intelligent Transportation Systems. Intelligent Transportation Systems, 2001. Proceedings, Oakland, CA, 25-29 August 2001, 1206-1211. https://doi.org/10.1109/ITSC.2001.948835
Dimitrakopoulos, G. and Demestichas, P. (2010) Intelligent Transportation Systems. IEEE Vehicular Technology Magazine, 5, 77-84. https://doi.org/10.1109/MVT.2009.935537
Adeli, H. (2008) Smart Structures and Building Automation in the 21st Century. International Symposium on Automation in Construction, 25, 5-10. https://doi.org/10.3846/isarc.20080626.5
Udd, E. (1993) Fiber Optic Smart Structures. Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, Volume 10266. https://doi.org/10.1117/12.145196
Stajano, F., Hoult, N., Wassell, I., Bennett, P., Middleton, C. and Soga, K. (2010) Smart Bridges, Smart Tunnels: Transforming Wireless Sensor Networks from Research Prototypes into Robust Engineering Infrastructure. Ad Hoc Networks, 8, 872-888. https://doi.org/10.1016/j.adhoc.2010.04.002
Lajnef, N., Chatti, K., Chakrabartty, S., Rhimi, M. and Sarkar, P. (2013) Smart Pavement Monitoring System. United States Federal Highway Administration.
Barbaresso, J., Cordahi, G., Garcia, D., Hill, C., Jendzejec, A. and Wright, K. (2014) USDOT’s Intelligent Transportation Systems (ITS) ITS Strategic Plan 2015-2019. https://doi.org/10.1109/JPROC.2011.2132790
Kenney, J.B. (2011) Dedicated Short-Range Communications (DSRC) Standards in the United States. Proceedings of the IEEE, 99, 1162-1182.
Redmon, J., Divvala, S., Girshick, R. and Farhadi, A. (2016) You Only Look Once: Unified, Real-Time Object Detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 779-788. https://doi.org/10.1109/CVPR.2016.91
Ren, S., He, K., Girshick, R. and Sun, J. (2015) Faster r-CNN: Towards Real-Time Object Detection with Region Proposal Networks. In: Advances in Neural Information Processing Systems, Springer, New York, 91-99.
Farrar, C.R. and Worden, K. (2007) An Introduction to Structural Health Monitoring. Philosophical Transactions of the Royal Society of London A: Mathematical, Physical and Engineering Sciences, 365, 303-315. https://doi.org/10.1098/rsta.2006.1928
Kim, S., et al. (2007) Health Monitoring of Civil Infrastructures Using Wireless Sensor Networks. In: Proceedings of the 6th International Conference on Information Processing in Sensor Networks, ACM, New York, 254-263. https://doi.org/10.1145/1236360.1236395
Wu, W., Qurishee, M.A., Owino, J., Fomunung, I., Onyango, M. and Atolagbe, B. (2018) Coupling Deep Learning and UAV for Infrastructure Condition Assessment Automation. 2018 IEEE International Smart Cities Conference (ISC2), Kansas City, MO, 16-19 September 2018, 1-7. https://doi.org/10.1109/ISC2.2018.8656971
Polson, N.G. and Sokolov, V.O. (2017) Deep Learning for Short-Term Traffic Flow Prediction. Transportation Research Part C: Emerging Technologies, 79, 1-17. https://doi.org/10.1016/j.trc.2017.02.024
Lv, Y., Duan, Y., Kang, W., Li, Z. and Wang, F.-Y. (2015) Traffic Flow Prediction with Big Data: A Deep Learning Approach. IEEE Transactions on Intelligent Transportation Systems, 16, 865-873.
Mocanu, E., Nguyen, P.H., Gibescu, M. and Kling, W.L. (2016) Deep Learning for Estimating Building Energy Consumption. Sustainable Energy, Grids and Networks, 6, 91-99. https://doi.org/10.1016/j.segan.2016.02.005
Li, C., Ding, Z., Zhao, D., Yi, J. and Zhang, G. (2017) Building Energy Consumption Prediction: An Extreme Deep Learning Approach. Energies, 10, 1525. https://doi.org/10.3390/en10101525
NHTS Administration (2017) Federal Motor Vehicle Safety Standards; V2V Communications. Federal Register, 82, 3854-4019.
Miller, G.A. (1995) WordNet: A Lexical Database for English. Communications of the ACM, 38, 39-41. https://doi.org/10.1145/219717.219748
Russell, B.C., Torralba, A., Murphy, K.P. and Freeman, W.T. (2008) LabelMe: A Database and Web-Based Tool for Image Annotation. International Journal of Computer Vision, 77, 157-173. https://doi.org/10.1007/s11263-007-0090-8
Von Ahn, L. and Dabbish, L. (2004) Labeling Images with a Computer Game. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, 319-326. https://doi.org/10.1145/985692.985733
Benfield, J.A. and Szlemko, W.J. (2006) Internet-Based Data Collection: Promises and Realities. Journal of Research Practice, 2, 1.
Kramer, V.H. and Weinberg, D.B. (1974) The Freedom of Information Act. The Georgetown Law Journal, 63, 49.
Caltrans. Performance Measurement System (PeMS). http://pems.dot.ca.gov
Dot, T. Smartway. https://smartway.tn.gov/traffic/
GDOT. The Georgia Department of Transportation’s Traffic Analysis and Data Application. https://gdottrafficdata.drakewell.com/publicmultinodemap.asp
UDO Transportation. ITS Public Data Hub. https://www.its.dot.gov/data/
Qurishee, M., Iqbal, I., Islam, M. and Islam, M. (2016) Use of Slag as Coarse Aggregate and Its Effect on Mechanical Properties of Concrete. Proceedings of International Conference on Advances in Civil Engineering, 3, 475-479.
Al Qurishee, M. (2017) Application of Geosynthetics in Pavement Design.
Al Qurishee, M. and Fomunung, I. (2000) Smart Materials in Smart Structural Systems.
Hasnat, A., Qurishee, M., Iqbal, I., Zaman, M. and Wahid, M. (2018) Effectiveness of Using Slag as Coarse Aggregate and Study of Its Impact on Mechanical Properties of Concrete.
Atolagbe, B. (2019) Automatic Mesh Representation of Urban Environments.
Islam, M.A. (2018) Intergrading Connected Vehicle Data into the Transportation Performance Measurement Process. The University of Alabama, Birmingham.
Islam, M.A. (2019) A Literature Review on Freeway Traffic Incidents and Their Impact on Traffic Operations. Journal of Transportation Technologies, 9, 504-516. https://doi.org/10.4236/jtts.2019.94032
Islam, M.A., Sisiopiku, V.P., Ramadan, O.E. and Hadi, M. (2019) A Framework for Performance-Based Traffic Operations Using Connected Vehicle Data. Simulation (NGSIM), 6, No. 8.
Al Qurishee, M., Wu, W., Atolagbe, B., El Said, S. and Ghasemi, A. (2019) Non-Destructive Test Application in Civil Infrastructure.
Al Qurishee, M., Wu, W., Atolagbe, B., El Said, S., Ghasemi, A. and Tareq, S.M. (2008) Wireless Sensor Network and Its Application in Civil Infrastructure.
Cdot (2020) California Department of Transportation. http://pems.dot.ca.gov/