The technological evolution emerges a unified (Industrial) Internet of Things network, where loosely coupled smart manufacturing devices build smart manufacturing systems and enable comprehensive collaboration possibilities that increase the dynamic and volatility of their ecosystems. On the one hand, this evolution generates a huge field for exploitation, but on the other hand also increases complexity including new challenges and requirements demanding for new approaches in several issues. One challenge is the analysis of such systems that generate huge amounts of (continuously generated) data, potentially containing valuable information useful for several use cases, such as knowledge generation, key performance indicator (KPI) optimization, diagnosis, predication, feedback to design or decision support. This work presents a review of Big Data analysis in smart manufacturing systems. It includes the status quo in research, innovation and development, next challenges, and a comprehensive list of potential use cases and exploitation possibilities.
KeywordsBig Data AnalysisSmart Manufacturing SystemsData MiningDecision SupportCyber-Physical Systems
ECSEL PMB. (2016) 2016 Multi Annual Strategic Research and Innovation Agenda for ECSEL Joint Undertaking. http://www.smart-systems-integration.org/public/ documents/publications/ECSEL%20MASRIA%202016.pdf
Schwab, K. (2016) The Fourth Industrial Revolution. World Economic Forum.
Chen, M., Mao, S. and Liu, Y. (2014) Big Data: A Survey. Mobile Networks and Applications, 19, 171-209. https://doi.org/10.1007/s11036-013-0489-0
Sloman, A. (2007) What’s a Research Roadmap For? Why Do We Need One? How Can We Produce One? http://www.cs.bham.ac.uk/research/projects/cosy/presentations/munich-roadmap-0701.pdf
SPIRE Association (2013) SPIRE ROADMAP. http://www.spire2030.eu/spire-vision/spire-roadmap
Begeer, R., Berg, A.T., Cohen, A., Colombo, A., Cristau, G., Damm, W., et al. (2016) ARTEMIS Strategic Research Agenda 2016. https://artemis-ia.eu/publication/download/sra2016.pdf
SPARC (2015) Robotics 2020 Multi-Annual Roadmap—For Robotics in Europe. http://sparc-robotics.eu/wp-content/uploads/2014/05/H2020 -Robotics-Multi-Annual-Roadmap-ICT-2016.pdf
EFFRA (2013) FACTORIES OF THE FUTURE—Multi-Annual Roadmap for the Contractual PPP under Horizon 2020. http://www.effra.eu/attachments/article/129/Factories%20of%20the%20Future%202020%20 Roadmap.pdf
Reimann, M. and Rückriegel, C. (2017) Priorities and Recommendations for Research and Innovation in Cyber-Physical Systems. Steinbeis-Edition, Stuttgart.
Taisch, M., Tavola, G. and Carolis, A.D. (2016) European Roadmap for Cyber-Physical Systems in Manufacturing. http://scorpius-project.eu/
openMOS Consortium (2016) openMOS—Open Dynamic Manufacturing Operating System for Smart Plug-and-Produce Automation Components. http://cordis.europa.eu/project/rcn/198382_de.html
TOREADOR Consortium (2016) TOREADOR—TrustwOrthy Model-awaRE Analytics Data platfORm. http://cordis.europa.eu/project/rcn/200253_en.html
LeanBigData Consortium (2016) LeanBigData—Ultra-Scalable and Ultra-Efficient Integrated and Visual Big Data Analytics. http://cordis.europa.eu/project/rcn/191643_en.html
CADENT Consortium (2016) CADENT—Competitive Advantage for the Data-Driven ENTerprise. http://cordis.europa.eu/project/rcn/202864_en.html
Jam Consortium (2016) Jam—Enhancing Fuel Efficiency and Reducing Vehicle Maintenance and Downtime Costs, Using Real-Time Data from Vehicle Sensors (IoT) and a Machine Learning Algorithm for Big Data Analysis. http://cordis.europa.eu/project/rcn/200341_en.html
GDC Consortium (2016) GDC—A Genetic Data CUBE—An Innovative Business Model Applied to Predictive and Prescriptive Analytics, Exploring Big Data and Empowering Cloud-Services and Urgent Computation. http://cordis.europa.eu/project/rcn/198412_en.html
Golden, B. (2014) As IDC Sees It, Tech’s “Third Platform” Disrupts Everyone. http://www.cio.com/article/2377568/cloud-computing/as-idc-sees-it-tech-s-third-platform-disrupts-everyone.html
Wang, Y. (2015) China Automotive Industry IT Application Market Forecast 2015-2019. https://www.idc.com/getdoc.jsp?containerId=CHE40707015
Lukac, D. (2015) The Fourth ICT-Based Industrial Revolution “Industry 4.0”??? HMI and the case of CAE/CAD Innovation with EPLAN P8. 2015 23rd Telecommunications Forum Telfor (TELFOR), 835-838.
Rathwell, G. and Ing, P. (2004) Design of enterprise Architectures. http://www.pera.net/Levels.html
The Open Group (2016) Service Oriented Architecture: What Is SOA?
Kruchten, P.B. (1995) The 4+ 1 View Model of Architecture. IEEE Software, 12, 42-50. https://doi.org/10.1109/52.469759
Ramparany, F., Marquez, F.G., Soriano, J. and Elsaleh, T. (2014) Handling Smart Environment Devices, Data and Services at the Semantic Level with the FI-WARE Core Platform. 2014 IEEE International Conference on Big Data (Big Data), 2014, 14-20. https://doi.org/10.1109/BigData.2014.7004417
DIN SPEC 91345:2016-04 (2016) Reference Architecture Model Industrie 4.0 (RAMI4.0). http://www.beuth.de/de/technische-regel/din-spec-91345/250940128
Wu, X., Zhu, X., Wu, G.-Q. and Ding, W. (2014) Data Mining with Big Data. IEEE Transactions on Knowledge and Data Engineering, 26, 97-107. https://doi.org/10.1109/TKDE.2013.109
University Alliance. What Is Big Data? http://www.villanovau.com/resources/bi/what-is-big-data/
ElMaraghy, H., Lee, J., Kao, H.-A. and Yang, S. (2014) Product Services Systems and Value Creation. Proceedings of the 6th CIRP Conference on Industrial Product-Service Systems Service Innovation and Smart Analytics for Industry 4.0 and Big Data Environment. Procedia CIRP, 16, 3-8. https://doi.org/10.1016/j.procir.2014.02.001
Luckenbach, T., Stackowiak, R., Licht, A. and Mantha, V. (2015) ORACLE—Improving Manufacturing Performance with Big Data—Architect’s Guide and Reference Architecture Introduction. http://www.oracle.com/us/technologies/big-data/big-data-manufacturing-2511058.pdf
Weps, W. (2016) Webinar: Manufacturing Analytics in der Umsetzung. https://www.bosch-si.com/de/newsroom/events/veranstaltungen-72384.html
Bloem, J., Doorn, M.V., Duivestein, S., Excoffier, D., Maas, R. and Ommeren, E.V. (2014) The Fourth Industrial Revolution—Things to Tighten the Link between IT and OT. Sogeti VINT2014.
Labrou, Y. and Finin, T. (1999) Yahoo! As an Ontology: Using Yahoo! Categories to Describe Documents. Proceedings of the Eighth International Conference on Information and Knowledge Management, Kansas City, 2-6 November 1999, 180-187. https://doi.org/10.1145/319950.319976
Ghemawat, S., Gobioff, H. and Leung, S.-T. (2003) The Google File System. Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles, 29–43. https://doi.org/10.1145/945445.945450
Dean, J. and Ghemawat, S. (2008) MapReduce: Simplified Data Processing on Large Clusters. Communications of the ACM, 51, 107-113. https://doi.org/10.1145/1327452.1327492
Shearer, C. (2000) The CRISP-DM Model: The New Blueprint for Data Mining. Journal of Data Warehousing, 5, 13-22.
Manyika, J., Chui, M., Brown, B., Bughin, J., Dobbs, R., Roxburgh, C., et al. (2011) Big Data: The Next Frontier for Innovation, Competition, and Productivity.
Morik, K. (2008) Der CRISP-DM Prozess für Data Mining. http://www-ai.cs.uni-dortmund.de/LEHRE/VORLESUNGEN/KDD/SS08/02_CRISP_4p.pdf
Umair, S. and Haseeb, Q. (2014) A Comparative Study of Data Mining Process Models (KDD, CRISP-DM and SEMMA). International Journal of Innovation and Applied Studies, 12, 217-222.
Brath, R. and Jonker, D. (2015) Graph Analysis and Visualization: Discovering Business Opportunity in Linked Data. Wiley, Hoboken. https://doi.org/10.1002/9781119183662
Oracle (2016) Oracle Big Data Spatial and Graph—Data Sheet. http://download.oracle.com/otndocs/products/bigdata-spatialandgraph/bdsg-data-sheet.pdf
Gupta, A. (2016) Graph Analytics for Big Data. https://www.coursera.org/learn/big-data-graph-analytics
Lohr, S. (2012) The Age of Big Data. The New York Times.
Beyer, M. (2011) Gartner Says Solving “Big Data” Challenge Involves More Than Just Managing Volumes of Data. https://www.gartner.com/newsroom/id/1731916
Grossman, R.L., Kamath, C., Kegelmeyer, P., Kumar, V. and Namburu, R. (2013) Data Mining for Scientific and Engineering Applications. Springer, US.
Kambatla, K., Kollias, G., Kumar, V. and Grama, A. (2014) Trends in Big Data Analytics. Journal of Parallel and Distributed Computing, 74, 2561-2573. https://doi.org/10.1016/j.jpdc.2014.01.003
Valiant, L.G. (1990) A Bridging Model for Parallel Computation. Communications of the ACM, 33, 103-111. https://doi.org/10.1145/79173.79181
Malewicz, G., Austern, M.H., Bik, A.J.C., Dehnert, J.C., Horn, I., Leiser, N., et al. (2010) Pregel: A System for Large-Scale Graph Processing. Proceedings of the 2010 ACM SIGMOD International Conference on Management of Data, 135-146. https://doi.org/10.1145/1807167.1807184
Martella, C. (2012) Apache Giraph: Distributed Graph Processing in the Cloud. FOSDEM.
Angles, R. (2012) A Comparison of Current Graph Database Models. 2012 IEEE 28th International Conference on Data Engineering Workshops (ICDEW), Arlington, 1-5 April 2012, 171-177. https://doi.org/10.1109/ICDEW.2012.31
Maier, A. (2014) Online Passive Learning of Timed Automata for Cyber-Physical Production Systems. 2014 12th IEEE International Conference on Industrial Informatics (INDIN), July 2014, 60-66. https://doi.org/10.1109/INDIN.2014.6945484
Stokic, D., Scholze, S. and Barata, J. (2011) Self-Learning Embedded Services for Integration of Complex, Flexible Production Systems. IECON 2011-37th Annual Conference on IEEE Industrial Electronics Society, Melbourne, 7-10 November 2011, 415-420. https://doi.org/10.1109/IECON.2011.6119346
Wang, G., Su, X. and Pan, X. (2012) Computer Aided Visualized Fault Tree Analysis. 2012 Fifth International Symposium on Computational Intelligence and Design (ISCID), 28-29 October 2012, 265-268. https://doi.org/10.1109/ISCID.2012.74
Rocha, A.D., Monteiro, P.L. and Barata, J. (2015) An Artificial Immune Systems Based Architecture to Support Diagnoses in Evolvable Production Systems Using Genetic Algorithms as an Evolution Enabler.
Brookshear, J.G. (1989) Theory of Computation: Formal Languages, Automata, and Complexity. Benjamin-Cummings Publishing Co., Inc., Redwood City.
Hua, X.L., Gondal, I. and Yaqub, F. (2013) Mobile Agent Based Artificial Immune System for Machine Condition Monitoring. 2013 IEEE 8th Conference on Industrial Electronics and Applications (ICIEA), Melbourne, 19-21 June 2013, 108-113.
Mohammadi, M., Akbari, A., Raahemi, B., Nassersharif, B. and Asgharian, H. (2013) A Fast Anomaly Detection System Using Probabilistic Artificial Immune Algorithm Capable of Learning New Attacks. Evolutionary Intelligence, 6, 135-156. https://doi.org/10.1007/s12065-013-0101-3
Wu, X., Kumar, V., Quinlan, J.R., Ghosh, J., Yang, Q., Motoda, H., et al. (2008) Top 10 Algorithms in Data Mining. Knowledge and Information Systems, 14, 1-37. https://doi.org/10.1007/s10115-007-0114-2
Garshol, L.M. (2012) Introduction to Machine Learning. http://de.slideshare.net/larsga/introduction-to-big-datamachine-learning
Marvuglia, A. and Messineo, A. (2012) Monitoring of Wind Farms’ Power Curves Using Machine Learning Techniques.Applied Energy, 98, 574-583. https://doi.org/10.1016/j.apenergy.2012.04.037
Bange, C. and Grosser, T. (2012) Big Data—BI der nachsten Generation. http://www.computerwoche.de/a/big-data-bi-der-naechsten-generation
Ranjan, V. (2010) A Comparative Study between ETL (Extract, Transform, Load) and ELT (Extract, Load and Transform) Approach for Loading Data into Data Warehouse. http://www.ecst.csuchico.edu/~juliano/csci693/Presentations/2009w/Materials/Ranjan/ Ranjan.pdf
Michalski, R.S., Carbonell, J.G. and Mitchell, T.M. (2013) Machine Learning: An Artificial Intelligence Approach. Springer Science & Business Media, Berlin.
Freitag, D. (2000) Machine Learning for Information Extraction in Informal Domains. Machine Learning, 39, 169-202. https://doi.org/10.1023/A:1007601113994
Andrieu, C., Freitas, N.D., Doucet, A. and Jordan, M.I. (2003) An Introduction to MCMC for Machine Learning. Machine Learning, 50, 5-43. https://doi.org/10.1023/A:1020281327116
Sebastiani, F. (2002) Machine Learning in Automated Text Categorization. ACM Computing Surveys, 34, 1-47. https://doi.org/10.1145/505282.505283
Vora, M.N. (2011) Hadoop-HBase for Large-Scale Data. 2011 International Conference on Computer Science and Network Technology (ICCSNT), Harbin, 24 December 2011, 601-605. https://doi.org/10.1109/ICCSNT.2011.6182030
Jung, M.G., Youn, S.A., Bae, J. and Choi, Y.L. (2015) A Study on Data Input and Output Performance Comparison of MongoDB and PostgreSQL in the Big Data Environment. 2015 8th International Conference on Database Theory and Application (DTA), 25-28 November 2015, 14-17. https://doi.org/10.1109/DTA.2015.14
Moniruzzaman, A.B.M. and Hossain, S.A. (2013) NoSQL Database: New Era of Databases for Big Data Analytics—Classification, Characteristics and Comparison. arXiv:1307.0191 [cs]
Chodorow, K. (2013) MongoDB: The Definitive Guide. O’Reilly Media, Inc.
Wang, G. and Tang, J. (2012) The NoSQL Principles and Basic Application of Cassandra Model. 2012 International Conference on Computer Science Service System (CSSS), 2012, 1332-1335. https://doi.org/10.1109/CSSS.2012.336
George, L. (2011) HBase: The Definitive Guide. O'Reilly Media, Inc.
Lima-Monteiro, P., Parreira-Rocha, M., Rocha, A.D. and Oliveira, J.B. (2017) Big Data Analysis to Ease Interconnectivity in Industry 4.0—A Smart Factory Perspective. In: Service Orientation in Holonic and Multi-Agent Manufacturing, Springer, Berlin, 237-245. https://doi.org/10.1007/978-3-319-51100-9_21
Ristoski, P., Bizer, C. and Paulheim, H. (2015) Mining the Web of Linked Data with RapidMiner. Web Semantics: Science, Services and Agents on the World Wide Web, 35, 142-151. https://doi.org/10.1016/j.websem.2015.06.004
Kart, L., Herschel, G., Linden, A. and Hare, J. (2016) Magic Quadrant for Advanced Analytics Platforms. https://www.gartner.com/doc/reprints?id=1-2YEIILW&ct=160210&st=sb
Berthold, M.R., Cebron, N., Dill, F., Gabriel, T.R., Kotter, T., Meinl, T., et al. (2009), KNIME—The Konstanz Information Miner: Version 2.0 and Beyond. SIGKDD Explorations Newsletter, 11, 26-31. https://doi.org/10.1145/1656274.1656280
Demsar, J., Curk, T., Erjavec, A., Gorup, C., Hocevar, T., Milutinovic, M., et al., (2013) Orange: Data Mining Toolbox in Python. Journal of Machine Learning Research, 14, 2349-2353.
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P. and Witten, I.H. (2009) The WEKA Data Mining Software: An Update. ACM SIGKDD Explorations Newsletter, 11, 10-18. https://doi.org/10.1145/1656274.1656278
Tuimala, D.J. and Kallio, A. (2013) R, Programming Language. In: Dubitzky, W., Wolkenhauer, O., Cho, K.-H. and Yokota, H., Eds., Encyclopedia of Systems Biology, Springer, New York, 1809-1811. https://doi.org/10.1007/978-1-4419-9863-7_619
Williams, G.J., et al. (2009) Rattle: A Data Mining GUI for R. The R Journal, 1, 45-55.
Shanahan, J.G. and Dai, L. (2015) Large Scale Distributed Data Science Using Apache Spark. 2323–2324. https://doi.org/10.1145/2783258.2789993
The Apache Software Foundation (2016) Apache Flink. https://flink.apache.org/
Alexandrov, A., Bergmann, R., Ewen, S., Freytag, J.-C., Hueske, F., Heise, A., et al., (2014) The Stratosphere Platform for Big Data Analytics. The VLDB Journal, 23, 939-964. https://doi.org/10.1007/s00778-014-0357-y
Marz, N. (2014) History of Apache Storm and Lessons Learned. Thoughts from the Red Planet.
Zaharia, M., Chowdhury, M., Franklin, M.J., Shenker, S. and Stoica, I. (2010) Spark: Cluster Computing with Working Sets. HotCloud, 10, 10.
Jovanovic, Z., Bacevic, R., Markovic, R. and Randjic, S. (2015) Android Application for Observing Data Streams from Built-In Sensors Using RxJava. Telecommunications Forum Telfor (℡FOR), 918-921. https://doi.org/10.1109/telfor.2015.7377615
Rocha, A.D., Peres, R. and Barata, J. (2015) An Agent Based Monitoring Architecture for Plug and Produce Based Manufacturing Systems. 2015 IEEE 13th International Conference on Industrial Informatics (INDIN), Cambridge, 22-24 July 2015, 1318-1323. https://doi.org/10.1109/INDIN.2015.7281926
Landset, S., Khoshgoftaar, T.M., Richter, A.N. and Hasanin, T. (2015) A Survey of Open Source Tools for Machine Learning with Big Data in the Hadoop Ecosystem. Journal of Big Data, 2, 1-36. https://doi.org/10.1186/s40537-015-0032-1
Nasser, T. and Tariq, R. (2015) Big Data Challenges. Journal of Computer Engineering & Information Technology, 9307, 2.
Eluri, V.R., Ramesh, M., Al-Jabri, A.S.M. and Jane, M. (2016) A Comparative Study of Various Clustering Techniques on Big Data Sets Using Apache Mahout. 2016 3rd MEC International Conference on Big Data and Smart City (ICBDSC), Muscat, 15-16 March 2016, 1-4. https://doi.org/10.1109/ICBDSC.2016.7460397
De Francisci Morales, G. (2013) SAMOA: A Platform for Mining Big Data Streams. Proceedings of the 2013 International Workshop on Data-Intensive Scalable Computing Systems, Denver, 18 November 2013, 777-778. https://doi.org/10.1145/2487788.2488042
Condie, T., Mineiro, P., Polyzotis, N. and Weimer, M. (2013) Machine Learning for Big Data. SIGMOD Conference, 2013, 939-942.
Nagorny, K., Scholze, S., Barata, J. and Colombo, A.W. (2016) An Approach for Implementing ISA 95-Compliant Big Data Observation, Analysis and Diagnosis Features in Industry 4.0 Vision Following Manufacturing Systems. Technological Innovation for Cyber-Physical Systems: 7th IFIP WG 5.5/SOCOLNET Advanced Doctoral Conference on Computing, Electrical and Industrial Systems, DoCEIS 2016, Costa de Caparica, Portugal, 11-13 April 2016, 116-123.