The diversity of e-commerce Business to Consumer systems and the significant increase in their use during the COVID-19 pandemic as a one of the primary channels of retail commerce, has made all the most important the need to measuring their quality using practical methods. This paper presents a quality evaluation framework for web metrics that are B2C specific. The framework uses three dimensions based on end-user interaction categories, metrics internal specs and quality sub-characteristics as defined by ISO25010. Beginning from the existing large corpus of general-purpose web metrics, e-commerce specific metrics are chosen and categorized. Analysis results are subjected to a data mining analysis to provide association rules between the various dimensions of the framework. Finally, an ontology that corresponds to the framework is developed to answer to complicated questions related to metrics use and to facilitate the production of new, user defined meta-metrics.
Campisi, T., Russo, A., Tesoriere, G., Bouhouras, E. and Basbas, S. (2021) COVID-19’s Effects over E-Commerce: A Preliminary Statistical Assessment for Some European Countries. In: Gervasi, O., et al., Eds., Computational Science and Its Applications—ICCSA 2021. Lecture Notes in Computer Science, Vol. 12954, Springer, Cham, 370-385. https://doi.org/10.1007/978-3-030-86979-3_27
Dionysiou, G., Fouskas, K. and Karamitros, D. (2021) The Impact of Covid-19 in E-Commerce. Effects on Consumer Purchase Behavior. In: Kavoura, A., Havlovic, S.J. and Totskaya, N., Eds., Strategic Innovative Marketing and Tourism in the COVID-19 Era, Springer, Cham, 199-210. https://doi.org/10.1007/978-3-030-66154-0_22
Suguna, M., Shah, B., Raj, S.K., et al. (2021) A Study on the Influential Factors of the Last Mile Delivery Projects during Covid-19 Era. Operations Management Research. https://doi.org/10.1007/s12063-021-00214-y
Winarsih, Indriastuti, M. and Fuad, K. (2021) Impact of Covid-19 on Digital Transformation and Sustainability in Small and Medium Enterprises (SMEs): A Conceptual Framework. In: Barolli, L., Poniszewska-Maranda, A. and Enokido, T., Eds., Complex, Intelligent and Software Intensive Systems. CISIS 2020. Advances in Intelligent Systems and Computing, Vol. 1194, Springer, Cham, 471-476. https://doi.org/10.1007/978-3-030-50454-0_48
Jílková, P. and Králová, P. (2021) Digital Consumer Behaviour and eCommerce Trends during the COVID-19 Crisis. International Advances in Economic Research, 27, 83-85. https://doi.org/10.1007/s11294-021-09817-4
Stefani, A. and Xenos, M. (2009) Meta-Metric Evaluation of E-Commerce-Related Metrics. Electronic Notes in Theoretical Computer Science, 233, 59-72. https://doi.org/10.1016/j.entcs.2009.02.061
International Standardization Organisation (2011) ISO 25010: Software Engineering—Software Product Quality Requirements and Evaluation (SQuaRE)—Measurement Reference Model and Guide. ISO, Switzerland.
Bures, M. (2015) Metrics for Automated Testability of Web Applications. Proceedings of the 16th International Conference on Computer Systems and Technologies, Dublin Ireland, 25-26 June 2015, 83-89. https://doi.org/10.1145/2812428.2812458
Martinez-Ortiz, A.L., Lizcano, D. and Ortega, M. (2019) Software Metrics Artifacts Making Web Quality Measurable. Proceedings of the 14th International Workshop on Automation of Software Test (AST’19), Montreal, 27 May 2019, 1-6. https://doi.org/10.1109/AST.2019.000-2
Mendes, E., Mosley, N. and Counsell, S. (2000) Comparison of Web Size Measures for Predicting Web Design and Authoring Effort. IEEE Software, 149, 86-92. https://doi.org/10.1049/ip-sen:20020337
Mendes, E., Watson, I., Triggs, C., Mosley, N. and Counsell, S. (2002) A Comparison of Development Effort Estimation Techniques for Web Hypermedia Applications. 8th IEEE International Software Metrics Symposium (Metrics 2002), Ottawa, 4-7 June 2002, 131-140. https://doi.org/10.1109/METRIC.2002.1011332
Mendes, E., Counsell, S. and Mosley, N. (2003) Do Adaptation Rules Improve Web Cost Estimation? Proceedings of the 14th ACM Conference on Hypertext and Hypermedia, Nottingham, 26-30 August 2003, 174-183. https://doi.org/10.1145/900051.900091
Nielsen, J. (2000) Designing Web Usability: The Practice of Simplicity. New Riders Publishing, San Francisco.
Offut, J. (2002) Quality Attributes of Web Software Applications. IEEE Software, 19, 25-32. https://doi.org/10.1109/52.991329
Xie, X., Mao, J., Liu, Y., de Rijke, M., Chen, H., Zhang, M. and Ma, S. (2020) Preference-Based Evaluation Metrics for Web Image Search. Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, China, 25-30 June 2020, 369-378. https://doi.org/10.1145/3397271.3401146
Crichton, K., Christin, N. and Cranor, L.F. (2021) How Do Home Computer Users Browse the Web? ACM Transactions on the Web, 16, Article No. 3. https://doi.org/10.1145/3473343
Chen, Y., Zhou, K., Liu, Y., Zhang, M. and Ma, S. (2017) Meta-Evaluation of Online and Offline Web Search Evaluation Metrics. Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, Tokyo, 7-11 August 2017, 15-24. https://doi.org/10.1145/3077136.3080804
Orehovački, T. (2019) Objective and Subjective Metrics Meant for Evaluating Quality of Social Web Applications. Proceedings of the 2019 8th International Conference on Software and Information Engineering, Cairo, 9-12 April 2019, 25-29. https://doi.org/10.1145/3328833.3328852
Abrahão, S., Olsina, L. and Oscar, P. (2003) Towards the Quality Evaluation of Functional Aspects of Operative Web Applications. In: Olivé, A., Yoshikawa, M. and Yu, E.S.K., Eds., Advanced Conceptual Modeling Techniques. ER 2002. Lecture Notes in Computer Science, Vol. 2784, Springer, Berlin, Heidelberg, 325-338. https://doi.org/10.1007/978-3-540-45275-1_29
Tamm, Y.M., Damdinov, R. and Vasilev, A. (2021) Quality Metrics in Recommender Systems: Do We Calculate Metrics Consistently? 15th ACM Conference on Recommender Systems, Amsterdam, 27 September-1 October 2021, 708-713. https://doi.org/10.1145/3460231.3478848
Sloss, B.T., Nukala, S. and Rau, V. (2019) Metrics That Matter. Communications of the ACM, 62, 88. https://doi.org/10.1145/3303874
Song, S., Bu, J., Shen, J., Artmeier, A., Yu, Z. and Zhou, Q. (2018) Reliability Aware Web Accessibility Experience Metric. Proceedings of the 15th International Web for All Conference, Lyon, 23-25 April 2018, Article No. 24. https://doi.org/10.1145/3192714.3192836
Kirsh, I. and Joy, M. (2020) Splitting the Web Analytics Atom: From Page Metrics and KPIs to Sub-Page Metrics and KPIs. Proceedings of the 10th International Conference on Web Intelligence, Mining and Semantics, Biarritz, 30 June 2020-3 July 2020, 33-43. https://doi.org/10.1145/3405962.3405984
Malhotra, R. and Sharma, A. (2015) A Web Metric Collection and Reporting System. Proceedings of the 3rd International Symposium on Women in Computing and Informatics, Kochi, 10-13 August 2015, 661-667. https://doi.org/10.1145/2791405.2791561
Beyer, B., Murphy, N.R., Rensin, D.K., Kawahara, K. and Thorne, S. (2018) The Site Reliability Workbook: Practical Ways to Implement SRE. O’Reilly Media, Sebastopol.
Stefani, A., Xenos, M. and Stavrinoudis, D. (2003) Modeling E-Commerce Systems’ Quality with Belief Networks. Proceedings of the IEEE International Conference on Virtual Environments, Human-Computer Interfaces and Measurement Systems, Lugano, 27-29 July 2003, 13-18.
Chen, L., Gillenson, M. and Sherrell, D. (2004) Consumer Acceptance of Virtual Stores: A Theoretical Model and Critical Success Factors for Virtual Stores. ACM SIGMIS Database: The DATABASE for Advances in Information Systems, 35, 8-31. https://doi.org/10.1145/1007965.1007968
Fogli, D. and Guida, G. (2018) Evaluating Quality in Use of Corporate Web Sites: An Empirical Investigation. ACM Transactions on the Web, 12, Article No. 15. https://doi.org/10.1145/3184646
Diaz, E., Arenas, J.J., Moquillaza, A. and Paz, F. (2019) A Systematic Literature Review About Quantitative Metrics to Evaluate the Usability of E-Commerce Web Sites. In: Karwowski, W. and Ahram, T., Eds., Intelligent Human Systems Integration 2019. Advances in Intelligent Systems and Computing, Vol. 903, Springer, Cham, 332-338. https://doi.org/10.1007/978-3-030-11051-2_51
Abbasi, M.Q., Weng, J., Wang, Y., Wang, I., Rafique, I., Wang, X. and Lew, P. (2012) Modeling and Evaluating User Interface Aesthetics: Employing ISO 25010 Quality Standard. Proceedings of the 8th International Conference on Quality of Information and Communications Technology, Lisbon, 3-6 September 2012, 303-306. https://doi.org/10.1109/QUATIC.2012.39
Fernandez, A., Insfran, E. and Abrahão, S. (2011) Usability Evaluation Methods for the Web: A Systematic Mapping Study. Information and Software Technology, 53, 789-817. https://doi.org/10.1016/j.infsof.2011.02.007
Lachner, F., Fincke, F. and Butz, A. (2017) UX Metrics: Deriving Country-Specific Usage Patterns of a Website Plug-In from Web Analytics. In: Bernhaupt, R., Dalvi, G., Joshi, A., et al., Eds., Human-Computer Interaction—INTERACT 2017. Lecture Notes in Computer Science, Vol. 10515, Springer, Cham, 142-159. https://doi.org/10.1007/978-3-319-67687-6_11
Ivory, M. and Megraw, R. (2005) Evolution of Web Site Design Patterns. ACM Transactions on Information Systems, 23, 463-497. https://doi.org/10.1145/1095872.1095876
Malak, G., Badri, L., Badri, M. and Sahraoui, H. (2004) Towards a Multidimensional Model for Web-Based Applications Quality Assessment. In: Bauknecht, K., Bichler, M. and Pröll, B., Eds., E-Commerce and Web Technologies. EC-Web 2004. Lecture Notes in Computer Science, Vol. 3182, Springer, Berlin, Heidelberg, 316-327. https://doi.org/10.1007/978-3-540-30077-9_32
Fetterly, M. Manasse, M., Najork, J. and Wiener, J. (2004) A Large-Scale Study of the Evolution of Web Pages. Software: Practice and Experience, 34, 213-221. https://doi.org/10.1002/spe.577
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P. and Witten, I.H. (2009) The WEKA Data Mining Software: An Update. ACM SIGKDD Explorations Newsletter, 11, 10-18. https://doi.org/10.1145/1656274.1656278