Robust Service Time Measurement Using Comparison Sequential Test

Abstract

The sequential comparison test is a tool for evaluation of the operational innovation in information technology service delivery processes. Due to the strong variability of these processes, the evaluation is done in comparison with the parallel running servers taken as reference. We consider the streams of service-completion events. When the time between events (TBE) is exponentially distributed, the binomial sequential probability ratio test (SPRT) can be used for evaluation. The effect of deviations from the exponential distribution on the characteristics of the test is analysed. We suggest a novel criterion that allows analysing robustness of the test. We show that the main factor influencing these characteristics is the coefficients of variation (CV) of the TBEs. Thus just by using CV of the TBEs, we may conclude whether the test is robust or not. We also suggest approach of handling the case when test for pair of single TBEs is not robust (case of CV > 1). Transition from a single server to a group of servers and from a single stream to a superposed stream of events improves robustness, since superposition of event streams brings the TBEs’ distribution closer to the exponential. Superposition makes it possible to deal with the problem for CV > 1. The analytical dependency of the fixed sample size test (FSST) robustness vs. CV permits simple estimation of robustness of the test in question. The advantage of the test is shown vs. the FSST, and illustrated on a real-life case.