Optimized control charts using indifference regions

Abstract

In statistical process monitoring, the CUSUM and EWMA control charts have received considerable attention because of their remarkable ability to detect small sustained shifts. In practice, small process variation and shifts are anticipated beforehand in many processes, so the focus should be on detecting a moderate to a large shift. The aforementioned charts identify minor changes in population parameters as out-of-control scenarios; thus, “small” and potentially practically insignificant shifts are producing signals. To counteract this, both charts are amended to accommodate an indifference region by optimizing the detection of a shift at the outer boundaries of the indifference region. The results show that the adapted CUSUM and EWMA monitoring schemes yield comparable results. On nearly all occasions, the CUSUM chart outperforms the EWMA chart, yet the EWMA chart seems more robust and is easier to interpret. Furthermore, we provide two practical examples to illustrate the use-case of optimized charts to mitigate small (unimportant) variations, such as seasonality and modest temporary shifts. Overall, this work provides a general approach tailored to practice in quality control, e.g., as prescribed by ISO standards. It also answers a recent call in statistical process monitoring literature to reconsider the design of control charts.

Keywords:

• average run length
• control charts
• CUSUM chart
• EWMA chart
• indifference region
• optimization
• statistical process monitoring

1. Introduction

Control charts are commonly used tools in statistical process monitoring (SPM). These charts aim to detect changes (e.g., shifts) or incidents in processes as a result of special causes. These changes are often evaluated in terms of shifts from some in-control process parameter, such as the mean or standard deviation. The aim is to detect such a shift as quickly as possible, which has been the starting point for a rich literature proposing various methods. For an overview of current directions in theory and applications of SPM, we refer to Woodall and Montgomery (2014).

The common practice for designing a control chart is to start with defining or modeling the in-control process behavior. For example, assuming normally distributed data, this can be done by estimating the in-control mean μ₀ and standard deviation σ₀. When monitoring the mean of a variable, the process is typically considered out-of-control in the literature when the current process mean $\mu ={\mu }_0+\delta {\sigma }_0/\sqrt{n}$ is unequal to μ₀ (equivalently $\delta \ne 0$), where δ represents the shift size and n, is the subgroup size. Control charts are typically designed to satisfy a specified false alarm rate (average run length) as long as the process is in control, which we denote as FAR₀ (ARL₀). Depending on the type of control chart, there may be options to tune the out-of-control performance to detect shifts of a specified size δ more quickly. However, for any value of μ other than μ₀, the process is usually considered out of control.

Recently, it has been questioned to reconsider this paradigm as it might misfit practice, for example, when applied in industry (Woodall and Faltin 2019). In their discussions, the authors appeal to amend the control chart design by using an in-control region in which a slight shift should not always be considered an out-of-control situation. They provide examples and explanations of when and why this approach would be beneficial.

The approach of using an indifference region has also received attention in the past, as can be found in Ewan and Kemp (1960); Freund (1957, 1960); Woodall (1985, 1986); Yashchin (1985, 2018), amongst others. The returning underlying motivation is that many assignable causes may lead to real but small process changes that may not be important to practice. For example, many processes contain minor seasonal influences or other day-to-day variations (e.g., different operators) that cause small (temporal) shifts in the process parameters. These effects may not be part of, or go unnoticed in a Phase I study. For example, this can occur when the considered time period is too short to incorporate seasonal patterns or when an estimator cannot completely capture this information. These issues can also be caused by overdispersion, as discussed in Goedhart and Woodall (2022). Furthermore, suppose the variation is small relative to the process specification limits and is temporal (such as for seasonal variation). In that case, it may be desirable to have an approach that can disregard these types of shifts that normally would cause signals. Acting upon these irrelevant signals even aggravates process variation—a phenomenon that is called overadjustment.

Related to this framework are the complementary concepts of statistical and practical significance in control charts. A statistically significant result does not necessarily imply practical importance. For similar discussions in a hypothesis framework, see for example Wasserstein and Lazar (2016), Snee and Hoerl (2018), Woodall and Faltin (2019), and Blume et al. (2019). The consideration of using indifference regions also resonates with the international standard as prescribed by the International Organization for Standardization (ISO 2020). In this specific standard, the use of acceptance control charts (Freund 1957) is recommended, which exhibits a region wherein the user is indifferent about the chart’s performance. An illustration of this is provided in Figure 1. In the figure, three regions centered around the target level are distinguished:

Figure 1. The theoretical setting of an indifference region (white); process variation manifests itself vertically. The upper and lower A_PL demarcate the acceptable region for the underlying process, with the center line (dashed) indicating the target value. The two R_PL lines demarcate the boundaries whereafter it is rejectable. Figure inspired by ISO 7870-3:2020.

1. The rejectable (out-of-control) region, which is outside the upper and lower Rejectable Process Limit (R_PL).

2. The indifference region, which is between the Acceptable Process Limit (A_PL) and the R_PL.

3. The acceptable (in-control) region, which is between the upper and lower A_PL. Within this region, the target or μ₀ (center line) is indicated by the “4” in the middle.

Note that the displayed regions relate to limits for individual observations. However, as mentioned in the standard (ISO 2020), the focus is on process acceptability rather than product disposition, and the limits are to be determined accordingly. The consequence is that the in-control situation is no longer considered a single value of the process metric of interest but an interval. This is the essence of the three-region approach discussed by Woodall and Faltin (2019). Such an approach qualifies for situations where the variation in the process can be tolerated to a larger extent, for example, when the specification limits are much wider than typical process variation. For this purpose, Woodall and Faltin (2019) illustrate how this paradigm can be accommodated by a cumulative sum (CUSUM) chart.

Performance comparisons of various control charts in different regimes have been fertile research ground, e.g., Srivastava and Wu (1993); Hawkins and Wu (2014); Diko, Goedhart, and Does (2020). As an extension of this line of research, we demonstrate how the CUSUM, EWMA, and Shewhart control charts can be designed and optimized using indifference regions and compare their performance for various scenarios.

The paper is organized as follows. In Section 2, we outline the control charts considered, the objectives, and the notation. In Section 3, we evaluate and compare the performance and robustness of the optimized control charts for various indifference regions in different scenarios. Next, a discussion is provided in Section 4, followed by two examples in Section 5 that demonstrate its value to practice. Finally, we conclude in Section 6.

2. Definitions and monitoring schemes

In this section, we outline the Shewhart, CUSUM, and EWMA control charts and submit them to the indifference region approach. We consider two-sided Phase II control charts to monitor a process for which observations X_ij are obtained each time period i in subgroups of size n, with $j=1,\dots ,n.$ The X_ij are assumed to follow a normal distribution with mean ${\mu }_0+\delta {\sigma }_0/\sqrt{n}$ and standard deviation σ₀, where δ represents the (standardized) size of a possible shift, and where μ₀ and σ₀ are assumed to be known following a Phase I study. We refer to Jones-Farmer et al. (2014) for an overview of Phase I analysis for process improvement and control. Note that δ = 0 means that the process is typically considered in-control.

Throughout we consider the standardized variables $Z_i=\sqrt{n}({\overline{X}}_i-{\mu }_0)/{\sigma }_0,$ where ${\overline{X}}_i=\frac{1}{n}{\sum }_{j=1}^n{X}_{\mathit{\text{ij}}}.$ Note that Z_i follows a standard normal distribution for δ = 0 and a $N(\delta ,1)$ distribution for $\delta \ne 0.$ If the in-control X_ij are not normally distributed, the well-known Box-Cox approach (for a comprehensive overview, see Sakia (1992)), or the approach of Chou, Polansky, and Mason (1998) using a Johnson transformation can be used to achieve (approximate) normality of these data. Evidently, transformations should be used with care; see, for example, the work by Khakifirooz, Tercero-Gómez, and Woodall (2021), who show that outliers can be masked in Phase I if transformations are used. Santiago and Smith (2013) showed clear shortcomings when using a transformation to normality when data stems from the exponential distribution.

2.1. Indifference regions and optimization framework

In Phase II, the monitoring phase, the process mean equals $\mu ={\mu }_0+\delta {\sigma }_0/\sqrt{n}.$ In most applications, the process is considered out-of-control for any value of $\delta \ne 0,$ which is different when using indifference regions. In particular, when considering the use of an in-control region as discussed in Woodall and Faltin (2019) and in the ISO standard (see Figure 1), a region is determined for which the process is considered in-control, even when $\mu \ne {\mu }_0.$ In the case of a two-sided control chart for detecting an increase or decrease in the process mean, the indifference region approach could be described as follows:

• The in-control region, $0\le \left|\delta \right|\le {\delta }_0,$ where δ₀ is the acceptable (standardized) shift size (worst acceptable).

• The indifference region: ${\delta }_0<\left|\delta \right|\le {\delta }_1,$ with δ₁ the shift size desired to be detected quickly (best unacceptable).

• The out-of-control region: $\left|\delta \right|>{\delta }_1.$

Comparing this set-up to that of Figure 1, then we observe that ${\delta }_0=A_{P{L}_U}-{\mu }_0={\mu }_0-A_{P{L}_L}$ and ${\delta }_1=R_{P{L}_U}-{\mu }_0={\mu }_0-R_{P{L}_L}.$ So, both δ₀ and δ₁ are to be determined based on the practitioner’s knowledge of which shifts are practical significance in the process. After this, choices need to be made on the desired properties of the control chart. Woodall and Faltin (2019) implement this three-region approach by adjusting the CUSUM chart to match the performance of the classical Shewhart control chart with standard limits at a given level $\delta ={\delta }_0.$

In this paper, we adapt this approach by adjusting the control charts to match a pre-specified in-control value ARL₀ when $\delta ={\delta }_0.$ Then, we optimize the out-of-control performance to detect shifts of size $\delta ={\delta }_1$ as quickly as possible. These two goals set the parameters of the CUSUM and EWMA control charts. Note that the Shewhart control chart depends on only one parameter, such that the value of ARL₀ dictates both the in-control and out-of-control performance directly; there is no optimization step.

2.2. Shewhart control chart

The standard Shewhart control chart comprises an upper control limit (UCL) and a lower control limit (LCL). It depends on a single parameter (the control limit constant) which we denote as L_S. The charting statistic for the Shewhart control chart equals Z_i, and the UCL and LCL equal L_S and $-L_S,$ respectively. A signal is provided if the charting statistic is above the UCL or below the LCL. A common choice for the control limit constant is L_S = 3, which yields a false alarm rate (FAR₀) of 0.0027 or, equivalently, an in-control average run length (ARL₀) of 370.4. Since $\mathit{\text{AR}}{L}_0=1/\mathit{\text{FA}}{R}_0$ for the Shewhart control chart, the required value of L_S when using indifference regions as described in Section 2.1 is found by solving the following equation: (1) $$1/\mathit{\text{AR}}{L}_0=1-P(\mathit{\text{LCL}}<Z_i<\mathit{\text{UCL}})=1-\Phi (L_s-{\delta }_0)+\Phi (-L_s-{\delta }_0),$$(1) where $\Phi (X)$ is the standard normal cumulative distribution function, and where $Z_i\sim N({\delta }_0,1).$

2.3. The CUSUM control chart

The CUSUM control chart was originally introduced by Page (1954). In this paper, we consider the standardized CUSUM chart as described by Montgomery (2020), which consists of two charting statistics, namely: (2) $$\begin{array}{cc}\hfill & C_i^{+}=\mathrm{\text{max}}(0,C_{i-1}^{+}+Z_i-k);\hfill \\ \hfill & C_i^{-}=\mathrm{\text{min}}(0,C_{i-1}^{-}+Z_i+k),\hfill \end{array}$$(2) where $C_0^{+}=C_0^{-}=0$ and k > 0 represents a reference value. A signal is given if $C_i^{+}>L_C$ or $C_i^{-}<-L_C,$ where L_C represents the control limit constant for the CUSUM chart.

The CUSUM chart is thus based on two parameters, the reference value k and the control limit constant L_C. The optimal reference value k to detect a shift of size δ₁ has been proven to be $k={\delta }_1/2,$ see for example Woodall (1986); Woodall and Adams (1993). The value of L_C can then be chosen to provide a specified value of $\mathit{\text{AR}}{L}_0.$

When considering the indifference region approach, note that an increasing shift toward $\delta ={\delta }_1$ can be viewed as a shift of size $({\delta }_1-{\delta }_0)$ from the target value δ₀, for which the optimal value of k would be $k=({\delta }_1-{\delta }_0)/2.$ The same holds, of course, for a decreasing shift from the target value $-{\delta }_0$ to $-{\delta }_1.$ This was also pointed out by Woodall and Faltin (2019), and can be incorporated as a target value in Equation (2)(2) $$\begin{array}{cc}\hfill & C_i^{+}=\mathrm{\text{max}}(0,C_{i-1}^{+}+Z_i-k);\hfill \\ \hfill & C_i^{-}=\mathrm{\text{min}}(0,C_{i-1}^{-}+Z_i+k),\hfill \end{array}$$(2) as follows: (3) $$\begin{array}{cc}\hfill & C_i^{+}=\mathrm{\text{max}}(0,C_{i-1}^{+}+Z_i-({\delta }_0+k));\hfill \\ \hfill & C_i^{-}=\mathrm{\text{min}}(0,C_{i-1}^{-}+Z_i+({\delta }_0+k)),\hfill \end{array}$$(3)

Since ${\delta }_0+k={\delta }_0+({\delta }_1-{\delta }_0)/2=({\delta }_1+{\delta }_0)/2,$ the optimal CUSUM chart when using our indifference region approach is equivalent to using $k=({\delta }_1+{\delta }_0)/2$ in the standard CUSUM of Equation (2)(2) $$\begin{array}{cc}\hfill & C_i^{+}=\mathrm{\text{max}}(0,C_{i-1}^{+}+Z_i-k);\hfill \\ \hfill & C_i^{-}=\mathrm{\text{min}}(0,C_{i-1}^{-}+Z_i+k),\hfill \end{array}$$(2) . The parameter L_C can then be tuned to match the specified ARL₀ for $\delta ={\delta }_0,$ which can be found using the approaches implemented in R by Knoth (2014).

2.4. The EWMA control chart

The EWMA control chart was introduced by Roberts (1959) as an alternative to the standard Shewhart control chart to be more capable of detecting small shifts. To do so, the charting statistic of the EWMA control chart (denoted as Y_i) takes a weighted average of the new observation and past performance and can be denoted as follows: (4) $$Y_i=(1-\lambda )Y_{i-1}+\lambda Z_i,$$(4) where $0<\lambda <1$ represents the weight given to the most recent observation, and where $Y_0=0.$ A signal is given when (5) $$\left|Y_i\right|>L_E \sqrt{\frac{\lambda }{2-\lambda }(1-{(1-\lambda )}^{2i})},$$(5) where L_E represents the control chart constant for the EWMA control chart.

Two parameters have to be chosen for the design of an EWMA monitoring scheme, namely λ and L_E. Note that when λ = 1, the EWMA chart is equivalent to the Shewhart control chart; for more details, see the seminal work by Lucas and Saccucci (1990). Typically, smaller values of λ are used when focusing on detecting minor shifts. However, contrary to the CUSUM control chart, there is no closed-form solution for the optimal value for either of the two parameters of the EWMA control chart. Montgomery (2020) found that values for λ in the interval $0.05\le \lambda \le 0.25$ work well in practice, while also mentioning that values 0.05 and 0.1 with appropriate control limits perform very well for both normal and non-normal distributions.

An optimization variant of the EWMA chart has been studied in Aparisi and García-Díaz (2007). First, they set several criteria—either predetermined or found via using a Taguchi loss function—on the in-control performance, possibly using an asymmetric indifference region. Next, they employ a genetic algorithm to optimize the EWMA chart while meeting the criteria at the outer ends of the in-control region. Our approach does not require the use of such a meta-heuristic. It directly searches for the quickest detection of (out-of-control) shifts of a particular size δ₁ while ensuring a specific in-control performance ARL₀ at δ₀, which determines the two parameters λ and L_E. To do so, similar to Hawkins and Wu (2014), we apply a grid search to determine the optimal combination of parameters for the chosen settings of ARL₀, δ₀, and δ₁. We use the approaches implemented in R by Knoth (2014) for this part. Furthermore, to make the comparison with the CUSUM control chart fairer, we henceforth take the steady-state limits that are obtained when $i\to \infty$ in Equation (5)(5) $$\left|Y_i\right|>L_E \sqrt{\frac{\lambda }{2-\lambda }(1-{(1-\lambda )}^{2i})},$$(5) , which means a signal is only produced if $\left|Y_i\right|>L_E \sqrt{\frac{\lambda }{2-\lambda }}.$

3. Comparing Shewhart, CUSUM and EWMA control charts

In the previous section, we outlined the methodology for adapting the Shewhart control chart and optimizing the EWMA and CUSUM monitoring schemes to incorporate the indifference region approach. Under these optimized settings for the different control charting schemes, we again use the routines provided by Knoth (2014) in R to compute the corresponding ARLs. We compare the various monitoring schemes in an extensive head-to-head comparison, considering the ARLs to detect an out-of-control shift. We also provide and discuss the optimal parameter values for the control chart designs. Subsequently, we will consider some robustness checks: zero-state versus steady-state performance in Section 3.2; dependency of the approach on Phase I estimates in Section 3.3; and finally, the impact of non-normality in Section 3.4.

3.1. Performance evaluation

For our evaluation we start with δ₀ equal to 0 (only an in-control point), and also consider δ₀ equal to 0.5, 1, and 2. For the out-of-control situations, we consider a tiny (${\delta }_1={\delta }_0+0.5$), small (${\delta }_1={\delta }_0+1$), medium (${\delta }_1={\delta }_0+2$), large (${\delta }_1={\delta }_0+3$), and a very large (${\delta }_1={\delta }_0+4$) shift. All charts are adjusted to yield a pre-specified value of ARL₀ when $\delta ={\delta }_0,$ where we considered $\mathit{\text{AR}}{L}_0=100$ and $\mathit{\text{AR}}{L}_0=500$ to use a smaller and larger value for comparison purposes.

In Tables 1 and 2, we list the parameter values obtained from the calculations described in Section 2 for the three control charts considered for $\mathit{\text{AR}}{L}_0=100$ and $\mathit{\text{AR}}{L}_0=500,$ respectively. For the Shewhart chart, a higher value of δ₀ obviously leads to a larger value of L_S. For the CUSUM chart, a larger value of δ₀ comes with a larger value of k but a lower L_C, rapidly decreasing when δ₁ increases. Moreover, the L_C parameter is hardly affected by an increase in δ₀ when ${\delta }_0\ge \mathrm{0.5.}$ To be specific, we observe similar patterns in the L_C values of δ₀ with equal size of the indifference region ${\delta }_1-{\delta }_0;$ e.g., $L_C=4.419,$ 4.418, and 4.418 for pairs $({\delta }_0,{\delta }_1)=(0.5,1),(1,1.5),$ and $(2,2.5),$ respectively.

Table 1. Shewhart, CUSUM, and EWMA control chart parameters for $\mathit{\text{AR}}{L}_0=100.$

		Shewhart	CUSUM		EWMA
δ 0	δ1	LS	k	LC	λ	LE
0	0.5		0.250	5.597	0.066	1.994
	1		0.500	3.502	0.183	2.336
	2	2.576	1.000	1.874	0.493	2.532
	3		1.500	1.131	0.788	2.570
	4		2.000	0.582	0.942	2.575
0.5	1		0.750	4.419	0.213	3.466
	1.5		1.000	2.852	0.360	3.250
	2.5	2.842	1.500	1.543	0.704	2.986
	3.5		2.000	0.876	0.959	2.860
	4.5		2.500	0.345	1.000	2.842
1	1.5		1.250	4.418	0.261	4.660
	2		1.500	2.849	0.422	4.149
	3	3.327	2.000	1.532	0.803	3.535
	4		2.500	0.861	1.000	3.327
	5		3.000	0.329	1.000	3.327
2	2.5		2.250	4.418	0.310	6.798
	3		2.500	2.849	0.487	5.766
	4	4.326	3.000	1.532	0.937	4.454
	5		3.500	0.860	1.000	4.326
	6		4.000	0.329	1.000	4.326

Table 2. Shewhart, CUSUM, and EWMA control chart parameters for $\mathit{\text{AR}}{L}_0=500.$

		Shewhart	CUSUM		EWMA
δ 0	δ1	LS	k	LC	λ	LE
0	0.5		0.250	8.585	0.047	2.594
	1		0.500	5.071	0.134	2.883
	2	3.090	1.000	2.665	0.365	3.045
	3		1.500	1.708	0.676	3.085
	4		2.000	1.110	0.886	3.090
0.5	1		0.750	7.267	0.118	4.568
	1.5		1.000	4.389	0.238	4.106
	2.5	3.386	1.500	2.326	0.505	3.710
	3.5		2.000	1.473	0.832	3.472
	4.5		2.500	0.900	1.000	3.386
1	1.5		1.250	7.267	0.140	6.261
	2		1.500	4.389	0.274	5.281
	3	3.878	2.000	2.323	0.553	4.473
	4		2.500	1.467	0.912	3.970
	5		3.000	0.892	1.000	3.878
2	2.5		2.250	7.267	0.163	9.378
	3		2.500	4.389	0.312	7.439
	4	4.878	3.000	2.323	0.608	5.888
	5		3.500	1.466	1.000	4.878
	6		4.000	0.892	1.000	4.878

To provide some intuition on this pattern, note that a unit increase of δ₀ when δ₀ is already large increases the limits of the Shewhart with the same size. For example, considering the cases ${\delta }_0=1$ and ${\delta }_0=2,$ the values of L_S are 3.327 and 4.326, respectively. For the CUSUM chart, this change is incorporated in the value k. For example, when δ₀ increases from 1 to 2, k jumps by 1 while the L_C values remain the same. This is because the charts essentially become one-sided for large $\left|{\delta }_0\right|.$ Focusing on the EWMA control chart, the optimal parameter patterns are more convoluted because of Equation (5)(5) $$\left|Y_i\right|>L_E \sqrt{\frac{\lambda }{2-\lambda }(1-{(1-\lambda )}^{2i})},$$(5) . Larger δ₀ values result in larger values of λ and L_E. The L_E values increase in δ₁ when ${\delta }_0=0,$ whereas when ${\delta }_0\ne 0,$ L_E decreases if δ₁ increases. Finally, when δ₁ increases, the EWMA control chart’s parameters converge to its Shewhart counterpart, i.e., λ close to 1 and L_E = L_S, echoing the literature that has acknowledged the excellent capability of a Shewhart control chart to detect large shifts.

From a computational point of view, note that all optimizations were performed by relying on a standard grid search of ARL-computation functions of Knoth (2014). However, we found that a small modification was needed in two settings. To be precise, in the case of ${\delta }_0=2$ and ${\delta }_1=2.5$ for $\mathit{\text{AR}}{L}_0=100$ and $\mathit{\text{AR}}{L}_0=500$ the grid search had to be adjusted, bounded from below to circumvent it to generate unrealistic parameter combinations or errors. The bounds can easily be retrieved from neighboring settings. Second, in these cases, we also had to increase the number of nodes for the Gauss-Legendre quadrature underlying the Nystroem method to solve the related ARL integral equation; in these cases, they were increased to 120, whereas in all other experiments, 40 were already sufficient.

In the Tables 3, 4, 5, and 6 the ARL values are obtained by using the control chart schemes as proposed in this paper. They are generated by setting $\mathit{\text{AR}}{L}_0=500$ at various δ₀ levels (δ₀ equal to 0, 0.5, 1, and 2, respectively). For clarity, the settings used to optimize the control charting schemes (i.e., the required ARL₀ at δ₀ and the detection of a shift of size δ₁) are displayed in bold type in reported performance across all tables. Note also that the choice of δ₁ does not impact the Shewhart chart, as it has only one parameter (L_S). Note that the performance difference between the optimized CUSUM and the EWMA control charts is typically small, and a CUSUM control chart generally outperforms the EWMA chart.

Table 3. ARL values for $\mathit{\text{AR}}{L}_0=500$ and ${\delta }_0=0.$

		${\delta }_1=0.5$		${\delta }_1=1$		${\delta }_1=2$		${\delta }_1=3$
δ	Shewhart	CUSUM	EWMA	CUSUM	EWMA	CUSUM	EWMA	CUSUM	EWMA
0	500	500	500	500	500	500	500	500	500
0.5	202	31.1	28.8	38.9	34.3	81.4	65.4	140	124
1	54.6	12.2	11.5	10.5	10.2	14.7	13.4	27.2	25.8
1.5	17.9	7.58	7.22	5.82	5.77	5.75	5.72	8.23	8.57
2	7.26	5.55	5.32	4.06	4.07	3.41	3.51	3.89	4.15
2.5	3.60	4.41	4.24	3.15	3.18	2.45	2.57	2.44	2.58
3	2.15	3.69	3.56	2.60	2.64	1.94	2.06	1.79	1.86
3.5	1.52	3.19	3.10	2.25	2.28	1.62	1.74	1.44	1.47
4	1.22	2.84	2.75	2.03	2.06	1.38	1.49	1.22	1.24
4.5	1.09	2.53	2.44	1.88	1.92	1.20	1.29	1.10	1.11
5	1.03	2.26	2.19	1.72	1.78	1.09	1.15	1.04	1.04
5.5	1.01	2.09	2.06	1.53	1.61	1.03	1.06	1.01	1.01
6	1.00	2.02	2.01	1.33	1.41	1.01	1.02	1.00	1.00

Bold values highlight the design values δ₀ and δ₁.

Table 4. ARL values for $\mathit{\text{AR}}{L}_0=500$ and ${\delta }_0=0.5.$

		${\delta }_1=1$		${\delta }_1=1.5$		${\delta }_1=2.5$		${\delta }_1=3.5$
δ	Shewhart	CUSUM	EWMA	CUSUM	EWMA	CUSUM	EWMA	CUSUM	EWMA
0	1E + 03	1E + 05	3E + 05	2E + 04	3E + 04	3E + 03	5E + 03	2E + 03	2E + 03
0.5	500	500	500	500	500	500	500	500	500
1	117	25.9	29.9	30.9	34.4	56.1	55.6	88.9	92.7
1.5	33.8	10.4	11.8	9.2	10.4	12.2	13.4	20.4	23.9
2	12.1	6.53	7.45	5.14	5.81	5.08	5.75	6.94	8.61
2.5	5.33	4.80	5.51	3.60	4.08	3.07	3.47	3.45	4.15
3	2.86	3.83	4.42	2.81	3.19	2.22	2.51	2.21	2.50
3.5	1.83	3.21	3.71	2.34	2.65	1.76	1.99	1.64	1.76
4	1.37	2.78	3.23	2.05	2.29	1.46	1.67	1.33	1.38
4.5	1.15	2.45	2.89	1.85	2.07	1.25	1.42	1.16	1.18
5	1.06	2.19	2.59	1.66	1.94	1.12	1.23	1.06	1.07
5.5	1.02	2.05	2.31	1.46	1.80	1.05	1.11	1.02	1.02
6	1.00	1.99	2.11	1.27	1.63	1.01	1.04	1.01	1.01

Bold values highlight the design values δ₀ and δ₁.

Table 5. ARL values for $\mathit{\text{AR}}{L}_0=500$ and ${\delta }_0=1.$

		${\delta }_1=1.5$		${\delta }_1=2$		${\delta }_1=3$		${\delta }_1=4$
δ	Shewhart	CUSUM	EWMA	CUSUM	EWMA	CUSUM	EWMA	CUSUM	EWMA
0	1E + 04	2E + 08	5E + 09	2E + 06	8E + 06	4E + 04	1E + 05	1E + 04	1E + 04
0.5	3E + 03	3E + 05	3E + 05	3E + 04	4E + 04	6E + 03	8E + 03	3E + 03	3E + 03
1	500	500	500	500	500	500	500	500	500
1.5	115	25.9	32.6	30.9	37.7	55.9	60.6	87.8	102
2	33.1	10.4	13.1	9.16	11.3	12.2	14.7	20.2	27.8
2.5	11.9	6.53	8.38	5.14	6.32	5.08	6.22	6.91	9.98
3	5.27	4.80	6.27	3.60	4.45	3.07	3.70	3.44	4.62
3.5	2.84	3.83	5.07	2.81	3.49	2.22	2.66	2.20	2.65
4	1.82	3.21	4.28	2.34	2.91	1.75	2.11	1.64	1.80
4.5	1.36	2.78	3.73	2.05	2.51	1.46	1.77	1.33	1.38
5	1.15	2.45	3.31	1.85	2.22	1.25	1.52	1.16	1.17
5.5	1.06	2.19	3.03	1.66	2.06	1.12	1.31	1.06	1.07
6	1.02	2.05	2.81	1.46	1.97	1.05	1.16	1.02	1.02

Bold values highlight the design values δ₀ and δ₁.

Table 6. ARL values for $\mathit{\text{AR}}{L}_0=500$ and ${\delta }_0=2.$

		${\delta }_1=2.5$		${\delta }_1=3$		${\delta }_1=4$		${\delta }_1=5$
δ	Shewhart	CUSUM	EWMA	CUSUM	EWMA	CUSUM	EWMA	CUSUM	EWMA
0	9E + 05	9E + 14	4E + 15	2E + 10	1E + 13	9E + 06	3E + 08	1E + 06	9E + 05
0.5	2E + 05	8E + 11	1E + 14	4E + 08	6E + 09	1E + 06	7E + 06	2E + 05	2E + 05
1	2E + 04	4E + 08	1E + 09	3E + 06	7E + 06	9E + 04	2E + 05	3E + 04	2E + 04
1.5	3E + 03	3E + 05	2E + 05	3E + 04	3E + 04	6E + 03	7E + 03	3E + 03	3E + 03
2	500	500	500	500	500	500	500	500	500
2.5	115	25.9	35.5	30.9	41.5	55.9	66.5	87.8	115
3	33.1	10.4	14.7	9.2	12.5	12.2	16.4	20.2	33.1
3.5	11.89	6.53	9.64	5.14	6.98	5.08	6.83	6.91	11.89
4	5.27	4.80	7.37	3.60	4.97	3.07	4.01	3.44	5.26
4.5	2.84	3.83	6.04	2.81	3.94	2.22	2.87	2.20	2.84
5	1.82	3.21	5.16	2.34	3.32	1.75	2.29	1.64	1.82
5.5	1.36	2.78	4.53	2.05	2.89	1.46	1.94	1.33	1.36
6	1.15	2.45	4.06	1.85	2.56	1.25	1.69	1.16	1.15

Bold values highlight the design values δ₀ and δ₁.

Specifically, in Table 3 (${\delta }_0=0$), we observe that the EWMA control chart yields better performance when ${\delta }_1=0.5.$ But for ${\delta }_1=1,$ its performance of detecting a larger shift size deteriorates as reflected by the out-of-control ARL scores to supersede the CUSUM counterparts. For ${\delta }_0>0,$ as exhibited by Tables 4, 5, and 6, the CUSUM control chart outperforms the EWMA chart uniformly, both for which they are designed to detect (the values in bold type) and the even larger (out-of-control) shift sizes. However, considering the in-control performance ($\delta <{\delta }_0$), the EWMA chart is favored because of the higher ARLs when the process is in control. The results when evaluating the same scenarios for $\mathit{\text{AR}}{L}_0=100$ using the settings of Table 1 are consistent with the ones presented in the tables for $\mathit{\text{AR}}{L}_0=500;$ therefore, these experiments are not included.

3.2. Steady-State performance

Concerning the latter, in Table 7, we evaluated the in-control ARLs of the corresponding charts in the steady state. By design, these ARLs should have a zero-state average run length near either $\mathit{\text{AR}}{L}_0=100$ or $\mathit{\text{AR}}{L}_0=500.$ Except in the cases of no acceptable region and the interest for small shifts (${\delta }_0=0$ and ${\delta }_1=0.5$ or 1) the differences are negligible. Furthermore, when either δ₀ or δ₁ increase, the steady-state and zero-state ARLs come closer together for both control charting schemes.

Table 7. Average steady-state run lengths of the in-control (zero-state) CUSUM and EWMA control charts, wherein the $\mathit{\text{AR}}{L}_0=100$ and $\mathit{\text{AR}}{L}_0=500$ are set for the δ₀ and are optimized for detection of a shift of size δ₁, see Tables 1 and 2).

Setting		$\mathit{\text{AR}}{L}_0=100$		$\mathit{\text{AR}}{L}_0=500$
δ0	δ1	CUSUM	EWMA	CUSUM	EWMA
	0.5	91.1	93.5	473	485
	1	96.4	97.0	490	494
0	2	99.1	99.1	498	498
	3	99.8	99.8	500	500
	4	100	100	500	500
	1	98.8	99.4	493	499
	1.5	99.3	99.5	497	499
0.5	2.5	99.8	99.8	499	500
	3.5	100	100	500	500
	4.5	100	100	500	500
	1.5	99.6	99.8	494	500
	2	99.8	99.8	498	500
1	3	100	99.9	500	500
	4	100	100	500	500
	5	100	100	500	500
	2.5	99.8	99.9	494	500
	3	99.9	99.9	498	500
2	4	100	100	500	500
	5	100	100	500	500
	6	100	100	500	500

3.3. Dependency on Phase I estimates

Another critical concern is related to our procedure of optimizing the CUSUM or EWMA control chart. In our optimization attempts, we find the parameters that result in a specific performance on a particular shift ($\mu ={\mu }_0+{\delta }_0{\sigma }_0$) while it is optimized to detect another, higher shift (${\mu }_0+{\delta }_1{\sigma }_0$); both are in terms of several standard deviations σ₀ of the mean ${\mu }_0.$ Thus, the framework presumes knowledge of μ₀ and σ₀, whereas in practice the true values are hardly available. There are different approaches possible to deal with this, where we will focus on two. One approach is to set pre-determined values of δ₀ and δ₁, and adjust the three regions based on the Phase I estimates. This is done in Section 3.3.1. Another approach is to use more practically informed limits where the three regions are fixed and where δ₀ and δ₁ are determined after Phase I estimation. This will be studied in Section 3.3.2.

3.3.1. Fixed design parameters (δ₀ and δ₁)

Naturally, in a Phase I study, μ₀ and σ₀ are estimated. So, for determined values δ₀ and δ₁ the uncertainty in the parameters affects the position of the indifference region and corresponding limits. As an illustration, consider without loss of generality a standard normal process (i.e., ${\mu }_0=0$ and ${\sigma }_0=1$) with ${\delta }_0=1$ and ${\delta }_1=3.$ In Figure 2, we illustrate these values in the case of known parameters on the left-hand side. In practice, we have to adapt our procedures to account for the process to be estimated at a different level, using ${\widehat{\mu }}_0,$ and to account for a different variation ${\widehat{\sigma }}_0.$ For example, while an actual mean shift of δ₀ standard deviations would be a Phase II mean of $\mu ={\delta }_0,$ our estimate of this shift size would be a mean of $\widehat{\mu }={\delta }_0-{\widehat{\mu }}_0.$ More importantly, it also entails that the performance for shifts of δ and $-\delta$ is no longer identical. Therefore, in the case of estimated parameters, we evaluate the control chart performance for the shift furthest away from our estimated mean.

Figure 2. Dependency of the framework on Phase I estimates, used in the computation of the AARL and SDARL metrics; in this example ${\delta }_0=1$ and ${\delta }_1=3$ (${\mu }_0=0$ and ${\sigma }_0=1$).

For implementation in R (with random seed 1), we evaluate the performance using estimated parameters by generating samples of sizes m from a $\mathcal{N}(0,1)$ distribution. We use the sample average ($\overline{X}$) and standard deviation (s) to estimate ${\widehat{\mu }}_0=\overline{X}$ and ${\widehat{\sigma }}_0=s/c_4(m).$ Note that the constant $c_4(m)$ is often used to obtain an unbiased estimate of σ₀. Next, note that the distance between the original indifference lines (δ₀ and $-{\delta }_0$) to the center line of the estimated control charts is equal to $|{\widehat{\mu }}_0-{\delta }_0|$ and $|{\widehat{\mu }}_0+{\delta }_0|,$ which are not identical (unless ${\widehat{\mu }}_0$ equals 0). The maximum distance is thus equal to $\left|{\widehat{\mu }}_0\right|+{\delta }_0.$ Since part of the objective is to constitute a specified ARL (ARL₀) for a shift of size δ₀ on either side, we choose to use the maximum distance as shift size in our evaluation. The ARL on the other side (smallest distance) will be larger, so the considered ARL values can be viewed as the minimum ARL for shifts δ₀ or $-{\delta }_0$ when parameters are estimated.

In addition, the estimation of σ₀ needs to be incorporated; remark the larger bandwidth (${\widehat{\sigma }}_0>{\sigma }_0$) on the right side of Figure 2. For the three charts under consideration, the following adjustments are made (cf. Section 2, where ${\sigma }_0\equiv 1$):

• ${\widehat{L}}_S=L_S{\widehat{\sigma }}_0$ for the Shewhart chart;

• ${\widehat{L}}_C=L_C{\widehat{\sigma }}_0,$ and $\widehat{k}=k{\widehat{\sigma }}_0=\left(\frac{{\delta }_0+{\delta }_1}{2}\right){\widehat{\sigma }}_0$ for the CUSUM chart;

• and ${\widehat{L}}_E=L_E{\widehat{\sigma }}_0$ for the EWMA chart.

We evaluate the ARL performance conditional on Phase I estimates using these adjusted parameters. For several Phase I sample sizes m, we generate 10,000 simulated samples, which are used to obtain 10,000 conditional ARL values. In addition, the scenarios (in terms of δ₀ and δ₁) are varied. The results are summarized in Table 8, which reports the average conditional ARL (AARL) and the standard deviation of the conditional ARLs (SDARL) for each setting.

Table 8. The average (AARL) and standard deviation (SDARL) over the in-control ARLs when applying Shewhart, CUSUM, and EWMA control charts over different Phase I estimates, where ${\delta }_1={\delta }_0+1$ and $\mathit{\text{AR}}{L}_0=100$ are fixed, but δ₀ and the Phase I sample sizes m vary.

		Shewhart		CUSUM		EWMA
Sample size	δ 0	AARL	SDARL	AARL	SDARL	AARL	SDARL
	0	113	69.5	101	53.9	94.6	42.4
	0.5	98.5	64.4	86.5	61.5	84.5	55.0
m = 100	1	103	82.2	99.6	106	97.3	92.1
	1.5	112	110	121	202	115	157
	2	123	150	158	429	140	283
	0	105	37.1	99.7	31.0	96.9	25.6
	0.5	94.8	35.7	86.5	34.8	85.6	32.2
m = 250	1	96.1	42.4	91.0	48.7	90.8	44.9
	1.5	99.0	51.3	97.3	68.1	96.5	60.5
	2	102	61.7	106	96.4	103	80.9
	0	102	24.8	99.7	21.2	98.2	17.8
	0.5	94.9	24.5	88.6	24.4	88.1	22.9
m = 500	1	95.3	28.5	90.8	32.1	90.9	30.0
	1.5	96.6	33.4	93.7	41.3	93.8	37.8
	2	98.2	38.7	97.4	52.5	97.0	46.7
	0	101	17.1	99.8	14.8	99.0	12.5
	0.5	95.6	17.3	91.0	17.5	90.6	16.5
m = 1000	1	95.7	19.9	92.0	22.4	92.3	21.1
	1.5	96.3	23.0	93.5	27.8	93.8	25.8
	2	97.1	26.2	95.3	33.9	95.5	30.8

In Table 8, we vary the sample size m and study the dependency of our framework on Phase I estimates by reporting the AARL and SDARL values. Indeed, when m is low, we find that the ARLs are more volatile, as SDARLs are larger and AARLs vary more for different values of δ₀. As expected, when m increases, the values become more accurate, and the SDARL decreases. Moreover, the EWMA chart seems to mitigate the Phase I uncertainty better than the CUSUM control chart, as observed by the lower SDARLs values. A possible explanation for the worse CUSUM chart performance is that the uncertainty around σ₀ is directly affecting both the control limits (via ${\widehat{L}}_C$) and the charting statistic (via $\widehat{k}$), whereas for the EWMA and Shewhart control charts it only affects the control limit—${\widehat{L}}_E$ and ${\widehat{L}}_S$, respectively—but not the charting statistic. Zooming in on their performances, we find that the EWMA chart is more robust when δ₀ is small, i.e., below 1.

Still, we observe that for higher δ₀ values, i.e., the values at which we fix the ARL₀ performance (here to 100), the problem is more persistent. So, using these methods only in cases with sufficient data is advisable to ensure sufficiently accurate estimates of μ₀ and σ₀, say around m = 500 observations when ${\delta }_0=2.$ This might feel restrictive, but considering the use-case of an (optimized) indifference region, one likely has extensive track records when considering such an approach.

As an additional analysis of the effects of parameter estimation, we evaluate the out-of-control AARL and SDARL for one specific set of parameters ($\mathit{\text{AR}}{L}_0=100,$ ${\delta }_0=1,$ and ${\delta }_1=2$). These results are shown in Table 9, where we also included the respective in-control (δ = 1, cf. Table 8) AARL and SDARL as well as the case of known parameters $m=\infty .$ We observe that the differences in AARL for varying sample sizes are small, as all AARL values are comparable to the case of $m=\infty .$ The main difference is in the SDARL values. As anticipated, the SDARL decreases when the sample or shift size increases.

Table 9. The average (AARL) and standard deviation (SDARL) over the ARLs when applying Shewhart, CUSUM, and EWMA control charts over different Phase I estimates for different shift sizes δ, where ${\delta }_0=1,{\delta }_1=2,$ and $\mathit{\text{AR}}{L}_0=100$ are fixed, but the Phase I sample sizes m vary. The case $m=\infty$ represents the case of known parameters.

		Shewhart		CUSUM		EWMA
Sample size	δ	AARL	SDARL	AARL	SDARL	AARL	SDARL
	1	103	82.2	99.6	106	97.3	92.1
	1.5	29.5	18.7	14.8	6.87	17.0	8.00
m = 100	2	10.6	5.21	5.70	1.30	6.68	1.67
	2.5	4.76	1.75	3.43	0.51	3.99	0.63
	3	2.61	0.68	2.49	0.29	2.89	0.34
	1	96.1	42.4	91.0	48.7	90.8	44.9
	1.5	28.2	10.2	14.8	3.96	16.8	4.59
m = 250	2	10.4	2.98	5.78	0.82	6.75	1.03
	2.5	4.70	1.03	3.47	0.33	4.03	0.40
	3	2.60	0.41	2.51	0.18	2.91	0.21
	1	95.3	28.5	90.8	32.1	90.9	30.0
	1.5	28.2	6.82	15.0	2.71	17.1	3.13
m = 500	2	10.4	2.01	5.86	0.57	6.84	0.72
	2.5	4.73	0.70	3.51	0.23	4.07	0.28
	3	2.62	0.28	2.53	0.13	2.93	0.15
	1	95.7	19.9	92.0	22.4	92.3	21.1
	1.5	28.3	4.84	15.2	1.96	17.3	2.26
m = 1000	2	10.5	1.43	5.92	0.41	6.90	0.52
	2.5	4.76	0.50	3.53	0.17	4.09	0.20
	3	2.63	0.20	2.54	0.09	2.95	0.11
	1	100	–	100	–	100	–
	1.5	29.5	–	16.1	–	18.2	–
$m=\infty$	2	10.8	–	6.11	–	7.11	–
	2.5	4.90	–	3.60	–	4.17	–
	3	2.69	–	2.58	–	2.98	–

3.3.2. Fixed indifference regions

So far, we studied the effect of Phase I estimates by adjusting the indifference regions using these estimates. It demonstrates the impact of estimation error when we have pre-determined values for δ₀ and δ₁. This helps to understand how Phase I uncertainty impacts the performance and therefore helps the reader decide their δ₀ and δ₁ when parameters are estimated.

Note that in practice, it might be wise for a practitioner to compensate a larger estimate of σ₀ by choosing a smaller value of δ₀. An alternative angle is thus to keep a fixed indifference region, where the choice of δ₀ and δ₁ are dependent on the Phase I estimates. This approach is outlined in Figure 3, where taking Phase I uncertainty into consideration leads to estimating ${\widehat{\delta }}_0$ and ${\widehat{\delta }}_1$ to ensure the same indifference region. So, after measuring ${\widehat{\mu }}_0$ and ${\widehat{\sigma }}_0$ we establish ${\widehat{\delta }}_0=\frac{{\delta }_0{\sigma }_0}{{\widehat{\sigma }}_0}$ and ${\widehat{\delta }}_1=\frac{{\delta }_1{\sigma }_0}{{\widehat{\sigma }}_0}.$ In the example displayed in Figure 3, the upper limit of the indifference region is equal to ${\mu }_0+3,$ which results in ${\widehat{\delta }}_1=3/{\widehat{\sigma }}_0.$ Similarly, we would find ${\widehat{\delta }}_0=1/{\widehat{\sigma }}_0.$

Figure 3. Dependency of the framework on Phase I estimates when having a fixed indifference region. It is used in the computation of the AARL and SDARL metrics; in this example ${\delta }_0=1$ and ${\delta }_1=3$ (${\mu }_0=0$ and ${\sigma }_0=1$).

Next, in a similar vein as in Section 3.3.1, we compute the AARL and SDARL using this approach. The corresponding tables, Table 10 for in-control and Table 11 for out-of-control performances, show similar patterns as the study that has been carried out in the previous section. In fact, we find that keeping the indifference regions fixed, by choosing suitable ${\widehat{\delta }}_0$ and ${\widehat{\delta }}_1,$ counteracts the Phase I uncertainty to some extent: a high/low ${\widehat{\sigma }}_0$ will decrease/increase ${\widehat{\delta }}_0$ and ${\widehat{\delta }}_1,$ yielding lower SDARL values in both tables. The out-of-control performance also improves, as can be observed by comparing Table 11 to Table 9. The explanation is that by fixing the indifference region, you already incorporate some prior knowledge about the process in the determination of the control limits.

Table 10. The average (AARL) and standard deviation (SDARL) over the in-control ARLs when applying Shewhart, CUSUM, and EWMA control charts over different Phase I estimates, where ${\delta }_1={\delta }_0+1$ and $\mathit{\text{AR}}{L}_0=100$ are fixed, but δ₀ and the Phase I sample sizes m vary.

		Shewhart		CUSUM		EWMA
Sample size	δ 0	AARL	SDARL	AARL	SDARL	AARL	SDARL
	0	114	72.1	99.1	52.3	92.9	41.3
	0.5	93.8	52.0	76.9	36.2	75.9	32.9
m = 100	1	91.6	48.4	76.7	36.0	77.9	34.5
	1.5	91.5	48.2	76.7	36.0	79.1	35.5
	2	91.5	48.2	76.7	36.0	79.8	36.1
	0	105	37.4	98.6	30.1	95.8	24.9
	0.5	92.5	29.1	82.2	23.3	81.7	21.5
m = 250	1	91.2	27.6	82.1	23.2	83.0	22.1
	1.5	91.2	27.5	82.1	23.2	83.7	22.5
	2	91.2	27.5	82.1	23.2	84.2	22.7
	0	102	25.7	99.0	21.8	97.5	18.3
	0.5	93.5	21.4	86.3	18.4	85.9	17.1
m = 500	1	92.6	20.5	86.2	18.3	86.9	17.4
	1.5	92.6	20.4	86.2	18.3	87.4	17.6
	2	92.6	20.4	86.2	18.3	87.8	17.7
	0	101	17.8	99.3	15.3	98.6	12.8
	0.5	94.8	15.1	89.6	13.4	89.4	12.4
m = 1000	1	94.2	14.4	89.5	13.3	90.1	12.6
	1.5	94.2	14.4	89.5	13.3	90.4	12.7
	2	94.2	14.4	89.5	13.3	90.7	12.8

Table 11. The average (AARL) and standard deviation (SDARL) over the ARLs when applying Shewhart, CUSUM, and EWMA control charts over different Phase I estimates for different shift sizes δ, where ${\delta }_0=1,{\delta }_1=2,$ and $\mathit{\text{AR}}{L}_0=100$ are fixed, but the Phase I sample sizes m vary. The case $m=\infty$ represents the case of known parameters.

		Shewhart		CUSUM		EWMA
Sample size	δ	AARL	SDARL	AARL	SDARL	AARL	SDARL
	1	91.6	48.4	76.7	36.0	77.9	34.5
	1.5	27.0	11.6	13.4	2.92	15.4	3.55
m = 100	2	10.0	3.38	5.52	0.75	6.44	0.91
	2.5	4.57	1.17	3.38	0.37	3.92	0.42
	3	2.54	0.47	2.47	0.25	2.86	0.27
	1	91.2	27.6	82.1	23.2	83.0	22.1
	1.5	27.2	6.8	14.2	1.97	16.2	2.34
m = 250	2	10.1	2.02	5.71	0.49	6.65	0.59
	2.5	4.63	0.71	3.45	0.24	4.01	0.27
	3	2.58	0.29	2.51	0.16	2.90	0.17
	1	92.6	20.5	86.2	18.3	86.9	17.4
	1.5	27.6	5.05	14.7	1.55	16.7	1.82
m = 500	2	10.2	1.50	5.81	0.37	6.77	0.45
	2.5	4.68	0.53	3.49	0.18	4.05	0.20
	3	2.60	0.21	2.53	0.12	2.92	0.12
	1	94.2	14.4	89.5	13.3	90.1	12.6
	1.5	28.0	3.56	15.1	1.11	17.1	1.31
m = 1000	2	10.4	1.06	5.90	0.26	6.87	0.32
	2.5	4.74	0.37	3.52	0.12	4.08	0.14
	3	2.62	0.15	2.54	0.08	2.94	0.09
	1	100	–	100	–	100	–
	1.5	29.5	–	16.1	–	18.2	–
$m=\infty$	2	10.8	–	6.11	–	7.11	–
	2.5	4.90	–	3.60	–	4.17	–
	3	2.69	–	2.58	–	2.98	–

3.4. Impact of Non-Normality

In addition to parameter estimation, many other aspects could be taken into account, such as the performance of the optimized charts under different data distributions. The EWMA control chart is known to be more robust to non-normality than the Shewhart control chart; see, for example, Borror, Montgomery, and Runger (1999). To assess the impact of non-normality on optimized control charts, we expand our experiments to cover some settings with skewed and heavy-tailed data distributions. The package SPC does not cover non-normal settings, so we resort to the well-established Markov chain approach. This approach is introduced to assess the performance of the CUSUM control chart in Brook and Evans (1972), and, for example, used in Borror, Montgomery, and Runger (1999) to assess the robustness of the EWMA chart.

The non-normal distributions considered are the Gamma distribution and Student’s t-distribution. It is known that for a Gamma distribution with parameters α (scale) and β (rate) the skewness is given by $\frac{2}{\sqrt{\alpha }},$ while for the symmetric t-distribution the excess kurtosis equals $\frac{6}{4-\nu }$ with ν the degrees of freedom. For comparison purposes, we added the normal distribution to the experiments in Table 12 as the middle case. As a result, moving upward from the normal distribution, the distribution becomes heavier tailed, while in the downward direction, it becomes more skewed. As a final note, the parameters for the Shewhart, CUSUM, and EWMA charts at each combination of δ₀ and δ₁ follow from Table 2 ($\mathit{\text{AR}}{L}_0=500$).

Table 12. The ARL values for different data distributions for $\mathit{\text{AR}}{L}_0=500$ and various combinations of δ₀ and δ₁.

		${\delta }_0=0$ & ${\delta }_1=2$			${\delta }_0=1$ & ${\delta }_1=3$			${\delta }_0=2$ & ${\delta }_1=4$
δ	Distribution	Shewhart	CUSUM	EWMA	Shewhart	CUSUM	EWMA	Shewhart	CUSUM	EWMA
	t (5)	95.8	139	165	245	378	666	673	995	2.22E + 03
	t (20)	253	332	359	1.75E + 03	4.59E + 03	1.27E + 04	2.01E + 04	5.76E + 04	4.23E + 05
	t (40)	342	407	423	3.53E + 03	1.17E + 04	3.51E + 04	8.57E + 04	3.65E + 05	4.76E + 06
0	Normal	500	500	500	9.49E + 03	4.40E + 04	1.31E + 05	9.33E + 05	9.48E + 06	2.55E + 08
	Gam(4.1)	110	153	203	362	639	1.48E + 03	1.73E + 03	3.44E + 03	1.54E + 04
	Gam(2.1)	79.2	109	148	210	333	692	740	1.27E + 03	4.34E + 03
	Gam(0.5.1)	48.8	65.3	87.7	91.9	124	210	202	280	621
	t (5)	43.4	16.1	14.9	131	174	177	424	647	1.01E + 03
	t (20)	50.2	14.9	13.6	303	361	353	3.46E + 03	9.10E + 03	1.72E + 04
	t (40)	52.2	14.8	13.5	380	424	416	7.04E + 03	2.33E + 04	4.54E + 04
1	Normal	54.6	14.7	13.4	500	500	500	1.90E + 04	8.79E + 04	1.65E + 05
	Gam(4.1)	26.7	15.4	13.3	81.0	102	107	362	639	1.11E + 03
	Gam(2.1)	23.8	16.0	13.5	61.2	77.4	81.6	210	333	544
	Gam(0.5.1)	21.4	18.7	14.8	41.1	51.3	53.9	91.9	124	178
	t (5)	9.11	3.39	3.52	33.1	13.6	17.0	142	187	173
	t (20)	7.57	3.41	3.51	32.4	12.4	15.0	305	363	342
	t (40)	7.40	3.41	3.51	32.7	12.3	14.8	381	424	408
2	Normal	7.25	3.41	3.51	33.1	12.2	14.7	500	500	500
	Gam(4.1)	7.36	3.57	3.68	20.0	12.9	13.6	81.0	102	101
	Gam(2.1)	7.60	3.62	3.76	18.6	13.4	13.6	61.2	77.4	77.0
	Gam(0.5.1)	9.02	3.66	4.03	17.9	15.9	14.6	41.1	51.3	51.1
	t (5)	2.19	1.95	2.07	6.48	3.05	3.76	33.4	13.6	19.0
	t (20)	2.16	1.94	2.06	5.47	3.07	3.71	32.4	12.4	16.8
	t (40)	2.16	1.94	2.06	5.36	3.07	3.71	32.7	12.3	16.6
3	Normal	2.15	1.94	2.06	5.26	3.07	3.70	33.1	12.2	16.4
	Gam(4.1)	2.51	1.96	2.07	5.74	3.22	3.96	20.0	12.9	14.6
	Gam(2.1)	2.68	1.96	2.06	6.03	3.28	4.10	18.6	13.4	14.4
	Gam(0.5.1)	3.47	1.89	2.03	7.44	3.31	4.67	17.9	15.9	15.3
	t (5)	1.17	1.35	1.49	1.79	1.76	2.11	6.48	3.05	4.12
	t (20)	1.21	1.37	1.49	1.82	1.75	2.11	5.47	3.07	4.03
	t (40)	1.22	1.38	1.49	1.82	1.75	2.11	5.36	3.07	4.02
4	Normal	1.22	1.38	1.49	1.82	1.75	2.11	5.26	3.07	4.01
	Gam(4.1)	1.21	1.43	1.54	2.07	1.78	2.13	5.74	3.22	4.32
	Gam(2.1)	1.19	1.45	1.57	2.20	1.79	2.13	6.03	3.28	4.48
	Gam(0.5.1)	1.00	1.53	1.66	2.75	1.77	2.10	7.44	3.31	5.17

Bold values highlight the design value δ₀.

Studying Table 12 in more detail, we find that the performance of the optimized control charts degrades quickly when deviating from normality, with the Shewhart chart showing the worst scores, in line with Borror, Montgomery, and Runger (1999) for non-optimized EWMA charts. Here, for ${\delta }_0=1$ or 2, and ${\delta }_1={\delta }_0+2$—two boldfaced blocks—the EWMA and CUSUM charts show comparable ARLs under non-normality. Between these two charts, the EWMA chart receives fewer signals in the in-control region (above the boldfaced blocks). In contrast, in most cases, the CUSUM chart has faster detection in the out-of-control region (below the boldfaced blocks).

4. Discussion

The motivation of the three-region approach is to focus on detecting practically significant shifts, not small shifts caused by minor (perhaps temporary) common causes. Such situations frequently occur in practice, see Freund (1957), and have recently received renewed attention by Woodall and Faltin (2019). Answering their call, we implement this approach to the EWMA and CUSUM control charting schemes by allowing a slight common cause variation in the in-control parameter via δ₀. The evasion of too many signals has also resonated in the literature on the economic design of control charts, e.g., Rahlm (1985) and Lorenzen and Vance (1986). Still, we stress that the approach outlined here is fundamentally different as the extra margin chosen by the user is optimized and thus does not follow economic considerations of the cost of reacting to signals. Besides, these costs are often hard to estimate, and the approach has several flaws that may ultimately lead to poor control chart designs, see Woodall, Lorenzen, and Vance (1986).

As opposed to the Shewhart control chart with a single parameter, the EWMA and CUSUM charts have additional flexibility to optimize toward the detection of particular shifts of size δ₁, allowing the optimization of these control charting schemes. The performance comparison reveals that the CUSUM control chart appears best suited to this task, except when δ₀ and δ₁ are both small. However, the EWMA control chart is slightly more robust. Besides performance and robustness, some other considerations play a role when selecting the right control charting scheme, which we discuss below.

4.1. Optimization issues

Obtaining the optimal parameter values for the CUSUM control chart is straightforward, as the optimal value of k is known to be $k=({\delta }_0+{\delta }_1)/2.$ The other parameter, L_C, can then be chosen to match a specified ARL value. For the EWMA chart, optimization always requires an evaluation of both its parameters simultaneously. Moreover, with this optimization, some scenarios generate λ values outside the desired region of $\lambda \in [0.05,0.25]$ as prescribed by Montgomery (2020). While it is, of course, possible to restrict the optimal parameter search to this interval, it comes at the cost that it will deteriorate the theoretical performance of the EWMA control chart. In addition, trying to distill patterns in the optimal parameters as a function of δ₀ and δ₁ (boundaries of the indifference regions), we find that for the CUSUM chart, they behave monotonically, while for the EWMA chart that is not the case as its parameters greatly interact.

Another computational issue with the EWMA may come forward when δ₀ is large (e.g., 2 or larger) and when δ₁ is close to δ₀ (e.g., ${\delta }_0+0.5).$ While the optimal parameters for the CUSUM charts are still easily obtained in these situations, the optimization of EWMA may run into issues, as the determination of parameters could become too difficult to use standard settings of Knoth (2014), i.e., for computation the number of quadrature nodes used in the approximation should be increased.

4.2. Small indifference regions

The issue in optimizing the EWMA control chart that creates computational issues is a too-small indifference region combined with a relatively high ${\delta }_0;$ the difference between δ₀ and δ₁ is only 0.5, while ${\delta }_0=2.$ In such a case, it comes down to two conflicting goals; at δ₀, we aim for a specific ARL₀ performance, while at δ₁, we optimize to detect such a shift quickly. So, as these goals are fundamentally different, choosing δ₀ and δ₁ close, i.e., having a tiny indifference region (especially for the EWMA chart), may lead to problems in this framework.

A strategy to avoid a too-small indifference region is to reconsider the settings of δ₀, δ₁, and ARL₀ from a practical point of view. The purpose of an indifference region is to have a range of values in which the performance is not so critical. Thus, a straightforward approach to alleviating potential computational issues of the EWMA chart would be to make the difference between δ₀ and δ₁ (much) larger. For example, choose a smaller value of δ₀, and set ARL₀ to a larger value. However, if the ARL₀ is set higher at a lower δ₀, one should collect more data to offset the possible estimation error, as seen in Section 3.3, where we study the impact of Phase I estimates in our approach. Another practical resolution would be to choose a larger value of δ₁.

4.3. Interpretability of the charts

Interpretability is another relevant aspect. Between the CUSUM and the EWMA control charts, the EWMA chart has the advantage that the charting statistic (although it is a weighted average) remains in the original unit of measure. The CUSUM chart does not have this property as it tracks a positive and negative cumulative sum, which only sums positive or negative differences from a reference value. Moreover, because no individual values are plotted in the CUSUM and EWMA charts, it cannot be observed directly from these charts whether an individual value is out-of-specification, which touches upon the difference between statistical and practical significance, see Woodall and Faltin (2019); Blume et al. (2019) for recent discussions. The Shewhart control chart does not have this discrepancy. However, due to their (much) better detection of persistent shifts, the modified CUSUM and EWMA charts are still recommended over the Shewhart control chart. In the next section, we apply the proposed methodology to some examples to highlight its value in practice.

Both charts are still amenable to various alternatives, which have not been studied to ensure a “fair” and comprehensible comparison. First, one could consider EWMA and CUSUM charts with different limits. Whereas we have taken fixed limits, one could opt for time-varying limits and other variations, see Knoth (2003) and Crosier (1986), which are part of Knoth’s R package spc. Second, in the spirit of the CUSUM chart to have two monitoring statistics, one could opt for two one-sided EWMA charts, see Gan (1993).

5. Practical examples

In this section, we consider two examples. The first example is inspired by the paper of Woodall and Faltin (2019), which exhibits a more extreme use case of implementing an indifference region. The second example involves wafers from a hard-bake process, which can be found in Montgomery (2020). The statistical software Minitab is used to execute the experiments and to generate the figures. The figures for the EWMA chart are manually rendered in Minitab to be able to implement steady-state limits, which is not an option in Minitab.

5.1. Case 1: Critical quality attribute (CQA)

Our first example extends to the case study reported in Woodall and Faltin (2019). Inspired by that example, we synthesized data with the same characteristics in Phase I, e.g., mean 6 and standard deviation 0.06. So, we simulated 200 data points from a normal distribution with these characteristics; we added some minor seasonal variation with an amplitude of one standard deviation to mimic insignificant and temporary shifts. The resulting values are given in the control chart of Figure 4. Note that the seasonal effect is only considered present in Phase II.

Figure 4. Shewhart control chart applied to CQA (with control limits at $3{\sigma }_0$).

In this case study, the specification limits are much wider than the control limits of the process, i.e., 4 and 8 versus 5.82 and 6.18, as observed in Figure 4. Therefore, it is sensible not to react to some small variation, e.g., small and expected seasonality, which can be considered common noise instead of special causes of variation. Thus, to modify the control charting schemes, we have to allow the process to return to “normal”. Note that the absence of seasonality in a Phase I study and the presence of it in Phase II might occur when the Phase I study was carried out during a limited part of a seasonal cycle. Or, for example, when the moving range (MR) estimator is used to estimate the process dispersion.

Because there is a great discrepancy between the control and specification limits, we adapt the EWMA and CUSUM control charts using the proposed approach. Taking some extreme values from Table 2, i.e., $\mathit{\text{AR}}{L}_0=500$ at ${\delta }_0=1$ (as the amplitude of the seasonal variation is one standard deviation) and ${\delta }_1=3,$ we adjust the original time-weighted charts. Both charts are given in Figure 5. When comparing the standard to the optimized versions, we observe that the original charts easily detect seasonal effects and produce many signals. Note that the ARL₀ at ${\delta }_0=0$ is 168 for the standard CUSUM control chart (k = 0.5; L_C = 4) and 560 for the standard EWMA control chart ($\lambda =0.2;$ L_E = 3); by default, the CUSUM chart is thus set more sensitive than the EWMA chart. Applying our procedure to accommodate the seasonal variation results in only one or two signals. Naturally, setting δ₀ to an even higher setting will fully mitigate the signals produced because of the seasonality. Yet, true shifts go unnoticed for a longer period, which will be part of our experiment in Section 5.2.

Figure 5. Standard and optimized ($\mathit{\text{AR}}{L}_0=500$ at ${\delta }_0=1,$ and ${\delta }_1=3$) CUSUM and EWMA control charts applied to CQA.

In a second adaptation of CQA, we excluded the seasonality but added two small temporary shifts and one larger persistent shift. Specifically, to our 200 data points, we added a shift of one standard deviation from observation 21 until 40, and we subtracted one standard deviation from observation 101 until 120. The persistent shift (3 standard deviations) occurs at observation 161. In line with the previous example, we use the parameters of Table 2 with ${\delta }_0=1$ and ${\delta }_1=3,$ such that our modified charts are designed to ignore the temporary variation while detecting the true persistent shift as early as possible. Indeed, in Figure 6 the standard Shewhart control chart detects extra signals, whereas the adjusted Shewhart with the limits raised to be at 3.878 standard deviations (σ₀) of μ₀ still finds one signal related to the temporary shift. As seen in the left panels of Figure 7, the standard CUSUM and EWMA charts easily detect the temporary shifts that occur far before the true persistent shift, which is, of course, detected by both. The panels on the right-hand side show that the adjusted CUSUM and EWMA charts can fully disregard these temporary shifts and immediately detect the true persistent shift at 161. This example underpins the promise of modifying the CUSUM and EWMA charts to make them less sensitive to small “unimportant” shifts.

Figure 6. Shewhart control chart applied to CQA (standard and optimized, ${\delta }_0=1$).

Figure 7. Standard and optimized ($\mathit{\text{AR}}{L}_0=500$ at ${\delta }_0=1,$ and ${\delta }_1=3$) CUSUM and EWMA control charts applied to CQA.

5.2. Case 2: Wafers from a hard-bake process

Where our first example was specifically designed to illustrate the purpose of the proposed method, our next example relates to Chapter 6 of Montgomery (2020) about wafers from a hard-bake process. In the original example, subgroups of size 5 are selected; we converted these subgroups to individual values, for which the historical mean is ${\mu }_0=1.5056$ and standard deviation ${\sigma }_0=0.0625.$ Moreover, as the USL and LSL for this process are set at 1.0 and 2.0, so one could question whether this process should be monitored so closely. Therefore, we implement the standard Shewhart control chart and the adapted one based on the corresponding parameter from Table 1.

The application of a standard Shewhart control chart (limits at 3 standard deviations) signals at observation 43, whereas the adapted chart signals at observation 45, as seen in Figure 8. Next, we use our framework of indifference regions where in addition to requiring the ARL₀ to be 500 at δ₀, we optimize for detecting specific shifts at δ₁ equal to 2 or 3, which is done in Figure 9. Note that in this scenario, the shift seems persistent, and thus should be deemed a true shift, which is also concluded in Montgomery (2020). Fortunately, our optimized versions are still capable of detection, but at a later stage. So, this example illustrates the tradeoff of employing an indifference region—to disregard unimportant shifts—and swiftness of detection in case of a true shift (not in the in-control region). The larger the indifference region, the later a true shift will be detected.

Figure 8. Shewhart control charts, standard and adapted, applied to the Flow Width data with parameters based on ${\mu }_0=1.5056$ and ${\sigma }_0=\mathrm{0.0625.}$

Figure 9. An application of optimized control charts using indifference regions. In all altered schemes ${\delta }_0=1$ where $\mathit{\text{AR}}{L}_0=500,$ while in (c) & (d) are optimized to detect a shift at ${\delta }_1=2$ and in (e) & (f) at ${\delta }_1=3;$ see also Table 2.

An interesting comparison is between Figures 9(c) and 9(d), where the EWMA chart only signals at 45, while the CUSUM chart still signals at 43. Finally, if ${\delta }_1=3$ is set to 2, both charts signal only at observation 45. Interestingly, the EWMA bounds decrease as the λ parameter has increased when comparing Figures 9(c) and 9(e).

6. Concluding remarks

In process reliability, this work leverages optimization to advance control chart schemes to better fit practice. Answering the call of Woodall and Faltin (2019) and aligning with the international standard, ISO (2020), we provide an approach accessible to the average practitioner approach that wants to utilize the leeway between the control limits (the range of the process fluctuations) and specification limits (the range in which products are acceptable). Often there is quite a gap between these different types of limits, which allow the sensitive time-series charts to return to normal; one can argue that for practice, many control charting schemes are overly sensitive and that somehow larger deviations from the target can be accounted for as normal process variation, i.e., common noise. The reduction of signals is also resonated in the literature on the economic design of control charts, but with a different motivation (Rahlm 1985; Lorenzen and Vance 1986).

To do so, we employ the framework of an indifference region to disregard signals from small shifts or minor seasonal effects. The CUSUM and EWMA control charts are adapted to meet a specified average run length ARL₀ at a mean shift of ${\delta }_0>0$ standard deviations. At the same time, they are optimized to detect a shift of ${\delta }_1(>{\delta }_0)$ standard deviations. In this way, the indifference region automatically demarcates two other regions; within ${\mu }_0\pm {\delta }_0,$ we find the in-control region and further outward, beyond ${\mu }_0\pm {\delta }_1,$ the out-of-control region.

Comparing the optimized EWMA and CUSUM charts to the standard Shewhart control chart and their unaltered counterparts, we show that they are particularly promising for practice when there is sufficient leeway compared to the specification limits of a process. Out of these two, we conclude that the CUSUM chart is the better choice as it generally outperforms the EWMA chart slightly and has some advantageous computational properties, i.e., one of its parameters can be immediately set, and thus only one parameter has to be determined. Although an EWMA chart is easier to interpret as it has the same unit of measure for its statistic as the original data, the EWMA parameter combinations can become tedious, especially when the indifference region is small. Finally, we find that in many instances, EWMA’s weight parameter λ lands outside the advised and common range reported in standard statistical literature (Montgomery 2020).

The optimization procedures and comparison of the Shewhart, optimized EWMA, and CUSUM charts are primarily focused on dealing with normally distributed data. When data do not come from a normal distribution, the EWMA and CUSUM charts outperform the Shewhart chart, as expected. However, the performances of these charts might be improved by using data transformations (Chou, Polansky, and Mason 1998). Finally, since the optimization procedures intrinsically rely on design choices and Phase I estimates, the robustness of the approach is assessed as well. Checking the sensitivity of the framework on Phase I estimates reveals that sufficient data should be available to have a reliable optimized EWMA or CUSUM chart—considering the use case of an indifference region, a condition that is easily met.

Acknowledgements

The authors would like to thank the editor and the referees for their helpful suggestions, leading to a more precise and comprehensive presentation of our work.

Additional information

Notes on contributors

Alex Kuiper

Alex Kuiper is an Associate Professor in the Department of Business Analytics at the Amsterdam Business School of the University of Amsterdam and a senior consultant at the Institute for Business and Industrial Statistics of the University of Amsterdam, the Netherlands. In 2013, he received a double MSc in Stochastics & Financial Mathematics and Econometrics from the University of Amsterdam. He completed his Ph.D. in Operations Research at the same university in 2016. His current research includes various topics, such as operations improvement, logistics, and healthcare optimization.

Rob Goedhart

Rob Goedhart is an Assistant Professor in the Department of Business Analytics at the Amsterdam Business School of the University of Amsterdam and a senior consultant at the Institute for Business and Industrial Statistics of the University of Amsterdam, the Netherlands. He obtained his MSc in Econometrics at the University of Amsterdam in 2014 and his Ph.D. in Statistics at the same university in 2018. His current research revolves around estimation techniques in statistical and predictive process monitoring as well as explainable artificial intelligence (XAI).