Section 9.5: Concluding Remarks
175
will ensure a test series with inconsequential serial correlation in terms of the
statistic to be tested.
The formula for To depends on the tested statistic (Thiebaux and Zwiers,
1984; Trenberth, 1984). For tests of the mean,
N
ß
To = 1 + 2 L(1- N)PtJ.
tJ.=l
(9.1)
where N :::; n - 1 is the maximum of lags /). of the estimated autocorrelation
function Pt:.. For tests of the standard deviation, PtJ. is replaced by p~ in (9.1),
and for tests of the correlation of series 1 and 2, PtJ. is replaced by PA;l . PA;2'
The autocorrelation function, PA, in all cases must be estimated and this
is often difficuIt to do weIl. Generally, the maximum lag N must be set weIl
below n so that the sam pie size for all N lag estimates is reasonably large
(> 30 or so). Then all PtJ. can be estimated with relatively small estimation
error. If not, then a time series model should be fitted to the data as outlined
in the previous section and used to specify the PA. In so me cases an AR(l)
model will be appropriate, giving simply
-
-A
PA = Pl
(9.2)
Using (9.1) and (9.2), To ,..., 2 for both a single AR(I) process with h = 0.3,
and two AR(I) processes, h;l = 0.6 and h;2 = 0.6, for tests of the mean
and correlation respectively. Consequently, the amount of data that must be
pruned depends on the statistic being estimated and tested.
9.4.4 Conservatism
In the example of Figure 9.4, note that pp tests of the univariate difference
in means at the nominal 2.5% level for h = 0.2 and at the nominal 1% level
for h = 0.3 are both operating at about the 5% level. If serial correlation
is modest but consequential a simple expedient to account for it in testing is
to reject the null hypothesis at a much lower level than would ordinarily be
appropriate.
9.5 Concluding Remarks
A knowledge of the material in this chapter, consultation of the listed references when needed, and some creativity will permit the application of permutation techniques in a wide variety of test situations. Their application in
turn permits a degree of objectivity in assessment of results of exploratory
studies that is often lacking in current practice. Obviously, difficulties remain that will pose obstacles for some problems, some of which have been
highlighted here. For example, work on the treatment of serial correlation
in tests of the standard deviation or correlation has not progressed as far as
175
will ensure a test series with inconsequential serial correlation in terms of the
statistic to be tested.
The formula for To depends on the tested statistic (Thiebaux and Zwiers,
1984; Trenberth, 1984). For tests of the mean,
N
ß
To = 1 + 2 L(1- N)PtJ.
tJ.=l
(9.1)
where N :::; n - 1 is the maximum of lags /). of the estimated autocorrelation
function Pt:.. For tests of the standard deviation, PtJ. is replaced by p~ in (9.1),
and for tests of the correlation of series 1 and 2, PtJ. is replaced by PA;l . PA;2'
The autocorrelation function, PA, in all cases must be estimated and this
is often difficuIt to do weIl. Generally, the maximum lag N must be set weIl
below n so that the sam pie size for all N lag estimates is reasonably large
(> 30 or so). Then all PtJ. can be estimated with relatively small estimation
error. If not, then a time series model should be fitted to the data as outlined
in the previous section and used to specify the PA. In so me cases an AR(l)
model will be appropriate, giving simply
-
-A
PA = Pl
(9.2)
Using (9.1) and (9.2), To ,..., 2 for both a single AR(I) process with h = 0.3,
and two AR(I) processes, h;l = 0.6 and h;2 = 0.6, for tests of the mean
and correlation respectively. Consequently, the amount of data that must be
pruned depends on the statistic being estimated and tested.
9.4.4 Conservatism
In the example of Figure 9.4, note that pp tests of the univariate difference
in means at the nominal 2.5% level for h = 0.2 and at the nominal 1% level
for h = 0.3 are both operating at about the 5% level. If serial correlation
is modest but consequential a simple expedient to account for it in testing is
to reject the null hypothesis at a much lower level than would ordinarily be
appropriate.
9.5 Concluding Remarks
A knowledge of the material in this chapter, consultation of the listed references when needed, and some creativity will permit the application of permutation techniques in a wide variety of test situations. Their application in
turn permits a degree of objectivity in assessment of results of exploratory
studies that is often lacking in current practice. Obviously, difficulties remain that will pose obstacles for some problems, some of which have been
highlighted here. For example, work on the treatment of serial correlation
in tests of the standard deviation or correlation has not progressed as far as