|
Each New Years for several years,
we have had a hypothesis/prediction for an increase in the Network
Variance in the 10 minute period surrounding midnight, and a focused
decrease in the Device Variance at midnight. (Formal predictions for NetVar
were in place for 1999 to 2005, and for DevVar fro 2001-2005.)
The analysis uses a signal
averaging procedure to compound the data for each successive New Year
progressing through the timezones.
This composite across timezones is the same
procedure as is used to see evoked potentials in psychophysiological
measures.
A simple picture of time differences around the world assumes 24
timezones, but the formal GCP analysis is based on
37 official timezones, from -14 to +12 hours from GMT, including 10
zones with half-hour offsets.
The formal analyses are based on the fully detailed data, with
1-second resolution. The analyses use all available eggs, including
a few that may be unsynchronized relative to network time.
The total number of eggs grows larger over the years, beginning with 9
in 1998-1999, increasing to 62 in 2004-2005.
An error was made in the matrix manipulation used for the signal
averaging when it was first designed. The mistake was discovered during
the re-analysis that was part of the Analysis 2004 project.
The graphs below show the original, incorrect presentation of the two
measures, NetVar and DevVar, in the top row, together with the new,
correct version of the analyses in the bottom row of each figure.
The first figure below shows all 7 years combined.
For the NetVar, the significance of the departure from
randomness can be estimated by comparison with the blue curve that
shows the p=0.05 threshold. For the DevVar, a permutation analysis
shows the combined likelihood of the magnitude of the reduction
and the proximity to midnight of the lowest point (proportion of
Mag x Prox > Orig Mag x Prox in 10000 permutations).
For the original calculation, this yields a non-significant p=0.553,
and for the new, correct version of the analysis, p=0.038.
The figures on the left show the Network Variance (NetVar), which is
the Stouffer Z across eggs, squared and cumulated. The prediction is
for a positive deviation over the whole 10-minute period.
The blue parabola gives an indication of the likelihood of the
accumulated deviation.
The figures on the
right show the Device Variance (DevVar), which is
the variance calculated across eggs and normalized as Z-scores for each
second, then squared. The resulting sequence is smoothed with a running mean
using a window of 240 points (4 minutes).
The variance reduction hypothesis is supported if the smoothed
curve displays a clear reduction of the variability of the squared
Z-scores around midnight.
The next figure shows the most interesting of these in a larger format.
It is the corrected data for the variance (device variance) prediction,
which says that we hypothesize a reduction in variance near midnight.
A permutation test shows that the depth and the proximity of the actual
smoothed curve shown here has a probability of 0.038.
If we use a 5 minute smoothing to explore an alternative choice, the picture is similar, as is the
p-value at 0.032.
In the course of the Analysis 2004 project, Peter Bancel proposed to
replace the complex, permutation based DevVar analysis with a simple
calculation of the device variance of all eggs over the 10-minute period.
This calculation is approximated by the +/- 5 min Z shown in the legend in
the individual figures. For the whole 7 year dataset, this measure yielded a
compound Z for the original, incorrect DevVar calculation of -1.159.
For the new, correct
DevVar, the compound Z is 1.701, which is marginally significant but
opposite to the prediction
of reduced variance. Calculated from Peter's normalized data, the Z for 6
years 2000-2005 is 1.126.
Assuming these are reasonably good
approximations, a conflict is apparent: these values for the new
calculation are backward relative to the original prediction
and opposite to the permutation-based outcome (Z=-1.774). This
suggests that the replacement measure does not represent the same
structure as was defined as the hypothesis test in 1999-2000.
For the DevVar, as noted above, the original
hypothesis test was shape based, and
permutation analysis was used to determine what the probability was for
the combined magnitude and proximity to midnight of the variance reduction.
A simpler analysis formulated in 2004 asked whether there was a
reduction in the overall device variance during the 10-minute period.
The former test is represented by the shape of the graph, and
the latter calculation is shown in the DevVar figure legends for the
individual years in the following figures.
|