|
Before we started collecting data in the newly established EGG network
in August 1998,
we thought about what kinds of events bring people together in a
widespread, shared focus of thought and emotion. The
first obvious candidate was the celebrations at New Years. The
transition from the old to the new is a focus point all around the
world. True, there are important "New Year" celebrations on
different dates, the Chinese New Year, the Persian New Year, and
ceremonies welcoming Spring,
but the main one is December 31 going into January 1st. Even in parts
of the world where there is another cultural New Year, there is a good
deal of attention paid to the midnight transition celebrated
in New York's Times Square,
in London, in Hong Kong, in Perth, in Hawaii -- practically everywhere
there are people. It is a natural because of the calendar,
and because it has momentum and is a really big party, worldwide.
More than any other notable moment in time, this one gathers us into a common
frame of mind. It has no strong emotion, though there's more love in
the air than usual, and isn't thought-provoking or
especially important. We come without much in the way of agenda other
than to wait together for the stroke of midnight, perhaps anticipating
a hug or a kiss from
someone close. We wait to lift a glass in a toast to the new year
coming, and we think a little about special times, some good, some bad,
that are now last year. Maybe we think about resolutions, but most of
us know already that we won't change much, even while believing it would
be a good idea. The New Year celebration is easy, and fun, and there
really is a general, gentle movement together. The simple common
interests that we share have no great importance, but they resonate
and turn us all in the same direction for a brief time. We keep track,
a little, while talking or dancing, or watching the show, and we are
ready when the count-down begins. There are few moments when so many
people think and feel in unison, and almost none that are so light and
pleasurable.
So, as midnight approaches on New Years Eve, an unusually large
proportion of humanity merge. Individualized movements and
expectations are put on hold, replaced by a kind of synchronized dance
of participation. The same kind of thing may happen when a terrible
event occurs, especially if it is an unexpected, surprising, awful thing.
New Years isn't like that, of course. On the contrary, it is
anticipated, prepared for, even traditional. It is almost like the rituals
of religious practice, but much simpler and easier to share. Just
focused attention to a moment with no intrinsic importance
or any deep meaning to distract us. An unusually relaxed moment in time.
Given all that, New Years is an ideal opportunity to consider
collective consciouness in its
clearest, purest form. No worries, no danger, no regrets. Brief and
precisely focused, the moment draws attention that is lightly and
willingly given, and probably because there are few competing
distractions, this momentary immersion in the abstraction of time is
a grand shared moment . And then we go back to the real
world, separating from each other and from the collaborative being we
were, momentarily.
So, what does it look like if we attempt to capture a signal in the sea
of noise our minds create in the world? If we really do collect and
share emotions and thoughts, we might expect that common focus to
produce a corresponding focus in the larger field of consciousness that
in some sense is already covering the earth with a sparkling, scattered
layer of thought and feeling. Think of those sparkles as notes in all
registers and rhythms, uncoordinated most of the time because there is
no score or conductor. But when there is something special, a shock
or surprise, a ritual or a celebration, then we might expect the
sparkling to develop ripples and waves that put some structure into the
unorganized display. Thinking in terms of sound, we can imagine the
random tunings of an orchestra changing to music at the rap of the
conductor's baton.
How the hypothesis is tested
We do two analyses on the data for each New Year, and we now have six
years to
examine. The basic notion is that as the New Year moment goes from
timezone to timezone around the world, there will be subtle but
detectable changes in the EGG data. To visualize this we make
a composite across all time zones of the period surrounding midnight.
We use a standard signal processing tactic
to reveal any faint patterns or structure
associated with the special moment of celebration as the old year
ends and the new begins.
The following graph is an example of what that looks like.
It shows a selection of ten
timezones presented separately, overlaid with a composite made by
summing across them. The combined trace (heavy red line)
is called a "signal average" or "epoch average". This procedure
helps to discover signal in a noisy background
because the random deviations tend to cancel each other,
while any consistent structure
tends to become more clear as more data are added.
The Meanshift
One of the formal analyses addresses shifts of the network output
from expectation. It looks at slight changes in the average score
across eggs for each second. We calculate a Z-score for each egg,
giving a normalized deviation from the expected score of 100. We then
make a Stouffer Z, summing algebraically across all the eggs, resulting in a
composite Z-score for each second. Next, the Stouffer Z-scores are
squared to give a Chisquare distributed quantity, and we plot the cumulative
deviation of the Chisquare from its expected value.
Finally, we do the signal averaging described above, to give a composite
across all time zones.
This complicated process is designed to represent any tendency for the eggs
to show correlated deviations. It is responsive to unusually large and
unusually
small scores, as well as consistency of behavior among the eggs. We are
looking for patterns of departure from random expectation, in the form
of correlated large excursions. The graphs show the accumulating
history of deviations over the 10-minute period around midnight, and
the terminal value corresponds to the test of significance.
The Variance
In the second analysis, the question is whether there are changes in the
variability among the eggs.
We picture the result by calculating the sample variance among the
eggs for each second, then making a composite by signal averaging the hour
surrounding midnight across all time zones, finally normalizing the data
as approximate Z-scores.
This gives an hour-long sequence of 3600 points, centered on midnight,
representing all the eggs and all timezones around the world
(there are 37, including those with half-hour offsets).
The graphical displays below use only the central half hour surrounding
midnight, which simplifies the picture by excluding overlaps with the
half-hour offset zones.
The variance measures
in this sequence are too noisy to directly reveal any structure, but
when smoothed by a moving average, momentary tendencies and persisting
trends can be seen.
We use a 4-minute averaging window, so each point in the final plot is the
average of 240 seconds centered on that point.
Robust calculations
Estimation of statistical significance for the variance measure requires
a different approach from the meanshift.
The figures below show the smoothed variance data for the New Years
transition at midnight, ± 15 minutes. We use a random permutation
analysis to find out how unusual the apparent structure in the data
may be. In this procedure, we randomly rearrange the actual data, and
count the number of times a minimum of greater magnitude (depth)
appears in 10,000 iterations, and ask how many times the random
permutations show the minimum point closer to midnight.
The combination of the these measures
of magnitude and proximity gives an estimate of how likely it is that
the apparent structure in the data is just a chance fluctuation.
However, this is a joint probability, and it cannot be directly compared
or combined with single-value probabilities.
My colleagues York Dobyns and Peter Bancel both recommend combining the two
measurables into a single measure, and using permutation analysis to
determine the probability of that measure against its null-hypothesis
distribution.
To test the hypothesis that there will be a reduction in variance and
it will occur near midnight, York suggests that a logical
candidate for a combined measure would be
VT = a*Vmin + b*dT, where Vmin is the variance at minimum, dT
is the absolute time interval from midnight, and a and b are pragmatically
chosen coefficients to give both measures roughly equal weight, that is,
to have their respective variations contributing about equally to the
variability of VT. Peter suggests a mutiplication of the two
aspects. It turns out that the two approaches give similar results with
suitably chosen coefficients.
Either method allows us to calculate the distribution of VT over the data
permutations, and compare that with the value of VT in the data. The
result is one
measurable, one distribution, and no meta-analytical problems.
In this case, VT = a*Vmin + b*dT becomes VT[i] = abs(100*V[i])+abs(1000/T[i])
in each permutation, and is compared with the original data value
VT[0] = abs(100*V[0])+abs(1000/T[0]),
which is expected to be
large if the minimum is deep and close to midnight.
The multiplicative version is similar, with
VxT[0] = abs(100*V[0])*abs(100/T[0]).
This result is shown in parentheses below, symbolized as VxT.
The full hour surrounding midnight was used
for these calculations, but a comparison made
using only the central half hour, midnight ± 15 mins, shows very
similar results.
Statistics and graphs
The following figures show the the "meanshift" analysis on the
left and the "variance" analysis on the right. The former tests the
prediction that the eggs will tend to produce relatively large and correlated
deviations during the 10-minute period centered on
midnight. The variance analysis tests
our prediction that as midnight approaches, the variability of the
data across the eggs will decrease,
reaching a minimum near midnight, then returning to normal.
1998-1999
The meanshift measure conformed to the a priori
prediction of a positive trend during the 10 minutes surrounding
midnight. The departure from expectation was mainly after
midnight, and the total deviation
corresponds to a probability of 0.085.
In the variance measure (which is a post facto analysis for this year)
the deepest minimum reached by the smoothed variance
was exceeded almost half the time (p = 0.447), but it was closer to
midnight in all but 15% of the random permutations (p = 0.147).
The combination of magnitude and proximity yields a joint
probability of p = 0.066.
The rigorous VT statistic for 1998-1999 gives a probability of 0.0647.
(0.063 for VxT).
A simpler measure of reduced variance around midnight agreed upon in
discussions with Peter Bancel in April 2004 shows Z = -3.199
for midnight ±5 minutes.
1999-2000
For the much anticipated "Y2K", the
meanshift measure again conformed to our prediction, with a
modest positive trend. The terminal value
corresponds to a probability of 0.128.
This was the first year for which an a priori prediction for variance
reduction was made, by Dean Radin. The analysis method was not
prespecified, so the current procedure, which was developed at that time,
is applied post facto for 1999-2000.
It is fully a priori in subsequent years.
The minimum reached by the smoothed variance
was extreme, with only about 1% of the permutations showing a deeper
minimum (p = 0.016). About a third of the cases were closer to
midnight in the random permutations (p = 0.312).
The combination of magnitude and proximity yields a joint
probability of p = 0.005.
The robust VT statistic for 1999-2000 gives a less impressive
probability of 0.0965.
(0.115 for VxT).
The simpler measure of reduced variance around midnight shows Z = -2.366
for midnight ±5 minutes.
2000-2001
The meanshift showed a persistent trend opposite to the
prediction, and the probability is p = 0.812.
The variance measure also did not show the expected reduction
around midnight. The deepest minimum reached by the smoothed variance
was exceeded over half the time (p = 0.565), and it is not especially
close to midnight, with 53% of the random permutations closer (p = 0.527).
The combination of magnitude and proximity yields a joint
probability of p = 0.298.
The robust VT statistic for 2000-2001 gives a probability of 0.523.
(0.265 for VxT).
The simpler measure of reduced variance around midnight shows Z = -0.493
for midnight ±5 minutes.
2001-2002
The meanshift is not distinguishable from a random walk,
and the calculated probability is 0.291.
The variance measure, in contrast,
has an impressive appearance that is a classic match to the prediction
of reduced variance around midnight. However, the permutation analysis shows
that the deepest minimum reached by the smoothed variance
was exceeded more than 80% of the time (p = 0.831). On the other
hand, the minimum was very close to
midnight with only 5% of the random permutations closer (p = 0.048).
The combination of magnitude and proximity yields a joint
probability of p = 0.039.
The robust VT statistic for 2001-2002 gives a probability of 0.023.
(0.030 for VxT).
The simpler measure of reduced variance around midnight shows Z = -2.059
for midnight ±5 minutes.
2002-2003
The meanshift measure strongly conformed to our
prediction, with a positive trend throughout. The terminal value
corresponds to a probability of 0.013.
In the variance analysis,
the result appears to be opposite to the prediction.
The deepest minimum reached by the smoothed variance
was exceeded only 19% of the time (p = 0.190), but it was very far from
midnight, with 99% of the random permutations being closer (p = 0.992).
The combination of magnitude and proximity yields a joint
probability of p = 0.188.
The VT statistic for 2002-2003 gives a probability of 0.3413.
(0.364 for VxT).
The simpler measure of reduced variance around midnight shows Z = 0.313
for midnight ±5 minutes.
2003-2004
The meanshift measure was persistently backward
relative to our prediction, with a negative trend through most of the
period. The terminal value
corresponds to a probability of 0.897.
The variance analysis again presents a classic picture conforming to the
prediction, but it remains a subtle effect that is just marginally
significant. The deepest minimum reached by the smoothed variance
was exceeded about a third of the time (p = 0.378), but it was closer to
midnight than in all but 9% of the random permutations (p = 0.095).
The combination of magnitude and proximity yields a joint
probability of p = 0.036.
The VT statistic for 2003-2004 gives a probability of 0.0432.
(0.060 for VxT).
The simpler measure of reduced variance around midnight shows Z = -2.187
for midnight ±5 minutes.
Combining Across Six Years
Finally, we look at the combined data from all
six years, presented in graphs similar to those for the
individual years.
For the meanshift measure, the Stouffer Z sequences for the six years
were averaged using the same Stouffer Z procedure across years. The graph
below left presents the result, which shows an impressive, persistent
trend beginning just before midnight.
Even with two of the six years showing a pattern
that is contrary to the prediction, the composite is marginally
significant. The terminal value has a probability of about 0.047.
We can get a different perspective
by combining the probabilities from the individual years.
Rosenthal gives an algorithm that yields a Chisquare statistic
from the sum of their logarithms: Chi = sum(-2*log(p)).
The result is p = 0.053 for the composite of six years,
which corresponds well with the probability
estimated from the Stouffer combination shown in the graph.
For the variance measure averaged across all six years,
the permutation analysis applied directly to the combined data
yields p = 0.355 for the minimum, p = 0.239 for its
proximity, and p = 0.087 for the combination.
A more rigorous calculation of the variance
probability, based on the VT
statistic estimated for the individual years, can be obtained
by combining the probabilities from the permutation analyses using the
Rosenthal algorithm.
The result for the six years 1999 to 2004 is p = 0.0067 (0.0061 for VxT),
which is considerably stronger than the estimate from the combined graph.
The difference arises because the six years
are spread out in terms of the proximity measure, so the combined graph
is less sharply focused.
Though there is a visually impressive minimum spike almost exactly at
midnight, a deeper transient occurs some seven minutes before, and this
is the one that is taken as dT in calculating the VT measure.
This souce of variability is ignored in the combination of individual
year probabilities. The graph covers midnight ±
30 minutes, rather than ± 15, to show the broader picture.
Estimating from the simple Z for the variance in the 10-minutes
around midnight, the combined Z representing the reduction of variance
for the six years is 4.079.
A Concluding Note
We should recall that these analyses, especially the
combination of the variance measure results, are in some degree post
facto. Only the last four years of the variance analysis were
fully prespecified, and if we compute the aggregate result from those
four years the probability is 0.022 (compared with 0.007 for the six
years).
Nevertheless, it is correct to say that there is evidence for structure
that should not be found in random data, associated with the brief
period of time surrounding the New Year transition.
The joint probability of the
result for the meanshift and variance analyses is
between 0.001 and 0.0003 (using 4 and 6 years, respectively,
of the variance measure).
|