Last Update 20 Jan 2005

This page is a working document and is not to be quoted or referenced

Return to GCP Home Page

Sliding the Event Time

The background of careful preparation for rigorous analysis can be envisioned as a conversion of the GCP database to a "data resource" which can be examined with power and flexibility. This small study is an example of what can now be done with some facility. Moreover, it has some suggestive implications that lead to further questions.

Peter's description:

I am studying the event set, as we proposed. As a preliminary, I looked at the aggregate event zscore when the event examination periods are shifted uniformly in time. That is, I slide the event periods over the dbase in 1/2 hr steps and calc the aggregate Z at each step. The attached plot shows this calc. The periods are shifted in 30 minute steps, from -24hr to +24. To make this calculation I removed 5 events longer than 24 hrs. So there are only 165 accepted events in the set, not 170. The aggregate zscore at zero offset is thus 3.8, instead of 4.03.

(It is perhaps interesting that moving the events by only 1/2 hr essentially kills the zscore, but one needs to remember that nearly half of the event examination periods are 1 hr long or less. So we "slide off" many events in only a few 1/2 hr steps. I was also interested to see if there's a difference before and after. In the plots you can see that there is more positive weight to the shifted z's after zero offset than before. I expected this, for various reasons, but the result is too weak to have much significance.)

This is a demonstration of the kinds of things that can be produced fairly easily at this point. We have developed a number of questions that are capable of informing us deeply about the nature and quality of the evidence. As we proceed, we expect to have many cases that, in Peter's term, "will require a lot of mulling," but we should learn much from the ability to visualize the data in many different ways.

See below for some more recent "mulling" that brodens this picture.

Eventslide Extended to Impulse Event Analysis

These notes explain the graphs below. Click on the figure to see full-size originals.

Background

The eventslide looks at the cumulative Z-score for events versus a time offset of the formal event examination periods. A negative timeshift, T<0, means that events are calculated using an earlier starting time for the event. A positive shift means the event is given a later start time than the formal prediction. The durations of the events are unchanged from the formal predictions in these calculations. A timeshift T, then, means that all events are given start times offset by T from the original predictions. Looking at successive shifts is like "sliding" the event periods in lockstep over the data.

The eventslide analysis is a way to judge qualitatively how sensitive the cumulative Z-score of events is to the exact prediction periods. One possibility is that the predictions merely guess or estimate the times when the data actually deviate anomalously. In this case, one would expect that sliding the event periods in time would give a gradual rise and fall of the Z-score. The width of the rise/fall would give some indication of the "real" width of events.

An alternate possibility is that the cumulative Z-score is due to an anomalous intuition on the part of experimenters. In this case the event periods are guesses about fluctuations in random data and a simple model would say that sliding the event start times should quickly extinguish the Z-score. An early look is shown in the plot eventslide2.gif. The time offset, T, steps in 1/2 hour intervals in this plot. The Z-score is 3.6 at T=0 and after a single step all significance is lost. Timesteps of 5 minutes give a similar result.

One could take this as an indication that an intuition model is favored. Certainly a model that assumes anomalous data deviations would have a difficulty explaining why the formal predictions should be so finely tuned to the data anomalies - especially since most of the prediction periods are rounded to whole hours. However, it is difficult to make this interpretation because the Z-score is calculated using events that range from 1 minute to 24 hours in length and a variety of recipes are applied, as per the formal prediction specifications.

To address these difficulties the eventslide analysis is refined in two ways:

1. Events of 1/2 hour or less and events longer than 1 day are removed from the event set.

2. A uniform recipe (the so-called 'deviation statistic' or 'dispersion statistic') is applied to the events.

Applying these simplifications excludes about 30 events. The reduced set has approximately 145 events and retains enough significance for the analysis. (The cumulative Z-score for the subset increases to about 4.0, using formal recipes, and z = 3.7 obtains using the deviation (dispersion) statistic).

Discussion of the Graphs

The top panel in the plot below compares the formal (red) and uniform (blue) recipes applied to this subset of events. The cumulative Z-score is calculated for timesteps of 15 minutes. [When studying the plots, remember that z-scores magnitudes less than about 2 are not significant. It is helpful to compare right and left hand panels].

The formal (red) trace in the top panel suggests that, although the slide analysis still shows a dominant spike at T=0, there is now some additional weight for offsets within 2 hours of T=0 when compared to the full event set in the previous graphic. Interestingly, the slide width of +- 2 hours is more prominent when the uniform deviation statistic is applied (blue trace). Folding the deviation statistic and reduced event set into the eventslide analysis suggests that there is a width of about 2 to 4 hours to the slide that was not evident in the first eventslide analysis. This favors the view that anomalous deviations occur in the data and that event predictions approximately frame these periods (although this is not to say that one couldn't make a model in which an experimenter effect accounted for the eventslide width).

Ideally, the eventslide analysis would use events that had equal lengths and unambiguous start times. It is thus interesting to identify the 55 events whose start points are both unforseeable and clearly defined. These "impulse" events include earthquakes and terrorist attacks, for example, but not events with immediate anticipation such as the death of the pope or the conclusion of a trial or other contest. The middle and bottom panels of the figure show the slide analysis for the subsets of impulse and non-impulse events.

The middle panel is the slide calculation using the formal recipes. The black trace is the set of impulse events and the red shows the slide analysis for the remaining events. The traces show a width as before, but there is not enough significance to examine the difference between the traces.

The bottom panel shows the same slide analysis using the uniform deviation statistic. Here an interesting and marginally significant trend obtains. The impulse events (black) and non-impulse events (blue) both show slide widths of 2-4 hours. However, the impulse events have weight shifted to early times whereas the non-impulse events have slide weight symmetrically distributed about T=0.

Interpretation

If the slide width is due to inexact guesses of deviation periods, the width should be symmetric about T=0 for events with ambiguous start and end points. This is the situation for the non-impulse events. The T=0 symmetry should be broken when the start period of the event is clearly defined. In this case, the data deviation and the event prediction are both related to a known moment in time. The expected shape of the slide width should be positively skewed (rising more sharply at negative timeshifts and tailing off at positive shifts). Qualitatively, this is seen in the impulse (black) trace, but the statistics are very weak [modeling the slide would allow a probability value to be assigned to the impulse slide skewness]. The impulse trace is also displaced toward negative timeshifts by 1-2 hours. This is an indication that the data deviations tend to start before the event start times. This would indeed be a surprising result, but there is a precedent in the September 11, 2001 analysis which found a strong statistical departure in one statistic roughly 3 hours before the terrorist attacks began. Stronger statistical evidence will be needed to study further the possibility of predictor data deviations.

One more thought (from Peter Bancel) on the slide interpretation for the impulse events: In the bottom panel of the plot, the impulse trace has a center of mass at negative shift. This could mean the deviations start before the event, as we've discussed. Another possibility is that the deviations, on average, last for a significantly shorter time than the examination periods specified in the predictions. For example, if the events all had positive deviations lasting 2 hours and the predictions were all for 4 hours, then the slide would begin to rise at -4 hrs, have a flat max between -2 and 0 hrs and return to baseline at +2 hrs. That would qualitatively describe the black impulse trace. Of course, that trace is for real events with many different exam periods. But a senario like this could explain a slide shifted to negative offsets.

Covar & Netvar Slides, and Simulations

Notes by Peter Bancel:

The following plots look at comparisons of netvar and covar over events. I want to use them for secondary support of the basic cumululative event Zscore.

The main point is that the covar measure shows a Z of about 2.6 over events at zero offset, where the netvar measure Z is about 3.8. Is this meaningful? The covar is an independent stat and suggests more is going on in the events than we see in the formal predictions. This is old news. But the plots flesh it out.

Comparing the covar and netvar slides shows that they appear to track. It looks like the structure for the covar and netvar is similar over the events. This is stronger evidence than just seeing a spike at slide time of zero offset for the covar, which would look like coincidence. The tracking over -2 to +6 hours supports, albeit weakly, that a similar cause is driving the two stats.

The next plot (SlidSim.gif) shows that the netvar and covar don't correlate in the full GCP database. Background: the netvar for events gives an average z of z = 0.3. To make the SlidSim, I selected a set of 500 4-hr simulated "events" at random from the database with the condition that their netvar zscore was normally distributed about z=0.3, as is the case for the formal events. Then I used the data to do a slide comparison of the two stats. Very nicely, we see that choosing the mean z=0.3 for the netvar gives a strong aggregate score with the slide peaking at offset= zero, *and* the covar slide on the same data is *flat*. I take this to mean that we can safely assume that the two stats don't correlate (we've already seen this by other means, but it's good to confirm the point) and therefore we can meaningfully compare the netvar/covar slide plots over the formal events.

A second thing that pops out of this is there is a clear difference when the events are broken into events involving "big" and "small" numbers of people. Big is 9/11 and small Astrid Lindgren, for example. This is another demo of what we've seen in other analyses: It looks like there is more effect size in big events.

Combining all the events gives a picture that is similar to that of the "big" events. Note that the covar shows some sigficant weight at large negative offset. This has no obvious interpretation other than chance fluctuation, and it means we should be careful not to over-interpret these findings.

A different way to do the slide is to make a constant window for all events. The window is referenced to the event start time and then moved backwards and forwards to give the slide. The slide using formal event lengths gives equal weight to all events, but different events have different lengths. That means that for a given slide offset, some events may be "outside" the slide and others partially inside. A constant window also weights events equally, but also uses a constant period of data for all events.

There are two ways to do it: reference the start of the events or the end of events. Referencing the start gives a fairly clean view of the lead-up to the events. Referencing the end gives a look at what happens after events. The figues below show both views using a 3-hour window and the dispersion statistic, which combines the netvar and covar. (They track each other, but are independent, so we can construct the dispersion statistic as (netvar + covar)/normalization denominator.)

The figures below display the results for the slide procedure defining the 3-hour window by the start of the event (left) and the end of the event (right). Most interesting is the end referencing. It suggests that there is a rebound of the stat *after* the event. Note: My convention is: The zero offset for the end referenced slide sets a 3-hr window which *ends* at the formally predicted end of events. A positive offset of 3 hours in this case, means that all events are calculated for data periods starting at the formal event end time and continuing for 3 hours from there.