Hi Roger,
Here is a quick note since I haven't checked in in a few days.
Been working hard.
Things are starting to look very interesting indeed.
I have been working on two fronts.
The big news is that I have looked at the alternate second data and there is a very clear correlation between the Stouffer z-sqrd cumdev [and its Janus twin, the network variance] for two datasets with alternating seconds of data. That is all odd seconds go to one set and the even seconds go to the other. There is a strong correlation which is exactly what one expects if an anomalous effect is responsible for the structure in the data. [The significance depends on the numbers and I'm calculating empirical distributions for my correlation function. So it will take a day or two to have more precision.]
But it's better than that.
The correlation only comes through for the post-9/11 data.
And it is coming from the same structure that we see correlating with the poll results. So this alternate second result is independent of the poll result and also supportive of it ( or visa vera) Anyway it is starting to make a very nice story, since we now have
1. event experiment significant for an effect on short timecsales and connecting to global scales.
2. alternate datasets showing significance on long timescales.
3. poll results connecting long time behavior to global events.
That's the short of it.
The attached html doc has some more details and some more are to come.
The technical point of importance is that I've devised a correlation test capable of detecting correlations in structure [as opposed to the too limited usual correlation coeffcients that miss detailed structure like peaks]. This is sketched in the doc.
The other front is the poll results. I'll use my structure correlation test to look at the data/poll correlation quantitatively. But another very interesting avenue is to study the correspondance by making a mapping from one to the other. I fiddled around a bit and it looks possible, but it will take some work. It would be a nice demo to show how you can generate the structure in the GCP data by a simple transformation of the poll data.
I think we have some red meat.
At this point we should reconsider the late April date for a meeting. This could be the good moment to do it and we could still make it if we move fast. Read over the doc and let me know what you think.