Hi Roger,
Here are two revised plots with envelopes updated
(tho' I'm still calculating...)
Netvarcorr.gif should replace index_gr_5 (give it
that name if you want it to load in the memo htm page)
NetvsDevcorr.gif replaces index_gr_9.
The basic result is a pval of better than 0.003 for
the alt-sec post 9/11 correlation.
The Z-score for that is 2.8.
The big picture that is emerging is
this:
We show correlation between GCP data and a societal
metric AND we show independently that the GCP data has
non-random structure (via the alt-sec analysis).
The icing on the cake is that we can unpack the
alt-sec analysis to show the data features that correlate on alternate seconds
are precisely the ones that correlate with the poll. So it's a check and mate
situation. I think we can show that this hangs together at the 0.001 pval
level.
The big things we are learning are:
1. We can test to see if data trends are non-random
(incredible!).
2. We can determine what stats capture the effect.
For instance is it the netvar or the devvar? (this was the goal
of the event-based analysis).
What we simplify for the moment is the possibility
that the effect has several aspects (it could be global consc + experimenter,
after all...)
So where do we go from here?
First, we should talk. Can we do it
today?
We need to move quickly if we want to try for a
Spring meeting.
Also, I need to make some commitments for the next
6-9 months in the next few days.
[I delayed decsions when I first saw the poll
correlation]
What we decide to do for the project effects my
choices.
Here's what I'd like.
1. Have an meeting in late April
If we want a meeting we should send the analysis
memo to Dean (and Marylin?) asap to get them excited and fix the Ions date. If
they're ok, send emails to principals and nail it
down.
2. Write a paper for FoP
A big lesson I learned (the hard way) during my
thesis and later during the post-doc at IBM was when one should cut the work
and sit down and write a Letter. My gut is telling me big-time this is a
cut-and-write case. I'm pretty sure we can get a Letter published. This is
also an excellent preparation for the April meeting. It will also help loads
for funding requests, so best to get it in the pipeline
now.
3. Find some money so I can put time into the
analysis.
Eternal problem but I'm a pumpkin without some
revenue.
Some immediate next things to
do:
1. Calculate the correlation of netvar
and presidential poll data.
2. See if the correlation for alt-secs
works on shorter timescales : look at 9/11.
If this is so we have independent
evidence that the strong 3-day deviation after 9/11 was not merely an
extraordinary chance fluctuation. That would be a substantial
result.
3. Check the correlation for alternate
minutes of data, instead of alternate seconds . [This will be a nail-in-the-coffin for "inherent electronic
autocorrelations in the devices"-type arguments against anomalous
interpretations. Actually, there is a good story with several parts to destroy
those objections]
These 3 are all quick to do.
There are important and obvious
further tracks to take. But most of them could potentially get bogged
down and take considerable time to get right.
One priority direction is to look for
another metric like the poll data.
Another is to look for a better stat than
the netvar.
[actually, I suspect that a measure
of the average reg pair-correlation is the underlying statistic. This is a
major component of the netvar...]
But I think we should focus on a draft
paper for the end of February.
-P
----- Original Message -----
Sent: Monday, January 10, 2005 10:42 AM
Subject: c