Gerard 't Hooft's Bet
On Mon, 18 Aug 2003, Axel Harvey wrote:
> I presume you have seen the bet in Gerard 't Hooft's web pages. It > seems to be aimed squarely at the GCP. He even links your site. > > http://www.phys.uu.nl/~thooft/parabet.html > http://www.phys.uu.nl/~thooft/paraplue.html > > (The link to the GCP site is reached from the latter URL. Go to the > bottom of that page and click on "scientifically detected".) > > Maybe you have mentioned this already in your website or emails - > sorry if I've missed it. What do you think?
I appreciate Gerard 't Hooft's interest and his willingness to take the time to think about the GCP database and results. It is a large corpus and thus it is perhaps understandable that he is not (apparently) aware of some aspects of the database that are relevant, and has incorrect impressions about the types of events we assess.
I had not seen his web page yet, though I have been in contact with 't Hooft once directly and other times indirectly by way of my colleague Dick Bierman. Thus I am aware of his basic idea. It is not a bad proposal in some respects, but it is ungainly and demonstrably unnecessary for a substantial proportion of the GCP database, even if one believes there are grounds for his concern. In addition, because he is looking for 5 sigma, it would, according to our experience (and statistical power estimates), require on the order of 100 events (about 2.5 years at our average rate) to have even .5 beta of reaching such a level. His plan also requires that everybody be blind to the data and that "An experiment must be analyzed in one single procedure, immediately after disclosure of its results." Together, if I understand him correctly, these conditions would require that we run the experiment blind for two or three years.
I have suggested to 't Hooft an alternative procedure that is also immune to the problem he is concerned about, namely:
That statement or description is not true for a substantial proportion of the GCP events that constitute the experiment. I suggest that for cases that are specified a priori, that is, before the event takes place, none of the concerns or preferred "explanations" in 't Hooft's description are valid. Thus, I propose that the results for the a priori cases be examined and compared with those that are necessarily specified after the fact. This is a simple and instructive test of the viability of 't Hooft's criticism.
About 45% of the formal predictions can be and are established before the event. For example, we have each year made predictions about the New Year transition period. Obviously, these can be made ahead of time. Similarly, for web- or media-organized global peace marches, or major religious gatherings or meditation events, we know the publicized, planned time and can make an a priori prediction. There actually are a considerable number of potentially interesting, often unique events for which such a priori specification is possible: The moment of the Dow Jones Industrial Average first breaking the 10,000 barrier; the major solar eclipse over Europe and the Middle East in August 1999; openings of the Olympic games; the Kumbh Mela in India; Earth Day celebrations; and so on. We have as of August 2003 a total of 147 formal predictions and tests of the general hypothesis that there will be departures from expected random behavior in the GCP data that are correlated with major global events. Of the 147 events or cases, 67 are defined by a priori predictions, and 80 are post facto because they address accidents, explosions, disasters, events of war, etc., which cannot be subject to a priori specification.
So, 't Hooft's description, above, is not and cannot be relevant to a substantial number of cases in the GCP database. The question is whether these cases show less evidence for the correlations we see overall in the database. The answer is no -- indeed they present a somewhat more persuasive bottom line. The a priori cases have a smaller mean probability (0.354 vs 0.410 for the post facto subset), and despite the smaller database size, they actually have a more extreme composite p-value (1x10e-6 vs 9x10e-4 for the post facto subset).
Going beyond this simple demonstration, Dick Bierman and I are discussing and preparing to test a procedure that meets, even for the post hoc cases, the standards 't Hooft proposes, while also allowing the project to proceed with little interference. Although I don't intend to engage in the Randi-style "bet" that 't Hooft uses to express his depth of conviction, I will keep him informed.