Standard Deviation of REG devices
Mike Meyer has constructed Java modules to process huge amounts of data. His application to 6 months of data showed that the mean of all the REG devices was within specification, but that the standard deviation for many of the eggs is slightly different from the theoretical value. The average change is an increase of 0.0127%. Using this value as an estimate for the change that would result in the usual calculations if the empirical value were substituted for the theoretical value gives some idea of the implications. An application to the formal database indicates that the net effect is to reduce the cumulative effect by a factor of 0.00025. Because this is not consequential, we will continue to use the theoretical value for computations. Mike has constructed a table showing the result of using empirical estimates for SD per egg, based on 6-month samples. The tabled values are (I believe) the Z-scores representing the difference between theoretical and empirical results, which would be zero in a perfect world.
Here are the details of a summary of the SD change by egg type, and a
worst-case assessment of the implication for the formal results accumulated thus far in the EGG project.
Theoretical SD 7.0711 PEAR device SD, N=4 7.0687 Mindsong SD, N=16 7.0732 Orion SD, N=7 7.0709 Prop smaller PEAR -0.00033941 Prop larger Mindsong +0.00029698 Prop smaller Orion -2.8284e-05 Prop larger All 3 +0.00012728 Proportional overestimate of Z-scores using All 3 is 0.00012728 Corresponding overestimate of Z2 is on the order of 0.00025456 The degrees of freedom across the 76 events are 26120 The original Chi-square based on theoretical SD is 27195 The reduced Chi-square based on the All 3 SD is 27188 The worst case reduction, based on the Mindsong SD is 27179 The theoretically based calculation gives probability 1.715e-6 The estimated SD from the All 3 calculation yields p= 1.9805e-06
My conclusion is that using the theoretical standard deviation does not produce misleading bottom-line conclusions. The difference of actual device parameters from theory is on the order of a hundredth of one percent, and this means that in general, the calculations are accurate to three or four decimal places. Since we are dealing with statistical estimates always, I can accept this level of inaccuracy as being well within the noise, even if it is a systematic error. At some point in the future, when someone wishes to do a more precise calculation, based on locally computed empirical standard deviations for all eggs, we will use it, but at this point, there are other interesting analyses and projects that must be given higher priority.