Bitwise Autocorrelation

We are sometimes led to new insights as a side effect of questions about different matters. For example, although our standard prediction for effects in the netvar statistic is a positive change, we have seen a tendency to negative-going netvar trends for some kinds of events, notably a subset of large, organized meditations. Looking for mechanisms that could help explain the sign of deviations requires delving into the generation of the bit sequences we sample at a basic level. One aspect of that is the use of a logical XOR operation to eliminate bias of the mean. The question is, how might this XOR affect the eventual bit sums we record. There are two types of XOR employed, one with an alternating bit-stream, and one that is a more complicated mask consisting of the (ramdomized) sequence of all bytes with an equal number of 1 and 0 bits. This is Peter's description of recent, suggestive work:

In the last email, I showed that IF the netvar deviation is driven by a short-lag autocorrelation, the deviation's sign will depend on the autocorr's sign. Furthermore, the effect of the XOR on the autocorr is simply to flip the sign of the deviation: XOR'd, positive autocorrelations at the bit level yield a negative trend in the netvar and visa-versa. While speculative as a model, it at least gives one *possible* explanation for pk-like deviations coupling to the netvar with a preferential sign: it's potentially due to the sign of an underlying autocorrelation. Netvar trends with positive or negative slope would then imply bit-level autocorrelations of one sign or the other.

The result is qualitative. We need to ask if it maintains for the XOR masks we have in place in the network. This bears looking into since the Orion and Mindsong masks are very different. Do the different masks imply different filters for an autocorrelation? Does any difference come through if we look at the two reg types separately across the events?

It turns out that the two masks "pass" an autocorrelation quite differently. The Orion 010101... mask cancels a long-lag autocorrelation efficiently, but a lag-1 autocorrelation perfectly matches the mask and is passed (albeit with the deviation's sign inverted wrt the autocorr's). The Mindsong, being 560 bits long before repeat, cancels short-lag autocorr's, but begins to pass autocorrelations more and more as the length of the lag increases.

Question: can the reg's different filtering characteristics be consistent with what we see in the event data? The short answer is yes. What we observe is that the netvar event cumdev is pretty much the same if we look at *only* mindsong regs or *only* orion regs (I've done this, but I don't know if you've seen this analysis by reg-type; see plots). So the question becomes, is there an autocorrelation for which the response is roughly the same for both reg types? It turns out that an exponentially decaying autocorrelation does it. This is nice since it is also the best guess model for any simple autocorrelation affecting reg behavior. A quick look suggests that an autocorr decaying over a lag of 10 to 20 bits balances the reponses fairly well.

There are some more details - such as what's happening to the covar - but I won't go into it all now. The bottom line is that *if* we ask how can the netvar go sometimes positive and sometimes negative, one answer is that a simple exponentially decaying autocorrelation at the bit level can do it, *and*, if this were the case, it would also be consistent with some important aspects of the internal structure we see in the event data.

Bitwise
Autocorrelation

Bitwise
Autocorrelation


GCP Home