Dependency and Correlation

In my last post I irreverently discussed the topic of correlation, or more specifically, the issue of causality. I want to delve a little deeper into this today. It’s important for the Neverending Story of covid, but actually it’s something I’ve been fascinated with for many years because when we tie in some of these ideas with Quantum Mechanics (QM) we find a very strange thing indeed.

So we’re going to see what might be involved in trying to figure out whether the Goo has caused an increase in Myocarditis. The data we have is going to be represented in terms of the variables G and M. These variables can take on the values 0 or 1. We’d have a table which would list our population accordingly. So one entry in the table, say for person 50 on the list, we might have G = 1 and M = 1, which would mean person number 50 has had the Goo and also has Myocarditis. Person number 51 might have the entry G = 0 and M = 1 indicating that this person hasn’t been Goo’ed but has Myocarditis. And so on.

Let’s put in some hypothetical figures to make things more concrete. We’re going to imagine that 1,000 people haven’t had the Goo (G = 0) and 9,000 people have been Goo’ed (G = 1). We look down our table (which has 10,000 entries in total) and work out how many Goo’ed have Myocarditis (M = 1) and how many unGoo’ed have Myocarditis (M = 1).

We end up with the following:

G = 0, M = 0 : number 900
G = 0, M = 1 : number 100

G = 1, M = 0 : number 7,200
G = 1, M = 1 : number 1,800

As we know, we can’t just look at the totals here - we have to look at the rates. Here we see that 10% of unGoo’ed people have Myocarditis and 20% of Goo’ed people have Myocarditis.

So we might make the statement that if you’ve been Goo’ed you’re twice as likely to have Myocarditis than if you remain Goo-free.

Ah - but now we’ve introduced a new idea - probability has snuck in with the word likely. And that’s OK - it’s the right idea to introduce - but it does mean we have to be a bit careful. Where we want to get to is some idea of whether the Goo is causing problems (with respect to Myocarditis) and to do that we need to think statistically - which ultimately means having to cosy up with our old friend probability.

When we say “twice as likely” what we are comparing are two probabilities. We’re making a statement that

P(M = 1 | G = 1) = 2 P(M = 1 | G = 0)

Ouch - that doesn’t look very friendly - but, fear not, it reads like this : the probability that you have Myocarditis given that you’ve had the Goo is twice the probability that you have Myocarditis given that you remain Goo-free. The vertical line here reads as “given”.

But what does the word probability even mean in this context? OK - here’s what it means in this context. You take the Goo’ed and put them in one room. You take the unGoo’ed and put them in another room. You then pick, at random, one person from the Goo’ed room - what’s the probability that the person you pick has Myocarditis? This is what we mean when we write the expression P(M = 1 | G = 1).

So is the incidence of Myocarditis correlated with the Goo?

Informally, yes. Technically speaking there is a statistical dependence between the two variables G and M. But also technically speaking, dependence and correlation are not quite the same things. We’re going to ignore this and assume that dependence and correlation amount to the same thing.

But dependence is not really the best way to frame this either. What we really should be thinking is whether knowing the Goo status confers extra information about the incidence of Myocarditis. If you were gambling, and wanted to place a bet on picking someone with Myocarditis, would you pick someone who was Goo’ed or Goo-free? Would you head for Room Goo, or would you head for Room Goo-free?

In this case, knowing the Goo status of someone would give you an advantage over someone who didn’t.

The person who didn’t know the Goo status (which means they didn’t have the advantage of having the separated rooms) would have a (100 + 1,800)/10,000 chance of picking (at random) someone with Myocarditis which is a probability of 0.19

The person who did know the Goo status would only make the pick from the Goo’ed and would have a probability of 1,800/9,000 = 0.2 of picking someone with Myocarditis.

It’s only a small increase in odds - there’s not a whole lot of extra information there, but there’s some.

Knowing the Goo status gives you a bit more predictive power.

If knowing the Goo status made no difference at all then the variables G and M would be independent - and knowing G would give no information at all about M - it would be irrelevant for any determination of M.

So it’s case closed then? Getting the Goo increases your chance of developing Myocarditis?

Not quite - and here’s where the issue of confounders comes in. A confounder is just a fancy name (we geeks do love our technical jargon) for saying that there may be other factors, other variables, at play.

Let’s suppose that our Goo-free population is younger than the Goo’ed population, on average. If there was, naturally, an increased risk of Myocarditis as you get older then the increased risk we think we have observed might be nothing to do with the Goo at all.

In physics, or at least the bit of physics I’ve been fascinated by, we would call this “confounder” a hidden variable. Myocarditis is correlated with age, Goo status is correlated with age - so the correlation between G and M is coming about because they are each separately correlated with the hidden variable A (age).

The correlation between G and M we initially saw in our data does exist, but it is not indicating causality, in this instance.

This is why so many data analysts and scientists have been calling for the release of the full vaccine data. In terms of our example above, all we’ve been given is the table for G and M - but we need the table to include G, M and A.

I know many of you reading will know all this stuff, but there will also be some for whom this might all be a bit fuzzy - so I hope that (a) you can forgive me and (b) that I’ve been able to de-fuzz things a bit.

But what has any of this got to do with QM?

I want to write about this in more detail at some point in the future, but in quantum mechanics there are experiments one can do in which you take 2 particles (electrons, atoms, etc) which share some common origin. You send one particle off to New York and the other off to Miami. The receiver in New York makes a measurement, and the receiver in Miami makes a measurement. What you find is that there is a correlation between the separated measurements.

This is not remarkable in itself - the particles have a common origin (they initially are correlated in some way). It would be like slicing a coin edgeways and sending the slices off - the person in New York might get the ‘head’ slice and they would instantly know the person in Miami got the ‘tail’ piece.

But what we find in QM is that if we try to measure various different things (in coin terms we might have head or tail, rough or smooth, silver or gold, etc) the correlation exists between the different things we measure. It’s difficult to explain what I mean succinctly here, so we’re just going to accept that explaining this correlation is not straightforward.

The idea emerged that all these observed correlations were the result of a set of hidden variables - that if only we knew them, we would be able to predict the results of these experiments and explain the correlations. It was really a statement that quantum mechanics, as we knew it, was really not strange - all the strangeness was coming about because we weren’t looking under the hood (or able to look) and seeing the engine.

In an extraordinary piece of work, John Bell was able to show that any theory that used the notion of hidden variables such that
(a) no influence between these variables could propagate faster than light and
(b) the variables had values that existed independent of measurement
would not be able to properly predict the results of certain experiments.

So if we assume that things actually have some definite properties (things like position, or speed etc) that exist in a tangible sense before measurement, and we assume that something in New York can’t affect things in Miami any faster than it would take light to make the trip, then we predict the wrong results.

It means that if we accept that measurements made there can’t affect a measurement here (at least not before the result has had chance to travel) then we’re left with the extraordinary conclusion that at a quantum level stuff does not have definite properties.

In somewhat poetic terms it’s like what we perceive as reality is an emergent property from an ocean of mere possibilities.

We don’t notice what must be a nightmarish vista of seething potential realities because when we have lots of quantum particles interacting this vagueness all gets rapidly squashed into a definite place - but at a fundamental level the building blocks of our universe behave in a very weird way. It’s a world where the fabric of matter does not have definite properties.

It’s not that we simply do not know those properties - it’s that they cannot have these definite properties - if we assume they do and try to construct a theory based on that assumption, then our theory will fail.

For all the mundanity and mendacity of our governments and health officials it’s as well to occasionally remind ourselves that the world is far from mundane - there are profound mysteries and limits to what we can know.

---