In my experience, when the smug and dim emit the "correlation is not causation" flatus, they are claiming that because two things are correlated there can be *no* causal effect. In other words, they are claiming that correlation *denies* causation. It's a complete inversion of reality because Fuck Me Dead, very often when you see correlation, there quite often *is* causation involved.
"Correlation is not Causation", when meant honestly should always be stated as "Correlation is *not necessarily* Causation, but jeez it could be zad quite often is".
And after that little rant, thanks again for another fine exposition. I studied Pure Mathematics and Physics including elementat QM (amongst other things) at the Australian National University back in the early 1980s when courses were actually rigorous, and your lessons always stir a lot of old memories, mostly fond (beer) , sometimes not so much (exam halls).😊
Yup - it's another of those magic incantations that get used to summarily dismiss something.
Just before I left the uni where I worked we were designing a new physics degree (degrees had to be accredited by the govt and 'pure' physics hadn't been one of those on offer - so the physics department was largely a "service" department for the engineering students).
I still have my old university notes back from when I did my first degree. The difference between what I did and the degree proposal we were working on was astonishing. And the proposal was crafted to be in line with the physics programs of 'good' institutions around the world.
Just looking at the list of courses was an eye-opener. Big, big difference between 'then' and 'now' - and that's before we even considered the content and level of approach.
**These days I get too easily exasperated at stupidity ...
**Being objective about things these days could easily lose you your job.
**Whenever you read any media article which contains the word probability (or phrases like “more likely”) then press the crap out of the big red caution button in your mind.
There does seem to be a correlation between many things, and many either assign it to, or disregard it because of, intuition. I've long thought that intuition is simply having a better sense of pattern recognition. And the patterns are all there in plain sight, but an increasing number these days seem unable to perceive them.
I enjoy writing these introductory more technical pieces - and they give me a nice break from worrying about the Loony Toons that make up a far too high proportion of those who have influence today. The flip side is that they take much longer to produce than an off-the-cuff rant.
I learned this the hard way when it came to my local public school. Terrific ratings, great test scores. Turns out the 95 percent of kids who are doing well are all at Russian School of Math twice a week, Kumon the other three, and reading vociferously on weekends. The coin is weighted. It’s like the Covid vaccine protecting against traffic fatalities.
And this is absolutely a mistake most modern [anti] racists make with skin color. Because it is the primary thing they notice, they assume causation. All other things— whether the person is having a violent episode or is on drugs or is otherwise presenting themselves or behaving threateningly— are ignored. And moreover, they don’t notice any incidents or parts of incidents don’t fit the narrative. It is pure superstition, like removing the white dice from a game of Yahtzee and substituting them with purple because you saw someone win once with purple dice, conveniently forgetting that others won with white dice during the majority of Yahtzee games ever played.
It reminds me of the old scenario when some black guy fails a job interview
"Is it coz I is black?" says the rejected applicant
"No, it's because you're a dick" says the interviewer.
But black people seem to have been elevated into a position where they can do no wrong - and any such claim of wrongness is automatically taken to be "racist".
We're all human, and humans are allowed to be dicks from time to me - it might be what defines us best.
All of this modern "science" which ascribes any apparent difference based on the single variable of skin colour as being evidence of "racism" is so far up the correlation IS causation rabbit hole that it's near the Earth's core and their brains are being melted.
More practically (and less mathematically) speaking, there are Bradford Hill's criteria for distinguishing between correlation and causation: https://en.wikipedia.org/wiki/Bradford_Hill_criteria. Strength of effect (correlation) is just one of the nine criteria used to wiggle out root causation among co-varying factors.
You didn't pay much attention to SPURIOUS correlation - the fact that NON-STATIONARY* variables will exhibit correlation simply by dint of them having a non-constant mean (or variance).
In quantitative discussions, I stop listening the moment anyone talks about observed correlations in non-I(0) variables... because it indicates a fundamental hole in their understanding of correlation.
Ignorance of the Gauss-Markov conditions causes first year Econometrics students to fail - and so it should. If a student doesn't understand what makes OLS "BLUE" by the end of first year, they have no place in the discipline. Correlation (as calculated by ρ) ASSUMES* a linear relationship.
STATIONARITY* - constant mean and finite variance of the variables - is one of the GM conditions.
* Because Substack's comment mechanism is absolutely SHIT*, it is necessary to resort to ALL-CAPS* to do emphasis.
Protip: if they can't organise em, b and i tags securely, they have SKIMPED ON DATASEC AS WELL* - and therefore nobody should entrust them with credit card numbers.
No I didn't. I wanted to keep things pretty basic.
And to be fair, I'm only partially acquainted with non-stationary distributions, never having had to use them in my work (all Markov and no coloured noise). I'm aware that some of the commonly used technical measures of correlation (such as the Pearson correlation coefficient) assume a linear relationship and that this can lead to problems with distributions that don't have the right properties for its use. But I'm not even close to being "expert" enough to discuss any of the differences between stationary and non-stationary situations with any degree of simplicity.
To make your example about police even more of a hassle, police are not identical so you'd need some way to account for that too, to say nothing of the different circumstances of a police working a remote rural area vs an inner city one.
(Which you of course touched upon but I felt it needed pointing out in crayon so to speak, since most of us 'umies regardless of L44t math-skills instinctively go for "all else being equal" as soon as something is emotional, fuzzy and inflammable.)
To paraphrase myself from a debate at university some 20-ahem years ago:
"Of course the police in Stockholm's council housing areas will arrest a high percentage of non-swedes! [Name of area] has a swedish population under 15%!"
The debate went on with others pointing out that the majority of victims of said arrestees were also migrants, and one professor made the comparison with the inner region of Northland: "You'd certainly expect the percentage of cases of arab moslems arrested for DUI while driving a snowmobile to be zero or close to it. Do you understand why?" That last adressed to a very voiciferous girl wearing a green parka laden with various political buttons, a "palestine shawl", DMs, tattered blue-jeans, and white dreadlocks plus facial piercings.
Of course, none of those elements have any causation on her opinions and beliefs - or vice versa - or each other, but they sure do have correlation together.
Speaking of autism and vaccines, what hurt the issue the most is three things: celebrities (a Beatle-wife among them if I remember correctly) claiming out of the blue that it is so in the 1990s, a researcher producing false results (don't remember his name) and the corporate media machine spinning this ever this. While I'm no biologist or doctor, surely the fact that Thiomersal (the Mercury isotope-containing preservative used up until quite recently) is used as a preservative for tattoo-inks and in much greater amounts and higher concentrations would be worthwhile to explore?
Personally, as real autism as opposed to SBaLP-syndrome* has a very complex genetic background, I'm leaning towards it being down to combinatorial and cumulative environmental factors. The probability of the genetic materials involved producing the type of brain dysfunctions and retardation typical for autism have increased from one generation to the next in an exponential fashion.
If so, comparative test-series among migrant populations vs indigenous ones could be useful. F.e. comparing diagnosed and fully jabbed british kids where both parents and grandparents are CHAVs or close to it, with 1st generation pakis or somalis and 4th generation migrant kids (social/cultural nativisation isn't a factor here) - and I think we all know (i.e. bellyfeel) that we wouldn't find the same percentages?
I like that comment about the use of "all things being equal" - and, as you say, they very rarely are.
I also loved the image of some frothed up Islamic nutjob arrested for being drunk in charge of a snowmobile 🤣
As for the autism stuff - it's worrying. I read an article a couple of weeks ago from someone who's kid is autistic and she's gone into the research and statistics in some detail. She didn't dispute the alarming rise in autism, and she also made the point that we're not talking about 'mild' cases either. Many of these kids will need serious help throughout their lives.
At the moment I can't see anything but some external factor (like vaccines, or the accumulation of toxins) as being responsible.
What is very telling is the massive unified opposition among researchers, decision makers and pharmacological corporations and scientist against investigating the issue.
What if it's this simple:
Natural prevalence of autism in humans = 1/50 000 births.
But: every shot of vaccine, be it with Thiomersal or aluminum as adjuvant, increases the risk, cumulative over generations.
So: first vaccinations in the 1950s. Those kids then have kids in the late 1960s and in the seventies. Then those people have their own kids in the 1990s.
Is it that improbable that every generation has an increased risk, so that the real prevalence now is 1/1 000 in western nations?
That's why I'd want a study taking the potential generational effect into account. We could also use unvaccinated parents vs vaccinated ones, possibly over generations even.
However, the interest is flat zero. It's too ingrained in us that "vaccines equals doubleplusgood" and it's too lucrative a business for any politician to order an investigation proper.
Remember, it took until 2013 until the german manufacturer of Neuroseyn admitted to having committed fraud and apologised. Over 50 years.
In my experience, when the smug and dim emit the "correlation is not causation" flatus, they are claiming that because two things are correlated there can be *no* causal effect. In other words, they are claiming that correlation *denies* causation. It's a complete inversion of reality because Fuck Me Dead, very often when you see correlation, there quite often *is* causation involved.
"Correlation is not Causation", when meant honestly should always be stated as "Correlation is *not necessarily* Causation, but jeez it could be zad quite often is".
And after that little rant, thanks again for another fine exposition. I studied Pure Mathematics and Physics including elementat QM (amongst other things) at the Australian National University back in the early 1980s when courses were actually rigorous, and your lessons always stir a lot of old memories, mostly fond (beer) , sometimes not so much (exam halls).😊
Yup - it's another of those magic incantations that get used to summarily dismiss something.
Just before I left the uni where I worked we were designing a new physics degree (degrees had to be accredited by the govt and 'pure' physics hadn't been one of those on offer - so the physics department was largely a "service" department for the engineering students).
I still have my old university notes back from when I did my first degree. The difference between what I did and the degree proposal we were working on was astonishing. And the proposal was crafted to be in line with the physics programs of 'good' institutions around the world.
Just looking at the list of courses was an eye-opener. Big, big difference between 'then' and 'now' - and that's before we even considered the content and level of approach.
"correlation *denies* causation"
Exactly. This needs to be a t-shirt or on a coffee-mug or something.
I'd have loved to have that slogan on a t-shirt when I was a student, just to stir things up a bit.
I do enjoy these mini courses.
The quote notes I copy and pasted;
**These days I get too easily exasperated at stupidity ...
**Being objective about things these days could easily lose you your job.
**Whenever you read any media article which contains the word probability (or phrases like “more likely”) then press the crap out of the big red caution button in your mind.
There does seem to be a correlation between many things, and many either assign it to, or disregard it because of, intuition. I've long thought that intuition is simply having a better sense of pattern recognition. And the patterns are all there in plain sight, but an increasing number these days seem unable to perceive them.
Thanks John
I enjoy writing these introductory more technical pieces - and they give me a nice break from worrying about the Loony Toons that make up a far too high proportion of those who have influence today. The flip side is that they take much longer to produce than an off-the-cuff rant.
I learned this the hard way when it came to my local public school. Terrific ratings, great test scores. Turns out the 95 percent of kids who are doing well are all at Russian School of Math twice a week, Kumon the other three, and reading vociferously on weekends. The coin is weighted. It’s like the Covid vaccine protecting against traffic fatalities.
And this is absolutely a mistake most modern [anti] racists make with skin color. Because it is the primary thing they notice, they assume causation. All other things— whether the person is having a violent episode or is on drugs or is otherwise presenting themselves or behaving threateningly— are ignored. And moreover, they don’t notice any incidents or parts of incidents don’t fit the narrative. It is pure superstition, like removing the white dice from a game of Yahtzee and substituting them with purple because you saw someone win once with purple dice, conveniently forgetting that others won with white dice during the majority of Yahtzee games ever played.
It reminds me of the old scenario when some black guy fails a job interview
"Is it coz I is black?" says the rejected applicant
"No, it's because you're a dick" says the interviewer.
But black people seem to have been elevated into a position where they can do no wrong - and any such claim of wrongness is automatically taken to be "racist".
We're all human, and humans are allowed to be dicks from time to me - it might be what defines us best.
All of this modern "science" which ascribes any apparent difference based on the single variable of skin colour as being evidence of "racism" is so far up the correlation IS causation rabbit hole that it's near the Earth's core and their brains are being melted.
More practically (and less mathematically) speaking, there are Bradford Hill's criteria for distinguishing between correlation and causation: https://en.wikipedia.org/wiki/Bradford_Hill_criteria. Strength of effect (correlation) is just one of the nine criteria used to wiggle out root causation among co-varying factors.
Yes - I wondered about touching on these, but decided not to. Figuring out 'causation' from time series data is quite challenging.
But isn't everything time series data?
You didn't pay much attention to SPURIOUS correlation - the fact that NON-STATIONARY* variables will exhibit correlation simply by dint of them having a non-constant mean (or variance).
In quantitative discussions, I stop listening the moment anyone talks about observed correlations in non-I(0) variables... because it indicates a fundamental hole in their understanding of correlation.
Ignorance of the Gauss-Markov conditions causes first year Econometrics students to fail - and so it should. If a student doesn't understand what makes OLS "BLUE" by the end of first year, they have no place in the discipline. Correlation (as calculated by ρ) ASSUMES* a linear relationship.
STATIONARITY* - constant mean and finite variance of the variables - is one of the GM conditions.
* Because Substack's comment mechanism is absolutely SHIT*, it is necessary to resort to ALL-CAPS* to do emphasis.
Protip: if they can't organise em, b and i tags securely, they have SKIMPED ON DATASEC AS WELL* - and therefore nobody should entrust them with credit card numbers.
No I didn't. I wanted to keep things pretty basic.
And to be fair, I'm only partially acquainted with non-stationary distributions, never having had to use them in my work (all Markov and no coloured noise). I'm aware that some of the commonly used technical measures of correlation (such as the Pearson correlation coefficient) assume a linear relationship and that this can lead to problems with distributions that don't have the right properties for its use. But I'm not even close to being "expert" enough to discuss any of the differences between stationary and non-stationary situations with any degree of simplicity.
To make your example about police even more of a hassle, police are not identical so you'd need some way to account for that too, to say nothing of the different circumstances of a police working a remote rural area vs an inner city one.
(Which you of course touched upon but I felt it needed pointing out in crayon so to speak, since most of us 'umies regardless of L44t math-skills instinctively go for "all else being equal" as soon as something is emotional, fuzzy and inflammable.)
To paraphrase myself from a debate at university some 20-ahem years ago:
"Of course the police in Stockholm's council housing areas will arrest a high percentage of non-swedes! [Name of area] has a swedish population under 15%!"
The debate went on with others pointing out that the majority of victims of said arrestees were also migrants, and one professor made the comparison with the inner region of Northland: "You'd certainly expect the percentage of cases of arab moslems arrested for DUI while driving a snowmobile to be zero or close to it. Do you understand why?" That last adressed to a very voiciferous girl wearing a green parka laden with various political buttons, a "palestine shawl", DMs, tattered blue-jeans, and white dreadlocks plus facial piercings.
Of course, none of those elements have any causation on her opinions and beliefs - or vice versa - or each other, but they sure do have correlation together.
Speaking of autism and vaccines, what hurt the issue the most is three things: celebrities (a Beatle-wife among them if I remember correctly) claiming out of the blue that it is so in the 1990s, a researcher producing false results (don't remember his name) and the corporate media machine spinning this ever this. While I'm no biologist or doctor, surely the fact that Thiomersal (the Mercury isotope-containing preservative used up until quite recently) is used as a preservative for tattoo-inks and in much greater amounts and higher concentrations would be worthwhile to explore?
Personally, as real autism as opposed to SBaLP-syndrome* has a very complex genetic background, I'm leaning towards it being down to combinatorial and cumulative environmental factors. The probability of the genetic materials involved producing the type of brain dysfunctions and retardation typical for autism have increased from one generation to the next in an exponential fashion.
If so, comparative test-series among migrant populations vs indigenous ones could be useful. F.e. comparing diagnosed and fully jabbed british kids where both parents and grandparents are CHAVs or close to it, with 1st generation pakis or somalis and 4th generation migrant kids (social/cultural nativisation isn't a factor here) - and I think we all know (i.e. bellyfeel) that we wouldn't find the same percentages?
I like that comment about the use of "all things being equal" - and, as you say, they very rarely are.
I also loved the image of some frothed up Islamic nutjob arrested for being drunk in charge of a snowmobile 🤣
As for the autism stuff - it's worrying. I read an article a couple of weeks ago from someone who's kid is autistic and she's gone into the research and statistics in some detail. She didn't dispute the alarming rise in autism, and she also made the point that we're not talking about 'mild' cases either. Many of these kids will need serious help throughout their lives.
At the moment I can't see anything but some external factor (like vaccines, or the accumulation of toxins) as being responsible.
What is very telling is the massive unified opposition among researchers, decision makers and pharmacological corporations and scientist against investigating the issue.
What if it's this simple:
Natural prevalence of autism in humans = 1/50 000 births.
But: every shot of vaccine, be it with Thiomersal or aluminum as adjuvant, increases the risk, cumulative over generations.
So: first vaccinations in the 1950s. Those kids then have kids in the late 1960s and in the seventies. Then those people have their own kids in the 1990s.
Is it that improbable that every generation has an increased risk, so that the real prevalence now is 1/1 000 in western nations?
That's why I'd want a study taking the potential generational effect into account. We could also use unvaccinated parents vs vaccinated ones, possibly over generations even.
However, the interest is flat zero. It's too ingrained in us that "vaccines equals doubleplusgood" and it's too lucrative a business for any politician to order an investigation proper.
Remember, it took until 2013 until the german manufacturer of Neuroseyn admitted to having committed fraud and apologised. Over 50 years.