"It seems that for success in science or art, a dash of autism is essential." ~ Dr. Hans Asperger

"The Way of a Warrior is to establish harmony." ~ Morihei Ueshiba O'Sensei, Founder of Aikido

Wednesday, May 1, 2013

Pseudologic of Genetic Predictions

An acquaintance of mine just wrote an interesting piece titled It’s Time to Stop Pouring Research Money Down the Genetic Sinkhole which alludes to an undeserved faith people have in the predictive power of genetics.
Although autism is becoming more widely understood as a neural difference, not a “psychiatric disorder,” it is one of the profiles that is subject to this intense witch hunt. In his article [Five Decades of Gene Finding Failures in Psychiatry], [Jay Joseph] mentions autism in passing, as an example of one of the many areas where genetic research has been a failure.
But the problem is larger than that. Science understands (if one can even be so generous as to use that word) a very tiny fraction of the human genome. Most research to date has focused on the 1% or so of DNA that codes for proteins (i.e. has the genetic instructions for making the proteins that constitute the human body). The other 99% remains pretty much a complete mystery, although progress is starting to be made in analyzing that. The ENCODE project (ENCyclopedia Of DNA Elements) is still very much in its infancy.
Conditions (defined by behavioral observation) may correlate with certain genetic markers, but if we only observe the genetic markers how well does it predict these other conditions?
The fact is science is not a perfectly deterministic system.  Randomness and uncertainty affect all observations/measurements.  As such, we are stuck dealing with conditional probabilities.  Trying to exactly equate these logical propositions is inherently flawed and likely to fall victim to the Base Rate Fallacy.

(found this on Facebook)

Basically you are looking at the overlap of the events, e.g. Pr(moon & chicken) or Pr(autism etc. & genetics), and the proportion of that joint probability to that of the single event you condition upon; whatever serves as the predictor.

Few people have been to the moon so that is a rare event that greatly overlaps (or is encompassed by) chicken-eating.  Flipping it around (via Bayes' Theorem) we see a ton of people have eaten chicken but only a tiny few have ever been to the moon, therefore chicken-eating is not a good predictor.

For those who are more mathematically inclined:
Pr(A|B) "probability of observing A, given B has occurred or is known"

Genetic researches must assess the prevalence of these so-called markers in the general population.  Otherwise the strength/validity of the observed associations is highly suspect.

Taken to an extreme, we see the dystopian future depicted in the film GATTACA (one of my favorites).  Everything is the product of extreme eugenics engineering.  Anyone conceived in the old-fashioned way is an "Invalid."  I find this scene very poignant where this detective is speaking to the director about exceeding potential:

(At 1:25)
"No one exceeds his potential"
"If he did?"
"It would simply mean we did not accurately gauge his potential in the first place."
This big admission of their methods' fallibility really struck me.  You can never absolutely gauge potential. It's impossible. Measurement is an inherently flawed process. In my line of study, we try to design measurements/tests that are reliable but we have to account for the uncertainty from random error. We try to increase reliability by minimizing error but no measurement is perfect as that would require infinite precision.  If someone's job or life is at stake, you better have some strong evidence. The people in Gattaca were overconfident in the scope of their measurements and ignored the inherent limits of the information.

Even Edward Thorndike, one of the pioneers of psychological measurement, acknowledged that such tests are imperfect and were never meant to be an absolute prediction of your station in life (unlike his contemporary Lewis Terman):
"On the other hand to assume that we have measured some general power which resides in him and determines his ability in every variety of intellectual task in its entirety is to fly directly in the face of all that is known about the organization of intellect." (p. 126)
Thorndike, Edward L. (1921). Intelligence and its measurement: A symposium--I. Journal of Educational Psychology, 12(3), 124-127. [pdf]

No comments:

Post a Comment