Flaws in Holmes Meta-Analysis - A Problem of Perspective
“Despite associations between CYP2C19 genotype, clopidogrel metabolism, and platelet aggregation, this systematic review and meta-analysis does not demonstrate a clinically important association of genotype with cardiovascular outcomes…”
Not all research supports CYP2C19 genotyping and with the FDA’s recent recommendations, timely scrutiny and repartee is key to progress. A new meta-analysis by Dr. Holmes and colleagues of the University College London argues that, with relation to cardiovascular events, CYP2C19 genotyping is not as useful as it seems. The researchers analyzed data from more than thirty studies and concluded that most of the previous support was actually the result of small-study bias and that, with the possible exception of stent-thrombosis, no significant relationship could be found between CYP2C19 genotypes and cardiovascular events.
Should Holmes and colleagues be correct, it would suggest that the FDA’s decision may be misguided. However, other researchers have pointed out what may be a few key flaws of their work.
Dr.’s Siasos, Tousoulis, and Stefanadis of the University of Athen’s medical school call attention to the meta-analysis' own conclusions regarding stent thrombosis. Holmes and colleagues seem dismissive of its importance, even though stent thrombosis can be a major clinical problem.
Furthermore, Dr.’s Schuldiner, Vesely, and Fisch of the University of Maryland point out the meta-analysis methods may be flawed as well. “With regard to small-study bias, sample size is not a determinant of effect size, only the variance around the estimate,” Schuldiner, Vesely, and Fisch point out. But perhaps the single biggest problem that they find is that Holmes and colleagues did not distinguish between loss-of-function homozygotes and heterozygotes. Holmes defends this decision by referencing other published studies, but even so, this disregards the fact that different genotypes may have drastically different reactions. A person with a double deletion is going to react much more strongly than one with two intermediate alleles and including them both in a single data point may hide strong, clinically important associations.
It seems that Holmes and colleagues are, in essence, seeing the forest but not the trees. After all, the focus of pharmacogenetic testing is individualized reactions, not just major trends. Nevertheless, Holmes’ study is a valuable foil for defenders of CYP2C19 genotyping, but, as Shuldiner and colleagues put it, until there is a consensus: “clinicians must evaluate the full body of evidence… other directives may be denying patients access to potentially life-saving individualized alternative therapies.”