Since I'm fresh out of rehab (I'm addicted to placebos), I decided to set my sights on the drug topiramate (Topamax), where in which certain peeps (i.e., employees of Jassen) are trying to get approval for the treatment of alcohol dependence (1).
Originally, I was going to list each researcher attached to this study (there are 14), and expose the ones that are paid employees of Janssen, and blah, blah, blah..., but then I read the study, and I decided that it was not necessary to that. The study is total crap.
How do I know the study is total crap? The short answer is that it was published in JAMA.
Here's my long answer:
The introduction begins with with this, "Hypothetically, topiramate, a sulfamate-substituted fructopyranose derivative, can decrease alcohol reinforcement and the propensity to drink."
"What the hell is a 'sulfamate-substituted fructopyranose derivative?'" Fuck if I know.
What I do know is this, it makes you forget that the article started with the word "hypothetically." See if you can detect the difference in meaning between these two sentences:
Topiramate...can decrease alcohol reinforcement and the propensity to drink, and
Hypothetically, topiramate...can decrease alcohol reinforcement and the propensity to drink.
The first sentence states what the drug is able to do, while the second sentence states what the drug may be able to do in the make-believe world of Candyland.
The study enrolled 371 men and women diagnosed with alcohol dependence according the DSM-IV-TR-LMNOP-XXX-666. For those of you who forgot, the diagnosis of alcohol dependence requires >/= 3 of the following criteria: tolerance, withdrawal, use of more alcohol than intended (this criterion is stupid by the way), unsuccessful effort to cut down on alcohol use, spending much time obtaining, using, or recovering from the effects of alcohol, reduction in social, occupational, or recreational activities, and continued use despite the presence of persistent physical or psychological problems.
But that's not all folks, this study has some more inclusion criteria for these people to meet: men had to consume 35 or more drinks per week (28 or more for women). That means these were heavy drinking people with alcohol dependence. (Important Note: The majority of people with alcohol dependence are NOT heavy drinkers, AKA binge drinking). The researchers also included 16 exclusion criteria (to read more on the effects of this, see 2, 3). Bottom line: This sample is not representative of people with alcohol dependence or those we see in clinical practice (not even in the ball park).
The primary outcome measure was a reduction in the "percentage of heavy drinking days." Do you remember the DSM criteria for alcohol dependence that I listed? You'll notice that the amount of alcohol consumed is not a criterion. Unlike with cigarettes, we don't have that nice curve to demonstrate that reduced alcohol consumption is related to reduced morbidity and mortality.
"Surely we can assume that though, right?"
No. Sorry.
Here's the bottom line: The primary result of this study is NOT IMPORTANT. It has nothing to do with alcohol dependence. If something reduces the amount of days that a person drinks 5 or more drinks in a day, does that mean the person no longer meets the criteria for alcohol dependence? To quote former Alaskan senator Ted Stevens, "NO!"
What other DSM condition is there, that this is considered an acceptable practice? What if the condition were depression? And what if we gave depressed people a pill that made them spend less time in bed? Can I conclude that this drug is an effective treatment for depression? "NO!" So why can these douche bags make the same conclusion about topiramate?
Here's another problem: If a participant tested positive for opiates, cocaine, amphetamines, antidepressants, propoxyphenes, and barbiturates at the time of randomization and before the beginning of the double-blind period, he or she was excluded from the study. If they tested positive for benzodiazepines in the week prior to randomization, they could be enrolled if they had a negative test 7 days later. After that, they could resume consumption of benzos without the knowledge of the experimenters (addicts can be sneaky that way). Hypothetically, participants could have increased their benzo consumption while they reduced their alcohol consumption (both are sedatives).
WARNING: STATISTICS 101
The results obtained from the useless primary measure are this: from baseline to week 14, there was a change from a mean of 81.91% [SD 20.04%] to 43.81% [SD 40.43%] for topiramate, and a change from a mean of 81.97% [SD 19.92%] to 51.76% [SD 37.43%] for placebo. The mean difference between the two was 8.44%.
Pop quiz: What's wrong with these numbers?
Answer: The standard deviations are big.
"What does that mean?"
This means (no pun intended) that the individual data points don't cluster around the mean (i.e., average). They are all over the place. Instead of creating a nice bell curve, they create a flat mound. A large standard deviation means that the data are volatile and hard to analyze. You cannot make good predictions from this kind of data.
Pop quiz: Instead of providing the mean difference (8.44%), which is meaningless, what number should the authors have provided?
Answer: the effect size.
The effect size will tell you whether the drug had a small, moderate, or large impact (i.e., does it actually do something). Luckily, I know how to calculate the effect size. It comes out to be 0.2. (actually 0.204061) That's small folks. Even antidepressants work better than that.
For any psychiatrists who might be reading, here's a problem for you folks. The average dose of topiramate of 171.4 mg (SD 107.6). Again, we have a huge SD. So, what is the optimal dose for these binge drinkers? You don't know? Well, neither do the researchers.
Here's the dirty secret with using significance tests in drug research: The populations studied are not selected randomly. Significance tests are designed to assess only sampling error (i.e., errors due to random sampling). There are no significance tests (strictly speaking) appropriate for testing differences when nonrandom samples are used. Results should always be viewed as tenuous. However, since these tests are all we have, we pretend that significant results actually mean something. It doesn't matter what you call it, psychiatry, psychology, we're all frauds.
Trivia: Many patients call Topamax, "Dopamax."Here's why - side effects that affected more than 10% of subjects in this study include: paresthesia (numbness or tingling, 51%), taste perversion (23%), anorexia (20%), pruritus (10%), difficulty with memory (13%), and difficulty with concentration and attention (15%).
Here's a nit-pick: the study ended after 14 weeks with no follow-up. Alcoholism tends to be a life-long condition for most people.
When you take 14 doctors (i.e., the researchers of the article), and you put their great minds together, you get these pearls of stupidity:
"Plausibly, individuals with certain subtypes of alcoholism might benefit the most from treatment with topiramate." There is not one bit of information in the article to support this claim.
"Our study had 3 limitations." Bullshit.
"Our finding in this study that topiramate is a safe and consistently efficacious medication for treating alcohol dependence is scientifically and clinically important." Horse shit.
I'm assigning homework to you people. This study was published less than two-years ago. It has been cited in other research papers 31 times (4). How many of those studies come close to providing a critique like this one? If this is how the peer-review process works, then my peers are morons.
Johnson, B., Rosenthal, N., Capece, J., Wiegand, F., Mao, L., Beyers, K., McKay, A., Ait-Daoud, N., Anton, R., Ciraulo, D., Kranzler, H., Mann, K., O'Malley, S., Swift, R., & , . (2007). Topiramate for Treating Alcohol Dependence: A Randomized Controlled Trial JAMA: The Journal of the American Medical Association, 298 (14), 1641-1651 DOI: 10.1001/jama.298.14.1641
Monday, March 23, 2009
Subscribe to:
Post Comments (Atom)
6 comments:
Well, indeed.
Although one point, I think "days spent in bed" or something like that would actually be a pretty good outcome measure in depression. In the sense that it's actually an objective measure - it's not just a subjective impression.
Put it this way: if an antidepressant reduced scores on the Hamilton scale but didn't decrease % of days spent in bed, would you think it was a good antidepressant? I wouldn't. But then I do tend to have a grudge against depression rating scales.
Here's my homework:
http://somatopsychic.blogspot.com/2009/03/homework-from-macguffin.html
Alcohol kills. There is not any current treatment that may cure alcoholism. The challenge is even greater before abstinence is achieved and this may require hospital admission. If a drug may just allow to reduce the number of alcohol units per day, this would be a great achievement.
In clinic, I've seen firsthand plenty of alcoholics which had good results with topiramate. Others, not so much, and there is still relapsing. Ignoring the study, which admittedly is flawed, the drug is still very helpful for some people. Of course, I'd love to see a better study done, so that we have better evidence, and more acceptance in the clinics. The bottom line is that it works for you or it doesn't. No amount of statistics will answer that question.
Lies, Damn Lies, and Statistics..... Unless one looks closely at the numbers it's hard to tell the difference.
The JAMA article is quite an interesting read.
People affected by alcoholism should investigate closely all avenues for treatment. Note, even the socially sanctioned AA admits to a very low success rate. Keep looking and don't give up hope.
This study doesn't prove that topiramate cures alcoholism but it shows that is has a positive effect, regardless of the flaws. You strike me as someone who has never written a grant. Unless you have an infinite source of money you'll never get a perfect study (that includes indefinite follow-up, large sample size to reduce the SD, etc).
This study shows that there is merit to a better funded study that is more convincing and will make it easier for someone else to write a grant for a better study.
It seems like you're creating a straw man (topiramate cures alcoholism) when the researchers are arguing something completely different.
Post a Comment