Since I'm fresh out of rehab (I'm addicted to placebos), I decided to set my sights on the drug topiramate (Topamax), where in which certain peeps (i.e., employees of Jassen) are trying to get approval for the treatment of alcohol dependence (1).
Originally, I was going to list each researcher attached to this study (there are 14), and expose the ones that are paid employees of Janssen, and blah, blah, blah..., but then I read the study, and I decided that it was not necessary to that. The study is total crap.
How do I know the study is total crap? The short answer is that it was published in JAMA.
Here's my long answer:
The introduction begins with with this, "Hypothetically, topiramate, a sulfamate-substituted fructopyranose derivative, can decrease alcohol reinforcement and the propensity to drink."
"What the hell is a 'sulfamate-substituted fructopyranose derivative?'" Fuck if I know.
What I do know is this, it makes you forget that the article started with the word "hypothetically." See if you can detect the difference in meaning between these two sentences:
Topiramate...can decrease alcohol reinforcement and the propensity to drink, and
Hypothetically, topiramate...can decrease alcohol reinforcement and the propensity to drink.
The first sentence states what the drug is able to do, while the second sentence states what the drug may be able to do in the make-believe world of Candyland.
The study enrolled 371 men and women diagnosed with alcohol dependence according the DSM-IV-TR-LMNOP-XXX-666. For those of you who forgot, the diagnosis of alcohol dependence requires >/= 3 of the following criteria: tolerance, withdrawal, use of more alcohol than intended (this criterion is stupid by the way), unsuccessful effort to cut down on alcohol use, spending much time obtaining, using, or recovering from the effects of alcohol, reduction in social, occupational, or recreational activities, and continued use despite the presence of persistent physical or psychological problems.
But that's not all folks, this study has some more inclusion criteria for these people to meet: men had to consume 35 or more drinks per week (28 or more for women). That means these were heavy drinking people with alcohol dependence. (Important Note: The majority of people with alcohol dependence are NOT heavy drinkers, AKA binge drinking). The researchers also included 16 exclusion criteria (to read more on the effects of this, see 2, 3). Bottom line: This sample is not representative of people with alcohol dependence or those we see in clinical practice (not even in the ball park).
The primary outcome measure was a reduction in the "percentage of heavy drinking days." Do you remember the DSM criteria for alcohol dependence that I listed? You'll notice that the amount of alcohol consumed is not a criterion. Unlike with cigarettes, we don't have that nice curve to demonstrate that reduced alcohol consumption is related to reduced morbidity and mortality.
"Surely we can assume that though, right?"
Here's the bottom line: The primary result of this study is NOT IMPORTANT. It has nothing to do with alcohol dependence. If something reduces the amount of days that a person drinks 5 or more drinks in a day, does that mean the person no longer meets the criteria for alcohol dependence? To quote former Alaskan senator Ted Stevens, "NO!"
What other DSM condition is there, that this is considered an acceptable practice? What if the condition were depression? And what if we gave depressed people a pill that made them spend less time in bed? Can I conclude that this drug is an effective treatment for depression? "NO!" So why can these douche bags make the same conclusion about topiramate?
Here's another problem: If a participant tested positive for opiates, cocaine, amphetamines, antidepressants, propoxyphenes, and barbiturates at the time of randomization and before the beginning of the double-blind period, he or she was excluded from the study. If they tested positive for benzodiazepines in the week prior to randomization, they could be enrolled if they had a negative test 7 days later. After that, they could resume consumption of benzos without the knowledge of the experimenters (addicts can be sneaky that way). Hypothetically, participants could have increased their benzo consumption while they reduced their alcohol consumption (both are sedatives).
WARNING: STATISTICS 101
The results obtained from the useless primary measure are this: from baseline to week 14, there was a change from a mean of 81.91% [SD 20.04%] to 43.81% [SD 40.43%] for topiramate, and a change from a mean of 81.97% [SD 19.92%] to 51.76% [SD 37.43%] for placebo. The mean difference between the two was 8.44%.
Pop quiz: What's wrong with these numbers?
Answer: The standard deviations are big.
"What does that mean?"
This means (no pun intended) that the individual data points don't cluster around the mean (i.e., average). They are all over the place. Instead of creating a nice bell curve, they create a flat mound. A large standard deviation means that the data are volatile and hard to analyze. You cannot make good predictions from this kind of data.
Pop quiz: Instead of providing the mean difference (8.44%), which is meaningless, what number should the authors have provided?
Answer: the effect size.
The effect size will tell you whether the drug had a small, moderate, or large impact (i.e., does it actually do something). Luckily, I know how to calculate the effect size. It comes out to be 0.2. (actually 0.204061) That's small folks. Even antidepressants work better than that.
For any psychiatrists who might be reading, here's a problem for you folks. The average dose of topiramate of 171.4 mg (SD 107.6). Again, we have a huge SD. So, what is the optimal dose for these binge drinkers? You don't know? Well, neither do the researchers.
Here's the dirty secret with using significance tests in drug research: The populations studied are not selected randomly. Significance tests are designed to assess only sampling error (i.e., errors due to random sampling). There are no significance tests (strictly speaking) appropriate for testing differences when nonrandom samples are used. Results should always be viewed as tenuous. However, since these tests are all we have, we pretend that significant results actually mean something. It doesn't matter what you call it, psychiatry, psychology, we're all frauds.
Trivia: Many patients call Topamax, "Dopamax."Here's why - side effects that affected more than 10% of subjects in this study include: paresthesia (numbness or tingling, 51%), taste perversion (23%), anorexia (20%), pruritus (10%), difficulty with memory (13%), and difficulty with concentration and attention (15%).
Here's a nit-pick: the study ended after 14 weeks with no follow-up. Alcoholism tends to be a life-long condition for most people.
When you take 14 doctors (i.e., the researchers of the article), and you put their great minds together, you get these pearls of stupidity:
"Plausibly, individuals with certain subtypes of alcoholism might benefit the most from treatment with topiramate." There is not one bit of information in the article to support this claim.
"Our study had 3 limitations." Bullshit.
"Our finding in this study that topiramate is a safe and consistently efficacious medication for treating alcohol dependence is scientifically and clinically important." Horse shit.
I'm assigning homework to you people. This study was published less than two-years ago. It has been cited in other research papers 31 times (4). How many of those studies come close to providing a critique like this one? If this is how the peer-review process works, then my peers are morons.
Johnson, B., Rosenthal, N., Capece, J., Wiegand, F., Mao, L., Beyers, K., McKay, A., Ait-Daoud, N., Anton, R., Ciraulo, D., Kranzler, H., Mann, K., O'Malley, S., Swift, R., & , . (2007). Topiramate for Treating Alcohol Dependence: A Randomized Controlled Trial JAMA: The Journal of the American Medical Association, 298 (14), 1641-1651 DOI: 10.1001/jama.298.14.1641