Evaluating "clinical study" claims, focus on Hydroxycut(TM)

[Cloud9Fitness]

A few months ago I noticed that a large number of weight loss supplements were claiming to have an increasing number of “clinical studies” which proved their efficacy. When I would go online and surf the blogs that criticized these clinical studies I was disappointed to find no concrete evaluations.

For every positive blog, I would find one with a negative critique. Unfortunately, the critiques focused on the doctors or actors in the commercial, and not on the supplement. This is especially unfortunate since personal trainers and consumers often browse blogs and forums for continuing education.

As a physician, I have developed a set of critical evaluation questions that allow all personal trainers and consumers to easily get to the truth behind supplements. You can use these questions whether you are reading a blog entry (like this one), a magazine article, or a scientific journal.

Question 1: Was the clinical study conducted on individuals that resemble my clients or myself?
If you are a 25 year old bodybuilder, then do you want information derived from a study of geriatric patients?

Question 2: Are the groups being compared head-to-head?
How do you know if something is more effective, unless it is tested directly against the competition or a placebo?

Question 3: Are the groups equal at the beginning of the comparison?
Many studies have unequal groups at the beginning of a study. This sneaky technique is utilized in some weight-loss studies to inflate results. You’ll see an example soon.

Questions 4 and 5: What is the primary outcome of the study, and does that outcome matter?
If your goal is to lose weight, then you want to see weight loss in humans, not increased 24-hour metabolism in a rat. Rat studies are done preliminarily to test safety, test dosing, and to build theories. They do not provide clinical proof for humans, and are too often misinterpreted that way. 24-hour metabolism studies may provide optimism for further research, but do not equate to long-term weight loss.

Question 6: Does the result found have any clinical significance?
First, the study has to tell you if there was a statistically significant difference in the results. This means that there is less than a 5% chance that you are accepting a false positive result (when using 95% significance level, p<0.05 or 95% CI). Second, you need to see if that difference matters. For example, if everyone in group A lost 1 pound, and everyone in group B lost 2 pounds, then there is a mathematical/statistical difference. However, the clinical difference between 1 and 2 pounds is miniscule. Imagine taking an expensive supplement that said, “Loose double the weight !!!!!” but the only difference was 1 pound. I’d be mad!

So let’s apply these questions to the clinical trials used to support Hydroxycut™. The commercials indicate “the subjects using the patented ingredients in Hydroxycut™ lost 4.5 times then from diet and exercise alone.” Let’s see for ourselves.

Study: Preuss et al. “Effects of a natural extract of (-)-hydroxycitric acid (HCA-SX) and a combination of HCA-SX plus niacin-bound chromium and Gymnema sylvestre extract on weight loss.”Diabetes Obes Metab. 2004 May;6(3):171-80.

Answer to question 1: Was the clinical study conducted on individuals the resemble my clients or myself?
There were 60 individuals (sex not mentioned) 21-60 years. BMI >26kg/m2 (29.9-55.5kg/m2)

Answer to question 2: Are the groups being compared head-to-head?
Three groups were tested head to head in something called a double blind randomized control trial.
Group 1 - 4.66 gm of HCA-SX (2.4 - 2.8gm active HCA)
Group 2 - 4.66gm HCA-SX, 4mg Chromium polynicotinate, and 400mg Gymnema sylvestre extract (These are the Hydroxycut™ active ingredients)
Group 3 - Placebo
All groups performed 30 minutes of walking 5 days/week and ate a 2000 kcal/day diet

Answer to questions 4 and 5: What is the primary outcome of the study, and does that outcome matter?
Weight loss, % Body fat, and fat mass loss over 8 weeks are the primary outcomes. These are important results for the general public.

Answer to question 6: Does the result found have any clinical significance?
Groups 1 and 2 with greater average (mean) weight loss than Group 3
Group 1 - 4.53kg Group 2 - 5.69kg Group 3 - 1.60kg
These appear to be clinically significant differences too.

But wait one second… I haven’t answered question 3 yet!

Answer to question 3: Are the groups equal at the beginning of the comparison?
The placebo group had an initial body weight of 80.44 kg (+/-2.36) and the Hydroxycut™ ingredient group had an initial body weight of 92.41 kg (+/- 3.84). The study does not explicitly say that the groups are statistically equal, nor does it appear that way when the mean difference in initial weight is almost a 15% weight difference. Why does this matter? Well, everyone was restricted to 2000 Calories. If one group is heavier, then a 2000 Calorie diet is a larger proportional change in diet in the heaviest group. So it would no surprise to me if they lost the most weight.

Another limitation also exists that may be beyond the scope of this blog. Quickly, the tolerances (+/-) were marked as standard errors, but the study did not provide the number of subjects in each group. That number is needed for a reader to determine a term known as the standard deviation. If I had this number I could have seen how mathematically different the initial body weight was.

Again, these questions are how I quickly approach any resource to judge the quality of the information inside. Feel encouraged to try them for yourself, or leave your thoughts/ideas.

[cycomiko]

Imagine taking an expensive supplement that said, “Loose double the weight !!!!!” but the only difference was 1 pound. I’d be mad!


They are obviously talking about making the weight loose, double the wobbles

[OldGuy]

Question 1: Who paid for the study?

[Cloud9Fitness]

Originally Posted by OldGuy

Question 1: Who paid for the study?

This study was conducted by the scientists that generated the formula, and the patent was funded by InterHealth Nutraceuticals. We have an electronic version of the study, and are unable to find any notification of the grant funding. It is probably published in the journal, but the actual journal it is published in is obscure and unavailable in our medical library.

[OldGuy]

I figured as such. In this case I wouldn't even waste time trying to answer your other questions.

[Cloud9Fitness]

Originally Posted by OldGuy

I figured as such. In this case I wouldn't even waste time trying to answer your other questions.

Interestingly, these are studies quoted on the TV commercials. Now we know!

[OldGuy]

And I'm sure they are thrilled over this news.