So onwards towards review. My comments are in () parentheses, and at the end of the summary:
Clinical screen study: Use of Biophotonic Skin Scanner to Assess Skin Carotenoids as a Marker of Anti-oxidant Status
Purpose: To determine whether age, BMI, gender, race, smoking status and use of Lifepak and consumption of fruits and vegetables are related to skin carotenoid readings.
Methods: 1375 employees of Nu Skin (a subsidary of Pharmanex) were recruited for this study. Subjects were asked to fill out a questionnaire and were also scanned. Body fat was assessed with a near-infrared device. (According to Futurex's references, the only validation done on this device is to correlate its reading with fat at the biceps site, with the reasoning that the biceps site is the most indicative of body fat. There have been no studies to validate this method against the gold standard, underwater weighing, or cadaveric testing).
Subjects who agreed to do so, also did urine testing (Pee on a strip and report the colour to the study coordinator. It's supposed to be a measure of free-radical levels. I've been unable to find a source from Vespro Life Sciences on its validity).
Statistics: Correlation coefficients were calculated for the relationship between all the variables and the scanning carotenoid result. Statistical tests were either t-tests or "correlation analysis".
Results: The readings from the scanner are normally distributed in the sampled population. There did not appear to be any significant differences between men and women, or between races in terms of scanning results, with the exception of Asians having a significantly higher mean reading than other races.
Biophotonic scan numbers differed in a proportional manner to the number of fruit/vegetable servings per day. (If you examine the means and standard deviations, it's clear that using means and standard deviations is not the appropriate summary statistic as the standard deviation is quite large and the scan value one would obtain by subtracting two standard deviations from the mean is less than 0. Given that a scan value of less than 0 is impossible -and not observed according to the summary earlier in the paper-, this indicates that the populations were likely of a non-normal distribution and medians would have been a better choice to publish--and may not have exhibited such a proportional pattern. It is also likely that the means reported are driven by a small proportion of individuals in each subgroup, which has implications as to the external validity (generalizability) of these particular results.)
Biophotonic results differed between individuals who consumed less carotenoids than those who consumed more caraotenoids. Those who consumed less than 15mg a day scored lower than those who consumed 15-30mg a day who scored lower than those who consumed more than 30mg a day. (A further look at these numbers shows a massively high standard deviation (as low as 1/3 of the mean value, and up to 1/2 the mean value). The fact that statistical significance was detected can almost certainly be attributed to the high sample size used in this study. The question is not whether the groups are different or not, but whether a mean scan of 16000 is practically different than a mean scan of 20000 on a scale that attain a value of 73000, despite whatever the stats may tell you. A signficant test can alway be achieved with sufficient sample size)
Scanner readings were inversely proportional to the urine test for free-radicals. However, 490 subjects were reported as having "high" levels of free-radicals, while the remaining 72 subjects were distributed with descending numbers amongst the "medium", "low" and "optimal" groups. [Whether this result is "consistent" as the author puts it, is up for grabs]
Smokers scored lower than non-smokers. [Same limitations as stated above]
Scan scores decreased with increasing BMI and body fat. [Same limitations as the huge standard deviation one above]
Scan scores increased with increased Lifepak use [No measure of variance was reported in this section].
Comments:
1. This isn't a peer-reviewed manuscript. It's not even published in a journal. There is no mention of ethics review by an IRB and quite frankly, on the scale of evidence, this would be ranked close to bottom in terms of "strength of evidence" by anyone looking to publish a systematic review on the scanner and Lifepak.
2. The author never made an attempt to verify the speculation that the difference detected between Asians and other races was due to increased carotenoid consumption. It may be that race confounds or interacts with the relationship between race and Biophotonic scanning. This could have been easily performed with an easy regression analysis.
3. Multiple t-testing without detecting significance does not validate a measurement, and certainly does not rule out confounding and interaction relationships. The analysis performed is of insufficient quality to make the statement that age, gender, and race, do not confound or interact with Biophonic scan results.
4. You can always detect a significant difference if you have a high enough sample size.
4.5 (edited in): The outcomes of this study are highly suspect. A non-validated questionnaire about food recall, which may be highly prone to recall bias, a non-validated test of body composition, a potentially non-validated test of free-radical levels in the urine--even if the statistical analysis was good and sound, good and sound analysis on invalid outcomes still results in invalid conclusions.
4.75. I'm not even going to TOUCH the whole "employees of Nu Skin" bit.
5. I don't see how this study of this quality validates the Biophotonic scanner. Given the high probability of making a type II error (detecting a statistically significant difference in a sample when a difference doesn't actually exist in the population), I'm not sure that this study even demonstrates sensitivity and discrimination ability.
|