Was there intention to deceive in the Hall et al. study?
(Versión en español: pinchar aquí)
This study demonstrated that, calorie for calorie, restriction of dietary fat led to greater body fat loss than restriction of dietary carbohydrate in adults with obesity.
It sounds like a relevant study, right? Apparently they show that obese people that restrict dietary fat lose more body fat than those that restrict the same calories from carbohydrates.
Is that so? Have they demonstrated what they say they have?
The study is » Calorie for Calorie, Fat Dietary Restriction Results in More Body Fat Loss than Carbohydrate Restriction in People with Obesity «.
Let’s have a look at the data they give us. In the figure below this line they show the change in body fat measured by DXA (dual-energy X-ray Absorptiometry) for the two diets:
Do you see the markers on the right side of the graph, the orange and blue ones, one overlapping the other? These are the fat mass loss they measured in the study: there was no difference between both diets. You should ignore the straight lines because they are just predictions made using mathematical models (and those models are obviously not that good).
Does that mean that there was no difference between diets? No. It means that that difference was not measured. May be if the study had lasted longer, the ultra-low-fat diet could have produced a greater loss of body fat than the other diet. I can’t think of a reason for that to be impossible. But that is not what they measured.
One small comment. The authors tell us that «One female subject had fat mass changes measured via DXA that were not physiological and were clear outliers. These data were excluded from the analyses «. What? The only direct method they used for measuring body fat was so «reliable» that they removed the data from one participant because it was not possible? And I have another question: why didn’t they repeat the measure rather than discarding the data from the participant? I don’t understand that they don’t give an explanation about what happened here.
Let’s read again the conclusions of the study: «This study demonstrated that, calorie for calorie, restriction of dietary fat led to greater body fat loss than restriction of dietary carbohydrate in adults with obesity.»
But we see that the body fat loss was not different between diets. How can they tell us the opposite? The reason is that as they didn’t get the difference they were looking for, they started computing fat loss indirectly, using equations: they knew the fat intake, approximately, they measured the respiratory quotient and from this ratio and using equations they computed the rate of fat oxidation, approximately. Then they assumed that all the ingested fat that wasn’t oxidized was stored as fat. They assumed that no fat could be lost with the feces or could have been used for other functions in the body. The assumption of the authors is that all that unaccounted fat was stored as body fat. To me, as long as the DXA measurement of body fat gave a result that contradicts this indirect computation, this is not enough for the grandiloquent use of the word «demonstrated».
Let’s read again the conclusions from the study: «This study demonstrated that, calorie for calorie, restriction of dietary fat led to greater body fat loss than restriction of dietary carbohydrate in adults with obesity.»
The authors speak of «body fat loss», «restriction of dietary fat» and «restriction of dietary carbohydrate». Were those the elements analyzed in their study?
«Body fat loss»
We are talking about an experiment in which diets were followed for just 6 days (please check the horizontal axis of the chart above) and the weight loss achieved in 6 days is completely irrelevant. Any diet, even starvation, can make you lose body fat and body weight in the short term. But that result can’t be used when talking about significant weight loss, i.e. to lose a significant amount of body weight in the long term. The interesting data comes from years, not from less than a week.
They should at least have talked about «body fat loss in the extremely short term», to avoid being misleading.
«Restriction of dietary fat»
They talk about dietary fat restriction, but in this study they only used one amount of dietary fat, 17 g per day. That is extreme and unsustainable in the long-term as it poses a risk of not getting essential nutrients (see). Our bodies need fat, not only as energy, but because of fat itself and those vitamins found in fats or whose absorption is facilitated by fat intake. The result of this study in no case can be extrapolated to arbitrary dietary fat reductions in a healthy diet.
They shouldn’t talk about «fat restriction», because they didn’t put to test several levels of fat restriction: they used only a dangerously low amount of fat in the diet.
«Restriction of dietary carbohydrates»
The lower carb diet had 140g carbs/day. That amount is much higher than the recommended values for weight loss with a low-carb diet. In fact we are talking about an amount large even for weight maintenance (see), although not everyone agrees on this point. Designing a diet with a high amount of carbohydrates suggests bad faith when designing the study: it is a straw man. If they really wanted to compare their unreal ultra-low-fat diet with a low-carb diet they should have used a well designed low-carb diet.
They shouldn’t talk about «restricting carbohydrates» instead of saying the specific amount they used: 140g. Their findings may not apply to diets with other levels of restriction and therefore the authors’ conclusions are misleading.
But there is an infinitely more important detail: if we talk about «diets», we can’t take into account an experiment with a duration of six days. First, because the transition to a low-carbohydrate diet has a transient period, that could take even weeks (see). If they measure parameters during those transition days what we will see is not the effect of the diet, but from the transition between diets. Any researcher in nutrition for sure knows that, so, again, questions about the authors’ good faith arise.
Let’s add more data, «One male subject erroneously received the RF diet on the first day of the RC study period and one female subject erroneously received the RF diet on the final day of the RC study period«. What is the effect of mixing the participant’s diets in the results of the study? The results of these two participants were added to the pool of data.
And although the text of the article tells us that «. Subjects received both isocaloric diets in random order», the truth is that it wasn’t so. Two of the participants received only one of the diets, the RC diet, and their data were not eliminated from the study, as can be seen in Table 3. In the text there is no explanation on why their data wasn’t removed.
Before going on, let’s read again what the authors wrote:
«This study demonstrated that, calorie for calorie, restriction of dietary fat led to greater body fat loss than restriction of dietary carbohydrate in adults with obesity.»
Body fat loss didn’t differ between diets, fat restriction reached values dangerous for participant’s health, carbohydrate restriction was outside that recommended for weight loss and at least four of the participants’ data are suspicious. Without loss of accuracy the authors could have written the following:
«This study, with a ridiculously short duration that gives irrelevant results for weight loss in the long term, calorie for calorie, an unsustainable and dangerously low in fat diet did not produce greater fat loss than a low-carb designed not to work too well«. It seems to me this is a more appropriate way of summarizing their findings.
I was talking about the relevancy of the long-term effects of the diets. And I add to my reflections the importance of appetite reduction. My hypothesis is that low carbohydrate diets may be useful for weight loss for the following reason: carbohydrate restriction allows the burning of body fat, so that, for example, a body that normally needs 2000 kcal, can get 200 kcal of them from body fat, thus requiring only to ingest 1800 kcal per day. That person eats less as a result of losing body fat. And not the other way around.
Why calorie restriction doesn’t work for weight loss? Because if that same person that needs 2000 kcal /day eats just 1800 kcal of a diet high in carbohydrates, that person is starving. And his/her body, his/her metabolism, will react trying to stop the weight loss (see). The rebound effect makes weight loss stall around month 6 after starting the diet (see).
That’s my hypothesis about why some diets may work where others don’t. Can we see that in a study in which diets are followed for less than a week?
Some people argue that this study is irrelevant, since the advantage of the low-carbohydrate diet comes from the loss of appetite, and therefore forcing diets to be isocaloric worsens the result of the low-carb diet. I agree that the advantage has to do with the loss of appetite, but not because of the resulting reduction in the caloric intake (see). In my opinion, the correct comparison is an ad libitum low-carb diet (something that was not done in this study and therefore it is another thing not to like about this study), where the low-carb diet is really low-carb (something also not done in this study) versus a low-fat diet that adjusts its calories to those consumed in the low-carb group: two isocaloric diets in which some participants are starving while others don’t. And then we wait to see what happens in the long term. May be ultra-low-fat diet don’t make you hungry and allow long-term weight loss, but that is something we can’t conclude from this study.
In conclusion, I don’t doubt that an ultra-low-fat diet could, hypothetically, be more effective for weight loss than a low-carb diet. But first they must clarify how they will tune the diet so that it is not a health hazard. If they succeeded with that, I wouldn’t argue against promoting this diet: it may be an option for weight loss. But today the questions are, for how long can you follow their ultra-low-fat diet in this study? What is the risk of not eating enough essential fatty acids or vitamins? What real foods can people eat to get those essential nutrients without, at the same, exceed the limits imposed on total fat intake? In any case, what we need to make clear is that in this study they speak of «dietary fat restriction» but they virtually remove dietary fat from the diet. Their findings can never be extrapolated to other conditions. Don’t let them fool you: this study was not a comparison of «fat restriction» versus «carbohydrate restriction» for weight loss.
This study has been proposed (see) as proof that insulin can’t explain why low-carb diets produce greater weight loss than other diets with the same calories (see) . Falacious and opportunistic argument. Firstly because we can not talk about «diet» when the participants didn’t even follow this way of eating for a whole week. And the same food has different effects within our body depending on the diet we followed in the previous days. The adaptation period should have been respected. And secondly because what has been seen in this experiment is that if fat intake is practically zero (ultra-low fat diet), at least in the very short term the rate of fat oxidation is maintained high and therefore our body burns more fat than the amount ingested. And consequently you lose body fat. That is, if we have alcohol and one match, in this experiment alcohol has been removed and it has been found that under these conditions there is no fire when you light the match. Does that mean that the match has nothing to do with the fire when there is plenty of alcohol? That is, have they demonstrated with this study that with a healthy fat intake keeping blood insulin levels low doesn’t play a role in the accumulation of body fat? Obviously not, they haven’t demonstrated that. Moreover, the hypothesis of insulin would be the opposite of what they tried to prove. In the analogy the insulin hypothesis would be that if there is no match, alcohol is not going to burn. Both «extreme situations» are possible and confirming one of them doesn’t mean the other one is false.
With that said, I don’t believe the insulin hypothesis is an absolute truth (see , see). It is a simple way to explain why counting calories is nonsense (see) and it is a simple way to explain why reducing carbs is healthier than other dietary approaches. I do believe talking about of insulin is better than talking about calories to make people understand how to eat healthy.
I have four questions.
The first one: is it reasonable to think that this study was designed, and its conclusions have been written, to fool people by saying that restricting fat is better than restricting carbohydrates for weight loss in people with weight problems? I copy again what the authors say: «This study demonstrated that, calorie for calorie, restriction of dietary fat led to greater body fat loss than restriction of dietary carbohydrate in adults with obesity.»
The second one: if these authors have demonstrated what they say they have demonstrated, how do they explain all those scientific studies that reach the opposite conclusions, such as this one? Do people systematically in all studies make mistakes that favor low-carb diets? Because if they really have demonstrated that restricting dietary fat produces more body fat loss than restricting carbohydrates, all those other studies (see,see) must have been poorly made. If they have demonstrated that…
The third one: how does the energy balance theory fit in the idea that a low-fat diet produces more weight loss than a low-carb diet, having both diets the same calories? It is interesting how some people have sacrificed, without hesitation, their energy balance ideas when they believed that the low-fat diet was winning.
The fourth and last one: the ultra-low-fat diet in this study is ultra restrictive in fat intake. Are now restrictive diets acceptable, as long as it is fat what is restricted? What about all those falacies about the long term unsustainability of a restrictive diet? Is there no longer a need to eat everything so that the diet is «sustainable» in the long term?
Finally, this is the summary that Ivor Cummins wrote about the article we are talking about:
A little knowledge is a dangerous thing – so much is artfully wrong with this study…..it’s unlikely to have happened by chance stupidity. This methodology is becoming common – comparing a ridiculously short-term isocalorific pseudo-low-carb (29%), to an ultra impossible low-fat (8%). It is becoming known that short-term ultra-low-fat (whilst ridiculous in real life), can generate the apparent effects that the experimenters are specifically mining for. Keeping the ‘low-carb’ not really low carb at all, enhances the errors, and tuning the timebase to be very short ensures no metabolic adjustment to allow benefit to accrue. The whole deceitful mess is massively magnified by eliminating the crucial hunger-driving factor from the equation. Yes, I suspect these guys know exactly what they are doing here; while it may not be direct remuneration into their personal bank accounts, the question of their funding is still pertinent. What is driving their deceit?