Saturday, May 30, 2020

Off the cutting-room floor: Fitness "science," a collection of bad thinking.

(Off the cutting-room floor: writing that didn't make the book. References to pitfalls come from that.)


People who run regularly are fitter (have better cardiovascular function) than people who don't. Based on that observed regularity, early fitness gurus helped start the jogging craze in the mid-1960s.

But there's an obvious problem with the causality: are people fitter because they run (running causes fitness) or are people able to run because they're fit (fitness causes running)?

We've been socially conditioned to accept the first implication without question, but as we've learned from pitfall 4 (hidden factors in correlations), these questions are more complex than they appear. In particular, as we know from pitfalls 2 and 3, results from self-selected samples and truncated data can be very deceptive.

What was the data, how was it modeled, and how do the results influence decisions?


When the data isn't representative


Because they didn't like the self-selected sample of runners, many aspiring fitness researchers run what they thought were controlled experiments: they took a group of sedentary people willing to participate in the experiment and divided them in two sides, half kept their sedentary life, half took up running. After some period, the runners' fitness was compared with the sedentary group. Runners were fitter.

However people with the best intentions to get in shape might not follow through, and many of these experiments had drop-out rates of more than 50% on the runner side (more than half the people who started the experiment abandoned it).

These studies traded off the hidden factor correlation and self-selection of people who don't run to begin with for the hidden factor correlation and self-selection of people who didn't finish the experiment of the runner side.

Eventually some researchers run controlled experiments with appropriate samples, tracking all participants (even the ones who dropped out) and analyzing the data properly. The studies showed lower effect sizes than those of self-selected samples, of course, but the results supported the relationship between running and fitness.



When information isn't what it appears


The controlled experiments showed that there was causality from running to being fit. Are we done?

In a paper that revolutionized the exercise world, a team led by Izumi Tabata showed that short but intense workouts could deliver the same gains as long sessions of low-intensity exercise like jogging.  (Yes, gym Tabatas are named after him, even though most of them don't follow the protocol of the paper. The paper is: "Metabolic profile of high intensity intermittent exercises" published in 1996 in the journal Medicine & Science in Sports & Exercise.)

This raised another issue: high-intensity exercise might worth through muscle strengthening: after all, if the muscles get stronger, there's less effort to do the same task, and that lower effort puts less stress on the cardiovascular system, so the person appears fitter.

This is a controversy still going on in the fitness industry and sports medicine. We'll sidestep it because we only care about the modeling implications: if this is a hidden mediating factor correlation (again pitfall 4!), then it can be tested to some extent by measuring the factor itself: muscle strength. (We'll leave it at that, because that's the modeling insight.)

We could ask whether it makes a difference which process (direct fitness effects or indirect effects via muscle strength) is at work, and with that we enter decision analysis.


When decisions don't follow from information


Why does the causal process make a difference? If it's a hidden mediating factor of muscle strength, we can get the muscles stronger with Tabatas, and if it's direct effect on fitness, we can get that fitness with Tabatas too. It's Tabatas in both cases, so why make a fuss?

Ah, but Tabatas (and similar techniques) aren't the only way to strengthen muscles, and they are high-impact activities; if the first causal relationship is the true one, then the elderly and those recovering from injury might get the same fitness gains from slow-movement weight training, but if it's the second causal relationship, they can't.

Knowing which model is right makes a difference to the elderly and those recovering from injury. That's worth some fuss.

We won't take a position on this, because the book is not about fitness, but it's clear that knowing which model describes reality, the direct causality of cardio exercise to cardiovascular fitness or the indirect causality of cardio exercise to muscle strength to cardiovascular fitness, can make a lot of difference for people who can't do high-impact exercise.

On a separate issue, given Tabata et al's results, why do people jog?

This brings us back to pitfall 10 and the difference between decision-making models and decision-support models. Determining whether to jog based on the effectiveness of jogging for cardiovascular fitness uses the model as the sole driver of decision making.

But even if we understand that Tabatas are a better form of cardiovascular exercise, and we take the result of the model as information for our decision, we might use other criteria to make the decision. We might enjoy the activity; or use it to socialize with coworkers; or understand that running is useful skill for life and like all skills needs training.

We make the decisions, not the models. Well, not always the models.

Saturday, May 9, 2020

I'm writing a short book

Greetings, carbon-based lifeforms,

No, I haven't given up blogging; it's just on hold, while  I'm writing a short book putting together some thoughts on what people do wrong with numbers, data, and models. It'll have a lot of pictures and be priced to move.

It'll include material from some blog posts and executive education materials, reworked a bit, of course, but there's a lot of extra work involved.

Here are some of the pictures, work in progress (as usual, click for bigger):

(Adapted from this blog post.)

 (Adapted from exec-ed materials, not a blog post.)

(Adapted from this blog post.)

(Adapted from this blog post.)

Live long and prosper,

JCS

- - - - - -

Yes, the greeting is a paraphrase of the title of an AC Clarke book of collected essays.  Call it a homage.