It’s been a while since I last wrote about my Running Cold Experiment and in that time I’ve completed several more trials. See Part 1, Part 2, and Part 3. Here’s what they look like:

Data

Revisiting my methodology

More interestingly, I discovered another issue with my methodology, again having to do with the fact that I only get minute-level resolution on my calories and METs, while most of my runs finish somewhere between the 10:30 and 11 minute mark.

I returned to the raw data in order to modify the Average METs and Calories Burned measurements for Trials 3 and 4 to include the 11th minute (they didn’t before, in previous posts). I did this because I thought it more appropriate to include the 11th minute in the calculations for Trials 5 and 6 because they were much closer to the 11 minute duration mark than the 10 minute mark, and I wanted to keep all the trials as consistent as possible.

But I am realizing that this is still problematic, and I needed some help to figure it out. There were some important things I needed to understand about how BodyMedia records their data in order to improve my methodology for how to record the samples.

So the problem is this: if I cut the data off at 10 minutes for the purposes of comparison, I effectively introduce a new variable to the experiment, which is the distance I run. It would make it different for each trial.

But if I include the 11th minute, that is problematic as well because it includes a substantial amount of cooldown time for my faster runs, which negatively impacts the average calories and METs for the run.

I was also noticing that when I looked at the data points closely and attempted to run my own averages, I was getting different numbers than what BodyMedia was calculating. Something just seemed to be off.

So I wrote to BodyMedia about the issue, and they took the time to write me a wonderfully detailed response to help me understand the finer points of how the armband works, and gave some good suggestions for making my experiment more rigorous.

Help from BodyMedia

Here are paraphrases of my questions and their response:

Me: I’m getting different averages from my manual calculations than what I see displayed in BodyMedia’s Activity Manager for the same period. Am I doing something wrong?

BodyMedia: Energy expenditure values represent the value over the minute. See the attached document [Nick: see “Feature Generation” on the page marked 1616] for how the sensors are sampled over the minute.

[Nick: in other words, when a calorie value is displayed for 5:34pm, it is the sum of the calories that were burned from the start to 5:33pm to the start of 5:34pm. It shows the calorie burn over the previous minute. Up to now I had been looking at it the other way, that 5:34pm would mean the start of 5:34pm until the start of 5:35pm. This explains why my calculations were off and everything immediately started matching up once I fixed it]

Me: Since I only get by-minute resolution, how can I deal with device synchronization issues (i.e. 5:34pm on my phone probably started before or after 5:34pm on my BodyMedia armband, so how do I know I’m looking at the appropriate time range on each device)?

BodyMedia: Minute resolution for energy expenditure is all that is available for Activity Manager.  SenseWear [BodyMedia’s enterprise version of the product designed for clinicians and their patients] users can set from 32Hz to 10 minute granularity…

Standard experimental procedure would have you avoid end effects by not using end data points. I would suggest using the 8 minutes of steady state data. If you want 10 minutes of data for your analysis, record for 12 minutes.

I start and end a lot of my experiments with 2 minutes of no steps. This helps me identify the boundaries of the test. You are guaranteed to get one minute of no steps in your data file (minimal energy expenditure) and I do not use that minute or neighboring minutes.  2 minutes on your watch give one full minute of  no steps and minimal Calories in the data because of any misalignment between the armband time and your watch time.

Action Items

This information really helped me to focus the experiment:

1. Revisit my BodyMedia data to make sure I am referencing the right time frame (i.e. timestamp 5:35pm = what happened between 5:34 and 5:35, not 5:35 and 5:36)
2. Start standing still for 2 minutes of no steps immediately before and after my run in order to help delimit it in the data
3. Start excluding the first and last minute of the run in my data. In other words, look at my shortest run, take the time frame that starts in minute 2 and ends in the second to last minute, and apply that time frame to all of the trials. This will help to get the best consistency. Fortunately I still have all my raw data so I can go back and adjust my existing metrics.

Next: See Part 5 where I start to make some comparisons across runs.