One of the strengths of science is its systematic approach to getting reliable information about the world by comparing outcomes of experiments where one parameter is varied while the others are held constant. This experimental approach comes satisfyingly close to letting us compare different ways the world could be — at least on many occasions.
There are some questions, though, where good experimental design requires more cunning.
A couple years ago, the local media in the San Francisco Bay Area ran a series of stories examining “the [human] body’s burden” of synthetic chemicals. The centerpiece of the coverage was the testing of the members of a Berkeley family to track the levels of various synthetic chemicals found in their blood, hair, and urine. Some of the results were shocking (such as the highest recorded level of a flame-retardant compound in a human being in the blood of the 20-month-old). And of course, while the technology exists to quantify the parts per billion levels of the compounds rather precisely, scientists don’t actually know what these levels mean, in the short term or the long term, for the health of these people.
So there’s a great scientific question that you might want to answer: what does exposure to these synthetic chemicals (at various levels of exposure) do to a human body?
Since we don’t know what the heck these chemicals do to humans, it might be quite hard to set up a study with human subjects where we could actually obtain informed consent (since the information … just isn’t there yet). Besides, the regulations governing research with human subjects require that we do preliminary studies in an appropriate animal system first. So, let’s go to the Mouse House and set up an experiment with animals.
The broad idea driving our experiment is, to determine the effects of X, we need to compare a set of organisms exposed to X and a set of the same organisms not exposed to X. If the only difference between the two populations is whether they have been exposed to X, we can at least tentatively attribute the difference in outcomes (if any such difference is observed) to exposure or lack of exposure to X.
Here’s a reasonable experimental design: Get yourself 200 mice. Set aside a certain number as your control group; they just get to be plain old, unexposed laboratory mice. Take the rest of the mice and break them into (say) three different “treatment” groups: one group gets high exposure to the flame retardant chemical, one mid-level exposure to it, and one low-level exposure. Except for the exposure, the “treatment” groups get the same treatment (food, cages, toys, etc.) as the control-group mice. All the mice get periodic blood tests and have their health assessed in the appropriate mousy ways. They get to live out their days in laboratory luxury, and when they die the cause of death and state of health at time of death will be determined. The effect of the flame retardant exposure on health will be assessed by comparing the “treatment” group mice to the control group mice.
Of course, you might ask why we need to do this with mice. There are plenty of humans out there, apparently, who are already exposed to these compounds. Presumably, to find out the effects of these compounds on human health, we might learn more by tracking the health of the already-exposed humans than by exposing and studying mice. (A mouse is not identical to a human in all relevant respects, after all.)
One problem is that it’s much easier to control for all sorts of other possibly relevant variables (like diet, exercise, etc.) working with mice in a lab than it is if you’re studying humans running free in the world. (Yes, I know, McDonalds and Starbucks and the persistent push to homogenize American culture are doing their best to help with this, but there’s still a good bit of variation in the ways people live.)
Another problem is one that surprised the researchers studying the Berkeley family for the story. Ideally, to determine the effects of the compounds on human health, you’d want to be able to compare the health of people exposed to them to the health of people not exposed to them. And, the researchers could not find any Americans who did not have some of these compounds in them! When they looked on other continents, it was pretty much the same story. In other words, it might be impossible to find an appropriate human control group to find out what (if anything) these compounds are doing to us.
Here, you might think that this would seal the deal for the mice — if we can set up an appropriate mouse control group, this is our best bet to get a definitive answer about the effects of these compounds. But there are a couple of potential problems. For one, it’s entirely possible that all the mice have these compounds in them already, too. (If the researchers checked this, it was not reported in the media coverage.) If the mice do already have the substances of interest in them, there’s no especially clean way to set up a mouse control group either.
If American laboratory mice screen negative for all these compounds, though, it might point to a fundamental problem with this animal model.
The fact that all the humans in the country (and beyond) seem to have these compounds in them points to their prevalence in our environment: in the food and water, in the air, in the dust, in the building materials, etc. Laboratory mice are not getting certain kinds of exposure that humans are (e.g., they don’t generally smoke, or wear nail polish, or use shampoo or lotion, or disposable diapers, etc., etc.); but many of the humans whose blood contains these compounds haven’t exposed themselves in these ways either. This would seem to indicate that these compounds are ubiquitous in our environment. And the laboratories in which the mice are raised are not entirely sealed off from that larger environment in which, presumably, the human exposures took place.
So, if the mice that have been exposed to the same environment seem not to show the presence of these compounds in their blood, urine, and hair, that would seem to indicate that the mouse’s body either doesn’t take up these compounds, or that it has some process for getting rid of them quickly. In other words, we’d already have good reason to think the mouse body responds differently to these compounds than does the human body. And this might be good reason to worry that studies of these compounds on mice wouldn’t tell us what we need to know about exposure to these compounds on humans.
In circumstances like these, is there a better experimental approach to this question? What would it look like?
Space station?
It would look like a human cohort study. So it wouldn’t be an experiment. Following a large group of people with varying exposure to the putatively harmful ‘Chemical A’ and measuring their other characteristics and relevant exposures and comparing these to disease outcomes years later.
The lack of a non-exposed group is not necessarily a problem as what you are generally looking for is a dose-response effect. Assuming the chemical A is actually a ‘bad thing’ then as the amount you have been exposed to over time increases so too should the odds of developing disease (even after you control for all the other factors we suspect or known also cause the disease). You can see this in older studies of the effects of cigarette smoking on lung cancer where virtually everybody smoked or were exposed to second hand smoke in the workplace, home, or in movie theatres etc.
The dose-response need not be a linear or straight line either. Small amounts of alcohol seem to reduce the liklihood of coronary artery disease, for instance. But alcohol intake also increases the odds of other disease so the best option seems to be small amounts that protect you from CAD without markedly increasing the other diseases.
The animal toxicology studies are generally going to tell you nothing unless you’ve already shown the dose-dependent association in humans and want to be sure that the effect is actually causal. Mice aren’t humans afterall.
Further experimental proof that chemical A causes disease at this stage would be testing whether a treatment that reduces this chemical in humans actually results in a reduction in later disease (i.e. the randomised (possibly placebo) controlled trial approach).
The ‘treatment’ could be changing environmental exposure (i.e. removing the chemical from the environment somehow), it could be a drug that helps the body remove the chemical or to stops it’s absorbtion into the body. By showing that reducing the exposure to chemical A reduces later disease you also help show that chemical A was causing part of the problem in the first place.
1. You could start with infant mice and begin the experiment at a given point in their lives.
2. You could begin with a group which all had the same levels. At least it would give you a common baseline, and the exposure rate could be controlled from that point.
You could take group 2 above and isolate them to see if concentrations fell. If they did in a consistent manner, you could breed successive generations until concentrations were at an acceptably low level.
All the above represent potential starting points.
I always worry about levels of previously untested-for chemicals in infants; they are much more difficult to get blood from than adults, and there may have been contamination of some kind.
Nevertheless, this is exactly the kind of thing that we need human clones for! Isn’t that why Bush keeps insisting that scientists start “human farms”? (Or did I hear him wrong?)
This is a nice example of science driven by fear, rather than by pure facts! Of course, the US has a long history of trying to figure out the “effect” of small doses of various synthetic chemicals. What is fascinating is that the assumption underlying all these studies is: it’s chemical, it’s artificial, therefore it MUST be bad for your health! Yet if you dig into it, you realize that there is no empirical proof of this, in general. As you point out, proving that a small exposure is harmful is a daunting task. Furthermore, most studies have to rely on a “linear” dose-effect model, which is impossible to prove once the effect beomes small enough. If you don’t know the dose-effect relationship to start with, and only get results with large doses, your study is basically useless. But we keep doing these studies because the fear of artificial, un-natural chemicals is such a strong psychological incentive.
Another point is that we are exposed to a multitude of “natural” chemicals. But despite being natural, many of these chemicals are of course highly toxic. But somehow, in the minds of many people, “natural” is “good”. That is of course one reason why many people die each year from eating toxic mushrooms! Some of these people would not have dared drinking a diet soft drink, for fear of aspartame…
Funnily enough, you could make a correlation between life expectancy and exposure to synthetic chemicals, and would probably conclude that such exposure is, in general, good for your health!
For those interested, Allan Mazur’s book “True warnings and false alarms” is a good account of how such fears about health and the risk of technology have led to a lot of useless research (and some useful too!).