One of the significant concerns surrounding the proposed "science transparency" changes at the EPA has to do with revisiting older "pivotal regulatory" science. For example, consider this study, which set the stage for air pollution standards. That paper is now more than 25 years old; you have to wonder how it has withstood the test of time. Let's take a look.
Before proceeding, let me make my bias clear. I don't think to revisit old scientific papers is as fruitful as refuting them with newer findings, and I think air pollution is bad – on a scale from clean, to cleaner, to cleanest, I am in that cleaner to the cleanest area.
The "Harvard Six Cities," formally, "An Association Between Air Pollution And Mortality In Six U.S. Cities," was initially published in the New England Journal of Medicine but was just one of several articles spun off from the research on those six communities. To really understand the Six Cities study, you have to begin with the original study design published in 1979 (see sources). The authors clearly understood the concerns that even today plague these epidemiologic studies.
"In many studies, the assessment of exposure was made on the basis of historical information or by extrapolation from single fixed monitoring sites. In some studies modeling was used to estimate the general levels of exposure. At times, these estimates have been verified by local measurements. Even with the delineation of the isopleths of pollution in a community, only rough estimates of the actual exposure of the residents can be obtained. Furthermore, most persons spend much time indoors, but few epidemiologic studies have monitored indoor air to determine the differences between indoor and outdoor air."
Their goal was to mitigate those problems. To that end, they chose cities providing "a gradation of anticipated exposure to pollution." But they wanted to minimize heterogenicity, so they chose towns of a specific size, 50,000, with "no confounding large occupational groups, such as underground coal miners." And few sources of pollution, along with an "ethnic mix [that] was not too heterogeneous," the Six Cities studies look at 8111 white individuals.
"We concluded that, as far as age, sex, and occupation were concerned, our respondents represented reasonably comparable samples from each city." [1]
And while a medical history was taken, there is no evidence provided that these cities were similar for their baseline health or socio-demographics, the only subgroups were based on smoking history and weight.
The study made use of three monitoring systems to assess air quality. First, a fixed central site, located "at a site representative of the total residential exposure." These single sites form the basis of the subsequent Six Cities report. Today, a single fixed site would be considered inadequate. They also made use of indoor monitors from a much smaller sample of 68 homes as well as personal monitors worn by volunteers. I have not found those individual measures but did surface their paper describing the relationship they reported between indoor and outdoor pollution. Briefly, they found that about 70% of outdoor air "infiltrated" indoors, full air conditioning of the building reduce infiltration to roughly 35%, "while preventing dilution and purging of internally generated pollutants." Those internal pollutants were, smoking, and to a much smaller extent, the use of gas stoves. [2]
"The outdoor respirable particulates and sulfates were found to have a uniform distribution across each of the cities, except Steubenville, where topography and industrial siting set up large gradients. This would indicate that the sample population in each city, except as noted in Steubenville, has approximately the same exposure to outdoor particulates. However, there are significant differences m the indoor mean concentrations measured at our sampling sites in each of the cities. Thus, actual personal exposure, which is determined in large part by the indoor environment, will vary significantly among people in the sample population in each city." [Emphasis added]
The authors found that the correlation between indoor and outdoor pollution was 53% for Steubenville, the most polluted and 46% for Portage, the least polluted – the two cities driving the findings of the Six Cities study. And while other cities showed a uniformity to the dispersion of PM2.5, that was not the case for Steubenville.
The bottom line, the dataset used in the Six Cities study, was probably not representative of individuals based on ethnicity or socio-demographics, two factors we would include in an air pollution study today. There is no evidence that the populations were balanced for their baseline health issues other than smoking and BMI. And the data collected and subsequently presented was from single-site monitors, which the authors acknowledge failed to capture "actual personal exposure," especially for Steubenville and Portage, the anchors of their subsequent analysis. I will leave the last word to the authors,
"The study was designed to measure changes in these parameters with time. Any differences found between cities in a cross-sectional examination of adults may be attributed to past events and such variables as socioeconomic status. Nevertheless, this preliminary analysis leads us to believe that our measurements may, indeed, reflect effects of current air quality on health… The ultimate test of the hypothesis must await the prospective collection of the data and their subsequent analysis."
The authors felt that their data best reflected longitudinal changes, not a cross-sectional comparison of cities. Still, it was a cross-sectional comparison that the Six Cities study, their subsequent analysis, reported.
The Six Cities Study
The study looked at mortality comparing the least polluted with the most, Portage vs. all others, with Steubenville being the most polluted. They found that the mortality rate was higher in Steubenville by about 26%. The risk of death from cardiopulmonary disease or lung cancer increased by 37%, but air pollution, by their calculation, had little effect on other causes of mortality.
The study characterized the measurements of outdoor PM2.5 as nearly uniform in each of the communities and that the infiltration of outdoor pollutants into indoors resulted in "strong correlations" with outdoor measures – a much different statement than had been made previously about the same data and using the same methodology.
The population of the Six Cities study bears little resemblance to today's U.S. population in a significant way, smoking. 25-30% of the population was smoking, today that percentage is 14%. And of course, PM2.5, those inhalable particles were higher; the cleanest Portage had an average of 11 μg M3, Steubenville the highest at 29.6 μg M3; today they both are around 8.5 μg M3 already below the national standard of 12 μg M3.
In my view, the Six Cities study will not hold up to scrutiny today; its data is based upon older methodologies and a far different population. It does not take into account risk factors that today, we believe, are important. There is no advantage in going back to reanalyze the data, in the same way, there is no advantage in rolling back the improvements in air quality we have already made. [3]
In the opinion of the EPA, it was the best science available at the time; it no longer can make that claim. But wouldn't future "pivotal regulatory science" benefit from an external review? Can we continue to argue that the Six Cities study represents our best evidence?
[1] Occupations were characterized as professionals, managers, clerical, craftsmen, transport operators, laborers, and farmers, and the overall sample was slightly older and female compared to the national census.
[2] "The mean impact of smoking a pack of cigarettes inside is to raise the respirable particulate levels by approx. 18 μg M3. In fully air-conditioned buildings, the recirculation of the interior air increases this impact to approx. 42 μg M3. Other sources in the home add approximately 15 μg M3 to the indoor respirable particulate concentrations."
[3] This is from the EPA website showing the trends in national air quality standards.
Sources: An Association Between Air Pollution And Mortality In Six U.S. Cities New England Journal of Medicine DOI: 10.1056/NEJM199312093292401; Effects of Sulfur Oxides and Respirable Particles on Human Health American Review of Respiratory Disease DOI: 10.1164/arrd.1979.120.4.767; and The Indoor Data Indoor-Outdoor Relationships of Respirable Sulfates and Particles Atmospheric Environment DOI: 10.1016/0004-6981(81)90036-6
