Comparisons of the energy and emission intensity of transportation modes are standard features of sustainable transportation research, policy, and advocacy. These comparisons are typically based on average energy and emission factors per passenger trip or per passenger-kilometer traveled. However, as acknowledged in the energy production sector, comparing average emission factors can misinform policy and other decisions because it fails to represent the marginal impact of changing demand. The objective of this paper is to quantify the difference between average and marginal energy and emission factors for passenger transportation modes. Transportation system operations data are used to estimate energy and emission factors per passenger-kilometer traveled for U.S. urban and intercity travel. Marginal emission factors range from 30% (intercity rail) to 90% (private vehicles) of average factors. For urban travel, private vehicles and public transit have similar average emission factors, but marginal factors are 50% lower for transit. The average emission factor for intercity rail is 10% lower than air travel and 30% lower than private vehicles, but the marginal factor is 60% and 80% lower, respectively. Using average energy and emission factors to represent the impacts of travel by different modes is biased against public transit and discounts the benefits of shifting travel away from private passenger vehicles.