Oil and gas pipeline leaks are the responsibility of a wide range of stakeholders including utilities, the public, construction firms and state safety program personnel. As a result, in the United State, state pipeline safety programs focus heavily on public education, outreach, and stakeholder participation so as to reduce the occurrence of these events. The US Pipeline & Hazardous Materials Safety Administration (PHMSA) directs federal support to state safety efforts through formula and project grant programs for state and local agencies that emphasize different aspects of collaboration. We combine these grant data with nationwide, longitudinal records on pipeline leaks, and use a multilevel Bayesian model to evaluate how different grant emphases relate to subsequent incident rates. State damage prevention grants are not associated with reductions in leaks but are associated with improvements in error recognition. Pipeline incidents are downstream from safety programs and impacts and identifying, minimizing, and correcting leaks requires states to motivate utility actions while also attempting to improve public awareness. This makes assessment of safety outcomes a challenge without better tracking of safety program communications and participation. In conclusion, we suggest improvements to data provision by states that could provide improved evidence for assessing safety improvements.