To support the development and characterization of chromophores with targeted photophysical properties, excited-state electronic structure calculations should rapidly and accurately predict how derivatization of a chromophore will affect its excitation and emission energies. This paper examines whether a time-independent excited-state density functional theory (DFT) approach meets this need through a case study of BODIPY chromophore photophysics. A restricted open-shell Kohn-Sham (ROKS) treatment of the S-1 excited state of BODIPY dyes is contrasted with linear-response time-dependent density functional theory (TDDFT). Vertical excitation energies predicted by the two approaches are remarkably different due to overestimation by TDDFT and underestimation by ROKS relative to experiment. Overall, ROKS with a standard hybrid functional provides the more accurate description of the S-1 excited state of BODIPY dyes, but excitation energies computed by the two methods are strongly correlated. The two approaches also make similar predictions of shifts in the excitation energy upon functionalization of the chromophore. TDDFT and ROKS models of the SI potential energy surface are then examined in detail for a representative BODIPY dye through molecular dynamics sampling on both model surfaces. We identify the most significant differences in the sampled surfaces and analyze these differences along selected normal modes. Differences between ROKS and TDDFT descriptions of the S-1 potential energy surface for this BODIPY derivative highlight the continuing need for validation of widely used approximations in excited state DFT through experimental benchmarking and comparison to ab initio reference data.