The wavelength down conversion approach to solid-state lighting (SSL) uses down conversion materials to produce visible light when excited by near-UV or blue emission from InGaN LEDs. This review discusses two classes of down conversion materials: phosphors and semiconductor quantum dots (QDs). Strong absorption of the excitation wavelength; high luminous efficacy of radiation, which enables white light with a high color rendering index and a low correlated color temperature; high quantum efficiency; and thermal and chemical stability are some of the criteria for down converters used in SSL. This review addresses the challenges in the development of down converters that satisfy all these criteria. We will discuss the advantages and disadvantages of several phosphor compositions for blue and near-UV LEDs. The use of core/shell architectures to improve the photoluminescence and moisture resistance of phosphors is presented. QDs are another class of down conversion materials for near-UV and blue LEDs. Strategies to improve the photostability and reduce the thermal quenching of QDs include strain-graded core/shell interfaces and alloying. We discuss Cd-containing II-VI QDs, and Cd-free III-V and I-III-VI QDs and their potential for SSL applications. Finally, a description of different methods to integrate the phosphors and QDs with the LED is given.