The ultrafast response of a pyroelectric sensor with near-infrared responsivity is demonstrated by combining a pyroelectric thermal detector with wavelength-selective nanoparticle absorbers. Thermal detectors, such as bolometric, pyroelectric and thermoelectric devices, are uniquely capable of sensing incident radiation for any electromagnetic frequency; however, the response times of practical devices are typically on the millisecond scale(1-7). By integrating a plasmonic metasurface with an aluminium nitride pyroelectric thin film, we demonstrate spectrally selective, room-temperature pyroelectric detectors from 660-2,000 nm with an instrument-limited 1.7 ns full width at half maximum and 700 ps rise time. Heat generated from light absorption diffuses through the subwavelength absorber into the pyroelectric film producing responsivities up to 0.18 V W-1 due to the temperature-dependent spontaneous polarization of the pyroelectric films. Moreover, finite-element simulations reveal the possibility of reaching a 25 ps full width at half maximum and 6 ps rise time rivalling that of semiconductor photodiodes(8). This design approach has the potential to realize large-area, inexpensive gigahertz pyroelectric detectors for wavelength-specific detection from the ultraviolet to short-wave infrared or beyond for, for example, high-speed hyperspectral imaging.