It is widely held that the first step in forming gas-giant planets, such as Jupiter and Saturn, was the production of solid 'cores' each with a mass roughly ten times that of the Earth1,2. Getting the cores to form before the solar nebula dissipates (in about one to ten million years; ref. 3) has been a major challenge for planet formation models(4,5). Recently models have emerged in which 'pebbles' (centimetre-to-metre-sized objects) are first concentrated by aerodynamic drag and then gravitationally collapse to form objects 100 to 1,000 kilometres in size(6-9). These 'planetesimals' can then efficiently accrete left-over pebbles(10) and directly form the cores of giant planets(11,12). This model is known as 'pebble accretion'; theoretically, it can produce cores of ten Earth masses in only a few thousand years(11,13). Unfortunately, full simulations of this process(13) show that, rather than creating a few such cores, it produces a population of hundreds of Earth-mass objects that are inconsistent with the structure of the Solar System. Here we report that this difficulty can be overcome if pebbles form slowly enough to allow the planetesimals to gravitationally interact with one another. In this situation, the largest planetesimals have time to scatter their smaller siblings out of the disk of pebbles, thereby stifling their growth. Our models show that, for a large and physically reasonable region of parameter space, this typically leads to the formation of one to four gas giants between 5 and 15 astronomical units from the Sun, in agreement with the observed structure of the Solar System.