Covalent organic frameworks (COFs) with well-defined and customizable pore structures are promising templates for the synthesis of nanomaterials with controllable sizes and dispersity. Herein, a thioether-containing COF has been rationally designed and used for the confined growth of ultrafine metal nanoparticles (NPs). Pt or Pd nanoparticles (Pt NPs and Pd NPs) immobilized inside the cavity of the COF material have been successfully prepared at a high loading with a narrow size distribution (1.7 +/- 0.2 nm). We found the crystallinity of the COF support and the presence of thioether groups inside the cavities are critical for the size-controlled synthesis of ultrafine NPs. The as-prepared COF-supported ultrafine Pt NPs and Pd NPs show excellent catalytic activity respectively in nitrophenol reduction and Suzuki-Miyaura coupling reaction under mild conditions and low catalyst loading. More importantly, they are highly stable and easily recycled and reused without loss of their catalytic activities. Such COF-supported size-controlled synthesis of nanoparticles will open a new frontier on design and preparation of metal NP@COF composite materials for various potential applications, such as catalysis and development of optical and electronic materials.