Covalent organic frameworks (COFs) are porous crystalline materials that are entirely composed of organic building blocks and can be assembled straightforwardly from solution. The main synthetic challenge associated with COFs, compared to other porous materials such as zeolites or metal organic frameworks, is their poor long-range order; typical sizes of crystal domains do not exceed a few tens of nanometers. Here, we develop a model of the molecular constituents of COF-5 and follow the early stages of its assembly dynamics from dilute solution. Our simulations indicate that under typical experimental conditions COF-5 formation happens not through nucleation but far from equilibrium through spinodal decomposition. This rapid assembly mode leads to a plethora of defects that are difficult to anneal and that are likely responsible for the limited crystallinity observed in the synthesis of many COFs. We analyze the driving forces for COF-5 formation and find that stacking interactions between aromatic molecular constituents are too strong. When these interactions are weakened, assembly proceeds through single nucleation events followed by slow growth. The COF-5 crystallites obtained in this way are essentially defect-free. These results suggest experimental strategies for growing COFs with enhanced crystalline quality.