Microgels are colloidal-scale particles individually made of cross-linked polymer networks that can swell and deswell in response to external stimuli, such as changes to temperature or pH. Despite a large amount of experimental activities on microgels, a proper theoretical description based on individual particle properties is still missing due to the complexity of the particles. To go one step further, here we propose a novel methodology to assemble realistic microgel particles in silico. We exploit the self-assembly of a binary mixture composed of tetravalent (cross-linkers) and bivalent (monomer beads) patchy particles under spherical confinement in order to produce fully bonded networks. The resulting structure is then used to generate the initial microgel configuration, which is subsequently simulated with a bead-spring model complemented by a temperature-induced hydrophobic attraction. To validate our assembly protocol, we focus on a small microgel test case and show that we can reproduce the experimental swelling curve by appropriately tuning the confining sphere radius, something that would not be possible with less sophisticated assembly methodologies, e.g., in the case of networks generated from an underlying crystal structure. We further investigate the structure (in reciprocal and real space) and the swelling curves of microgels as a function of temperature, finding that our results are well described by the widely used fuzzy sphere model. This is a first step toward a realistic modeling of microgel particles, which will pave the way for a careful assessment of their elastic properties and effective interactions.