The adsorption performance of the thermal energy storage (TES) system changes depending on the material properties of the adsorbent itself, but the change of the hardware structure can also substantially change the adsorption characteristics. In this study, a laboratory-scale adsorption-based TES system was constructed, and the adsorption performance of three adsorbents was evaluated in the same system to compare the adsorption performance between adsorbents. The adsorption characteristics of silica gel, zeolite 13X, and 4A, which are the most preferred adsorbents in the physical adsorption-based TES system, were selected for evaluation. Experiments with each adsorbent were performed, including heat recovery to evaluate the heat transfer effect and the amount of heat recoverable in the actual TES system. Experimental results have identified several key characteristics of the adsorption and performance of each adsorbent in the TES system, as well as operating parameters that determine the influence of adsorption performance on the TES system. The actual energy storage density of the adsorbent is affected not only by the enthalpy of adsorption of the material itself but also by other factors. These factors include the difference in thermal conductivity that causes a difference in temperature distribution and the magnitude of mass transfer resistance due to the shape of the adsorbent particle and the actual TES system reactor structure. If the reaction heat generated during the adsorption reaction cannot be effectively released, the adsorption performance is significantly lowered due to the increased temperature of the reactor inside. This phenomenon was commonly observed in adsorbents examined in the present study. The uptake amount, X [g/g], was increased by allowing the inside of the reactor to be maintained at a lower temperature through heat recovery. In case of silica gel, the temperature rise during adsorption reaction is not high due to the difference of isotherm characteristics compared with zeolites, but it is possible to absorb more amount of adsorbate and to recover heat for a longer time. The energy storage density is affected by the temperature increase effect and the uptake amount of adsorbate during the adsorption reaction. The experimental results show that the energy storage density of zeolite 13X is 15% and 28.7% higher than that of silica gel and 4A, respectively, and the temperature rise due to heat generation during adsorption reaction is also high, which is advantageous in adsorption TES system performance.