Monitoring of elemental composition of structural concrete is required to protect reinforcing steel in concrete against corrosion. Prompt Gamma Neutron Activation Analysis (PGNAA) technique has been used to measure the elemental composition of concrete samples. In this study 2.8 MeV neutron-based PGNAA setup at King Fahd University of Petroleum and Minerals (KFUPM) has been tested for elemental analysis of concrete sample. Later the setup will be used to measure chloride and sulfate ion concentration in concrete samples. Corrosion of reinforcing steel in concrete is believed to be caused by chloride and sulfate ions. In the KFUPM PGNAA facility the prompt gamma rays are produced through capture of thermal neutrons in bulk sample. The sample size, to be used with PGNAA setup, is determined through Monte Carlo simulations. In this study, prompt gamma ray yield from calcium and silicon present in the concrete sample was measured for various thickness of the front moderator of the PGNAA setup. The experimental yield of prompt gamma ray was compared with the results of the Monte Carlo simulations and an excellent agreement has been achieved between the two. The study has shown that the PGNAA technique can be successfully applied for elemental analysis of the concrete samples.