Load forecasting is an essential step in power systems operations with important technical and economical impacts. Forecasting can be done both at aggregated and stand-alone levels. While forecasting at the aggregated level is a relatively easier task due to smoother load profiles, residential forecasting (stand-alone level) is a more challenging task due to existing diurnal, weekly, and annual cycles effects in the corresponding time series data and fluctuations caused by the random usage of appliances by end-users. Exploring the available historical load data, it has been discovered that there usually exists an interesting trend between the data from a target house and the data from its surrounding houses. This trend can be exploited for improving the forecast accuracy. One can define several different features for each house, including house size, occupancy level, and usage behavior of appliances. While the number of such features can be large, the main challenge is how to determine the best candidates (features) for an input set without increasing the forecasting computational costs. With this objective in mind, we present a forecasting approach which combines ideas from Compressive Sensing (CS) and data decomposition. The idea is to provide a framework which facilitates exploiting the existing low-dimensional structures governing the interactions among residential houses. The effectiveness of the proposed algorithm is evaluated using real data collected from residential houses in TX, USA. The comparisons against benchmark methods show that the proposed approach significantly improves the short-term forecasts. (C) 2015 Elsevier B.V. All rights reserved.