This paper considers the problem of continuous state estimation from discrete context-based measurements. Context measurements provide binary information as obtained from the system's environment, e.g., a medical alarm indicating that a vital sign is above a certain threshold. Since they provide state information, these measurements can be used for estimation purposes, similar to standard continuous measurements, especially when standard sensors are biased or attacked. Context measurements are assumed to have a known probability of occurring given the state; in particular, we focus on the probit function to model threshold-based measurements, such as the medical-alarm scenario. We develop a recursive context-aware filter by approximating the posterior distribution with a Gaussian distribution with the same first two moments as the true posterior. We show that the filter's expected uncertainty is boundedwhen the probability of receiving context measurements is lower bounded by some positive number for all system states. Furthermore, we provide an observability-like result-all eigenvalues of the filter's covariance matrix converge to 0 after repeated updates if and only if a persistence of excitation condition holds for the context measurements. Finally, in addition to simulation evaluations, we applied the filter to the problem of estimating a patient's blood oxygen content during surgery using real-patient data.