In this article we address the problem of realizing a service-providing reasoning infrastructure for pro-active humanassistance in intelligent environments. We propose SAM, an architecture which leverages temporal knowledge represented asrelations in Allen’s interval algebra and constraint-based temporal planning techniques. SAM provides two key capabilities forcontextualized service provision: human activity recognition and planning for controlling pervasive actuation devices. Whiledrawing inspiration from several state-of-the-art approaches, SAM provides a unique feature which has thus far not been addressedin the literature, namely the seamless integration of these two key capabilities. It does so by leveraging a constraint-basedreasoning paradigm whereby both requirements for recognition and for planning/execution are represented as constraints andreasoned upon continuously.