This paper introduces a knowledge-driven approach to real-time, continuous activity recognition based on multi-sensor data streams in smart homes. The approach goes beyond the traditional data-centric methods for activity recognition in three ways. Firstly, it makes extensive use of domain knowledge in the lifecycle of activity recognition. Secondly, it uses ontologies for explicit context and activity modeling and representation. Thirdly and finally, it exploits semantic reasoning and classification for activity inferencing, thus enabling both coarse-grained and fine-grained activity recognition. In this paper we analyse the characteristics of smart homes and Activities of Daily Living (ADL) upon which we built both context and ADL ontologies. We present a generic system architecture for the proposed knowledge-driven approach and describe the underlying ontology-based recognition process. Special emphasis is placed on semantic subsumption reasoning algorithms for activity recognition. The proposed approach has been implemented in a function-rich software system, which was deployed in a smart home research laboratory. We evaluated the proposed approach and the developed system through extensive experiments involving a number of various ADL use scenarios. An average activity recognition rate of 94.44% was achieved and the average recognition run-time per recognition operation was measured as 2.5 seconds.