Knowledge-driven activity recognition has recently attracted increasing attention but mainly focused on simple activities. This paper extends previous work to introduce a knowledge-driven approach to recognition of composite activities such as interleaved and concurrent activities. The approach combines ontological and temporal knowledge modelling formalisms for composite activity modelling. It exploits ontological reasoning for simple activity recognition and rule-based temporal inference to support composite activity recognition. The presented approach has been implemented in a prototype system and evaluated in a number of experiments. The initial experimental results have shown that average recognition accuracy for simple and composite activities is 100% and 88.26%, respectively.