In multi-state life insurance, an adequate balance between analytic tractability, computational efficiency, and statistical flexibility is of great importance. This might explain the popularity of Markov chain modelling, where matrix analytic methods allow for a comprehensive treatment. Unfortunately, Markov chain modelling is unable to capture duration effects, so this paper presents aggregate Markov models as an alternative. Aggregate Markov models retain most of the analytical tractability of Markov chains, yet are non-Markovian and thus more flexible. Based on an explicit characterization of the fundamental martingales, matrix representations of the expected accumulated cash flows and corresponding prospective reserves are derived for duration-dependent payments with and without incidental policyholder behaviour. Throughout, special attention is given to a semi-Markovian case. Finally, the methods and results are illustrated in a numerical example.