In any life and pension insurance company, it is a central task to calculate the value of the liabilities toward the policyholders. In the classic model for such valuations, a continuous time Markov chain in a finite state space describes the state of the insured, and the interest rate, mortality rate, disability rate, and other transition rates are assumed to be deterministic. Broadly speaking this PhD thesis consists of various extensions of this model to address the modern needs of life insurance companies. These extensions can be categorised into two types: the inclusion of policyholder behaviour in the model, and the modelling of the interest and transition rates as stochastic processes.