This thesis covers work in mathematical and statistical aspects of population genetics.
In the first part of the thesis, we present the behaviour of two F -statistics, which are defined as the difference in the allele frequency at a given time point in one population and the difference in allele frequency between two populations, respectively. In order to calculate the first two moments of the allele frequency present in the mathematical expression of the F -statistics, mutation and migration as linear evolutionary forces are incorporated into Wright-Fisher model. We give some parameter conditions that cause the behaviour of the F -statistics to be non-monotonic over time, that is, to increase and then decrease over time.
In the second part of the thesis, we propose a new method to evaluate the statistical fit of principal component analysis (PCA) in admixture models and population structure inference.
Statistical tools such as residual and correlation coefficients are utilized to detect violations of model assumptions. We give the mathematical expressions of two correlation coefficient matrices of the residuals and some theorems about their properties. The method is demonstrated in both simulated and real data through matrix visualization.
Finally, I introduce a mathematical definition and estimate of kinship coefficient based on pedigree and genotype data. I propose a procedure for dividing the sample containing related individuals who have alleles copied from a common ancestor into some data sets. Each data set corresponds to the individual under study and is used in PCA method to estimate the allele frequency of the studied individual present in the kinship coefficient. In the presence of population structure, the proposed estimate performs well for the case of full-siblings.