Some problems in the theory and applications of Markov chains

Abstract

This thesis, written during my two-year term between January 1954 and January 1956 as a Research student of the Australian National University, consists of some work carried out during 1954 and early 1955. Sections of Chapter 2, and all of Chapters 3, 4, 6, 7, 8, are either published or in process of publication, and are available in a slightly different form in Biometrika (1955, 42), and the Australian Journal of Applied Science (1955, 6); Chapter 5 is also being submitted for publication. I feel it is safe to claim the greater part of the thesis as original work; but, in the circumstances, it would perhaps be more appropriate to specify the original parts of each chapter in some detail. Chapter 1 is a review of recent contributions to the field considered; to this no great originality can be ascribed. Chapter 2 consists of a connected account of those properties of Markov chains, most of them well known, which are required in the remaining parts of the thesis. My own contributions to this chapter are: 1) the somewhat new presentation of the proof in 2.5, p. 30-33, that the latent root 1 of the stochastic matrix for a regular chain is simple; 2) the new theorem in 2.7, p.39-42, derived from a theorem of Frechet; 3) the entirely new theorem in 2.9, p. 44-47, for the latent roots of the matrix R={Pij exp tij}. The work in Chapters 3, 4, 5, 6, 7, though based in part on suggestions of professor P.A.P. Moran, is fully my own. Naturally, some results due to other authors are used, or briefly summarised, but these are always clearly acknowledged. Finally Chapter 8 was prepared in collaboration with Professor Moran; each of us can, I think, claim an equal share of the work. I began by working out the discrete dam probably by Monte Carlo methods; Professor Moran redrafted the entire chapter, eliminated several errors and added various improvements. The final part8.4 on the continuous dam problem is entirely his and his only been included for the sake of completeness.