Understanding Bayesian probability?

[Evans]

At the moment, I'm particularly interested in dwelling into the analysis, theories and mathematical foundations to being acquainted with the basis of Bayesian probability. What is a good step-by-step mathematical foundation demanded from a person who lacks the rigor of complex mathematics?

[Dilaudid]

My favourite book on it is Information Theory, Inference and Learning Algorithms by David MacKay. You need to know maths pretty well (differentiation, integration, exponentials a must) but it's free online so take a look. It explains Bayesian probability very well in chapters 2 and 3 (including things like the Monty Hall problem).