The past decade has seen a growing trend of applying probability theory to intelligence systems that deal with complex real-world data with rich semantic structure and temporal and/or spatial dynamics. Probabilistic graphical model is a formalism that exploits the conjoined talents of graph theory and probability theory to build complex models out of simpler pieces, which offers a powerful language to elegantly define expressive distributions under complex scenarios, and provide a systematic computational framework for probabilistic inference. I discuss the mathematical underpinnings for recent developments in graphical models–including hierarchical and nonparametric Bayesian modeling, approximate inference, and relationships to other machine learning methods such as kernel machines and maximum margin learning–which lie in the theory of Bayesian statistics and convex analysis. I also discuss a number of applications: (1) Modeling topic evolution in document collections, (2) Unraveling actor functions in social networks, (3) Finding regulatory elements in DNA sequences, and (4) Reconstructing evolutionary history of human populations.