Feras Saad فِرَاسْ سَعَد

I am an Assistant Professor in the Computer Science Department at Carnegie Mellon University, affiliated with the Principles of Programming and Artificial Intelligence groups.

My research lab explores the design and implementation of scalable programming systems that enable flexible probabilistic modeling and sound probabilistic inference. We develop these ideas through a blend of theoretical and applied research in probabilistic programming languages; artificial intelligence; statistics and data science; and the mathematical foundations of computing with randomness.

Our projects are associated with freely available software libraries that help diverse practitioners use these systems in their own applications.

See research, publications and software for an overview.

Background

I received the PhD in Computer Science, and MEng/SB degrees in Electrical Engineering and Computer Science, from MIT. My theses on probabilistic programming were recognized with the Sprowls PhD Thesis Award in AI+Decision Making and Johnson MEng Thesis Award in Computer Science. Prior to CMU, I was a Visiting Research Scientist at Google.

Contact

Research

Our work combines ideas from probability and programming languages to build scalable systems for sound probabilistic modeling and inference. Current research themes include:

Probabilistic Programming Languages. Researchers from diverse fields―such as econometrics, cognitive science, and bioinformatics―use probabilistic modeling and inference to make sense of uncertain data. Probabilistic programming languages (PPLs) have emerged as a powerful and unified approach to modeling and inference over rich data-generating processes. PPLs aim to rigorously specify the semantics of computer-defined probability models, and automate and scale up the process of learning these models from data. [OOPSLA-24] [PLDI-24] [PLDI-21] [POPL-19] [PLDI-19] [AISTATS-17] [NIPS-16]

Automated Probabilistic Model Discovery. A grand challenge in AI is automatically discovering accurate and interpretable models let us to reason about observed data. I am interested in studying this problem through the lens of probabilistic program synthesis, formalized as Bayesian structure learning over expressive generative model families. [Nature Commun-24] [ICML-23] [UAI-21] [POPL-19] [AISTATS-18]

Statistical Estimation & Testing. As probabilistic programs become more widespread, new techniques are needed to infer their statistical properties through the computational inferences that they expose. These methods integrate static or exact program analysis with simulation-based data-driven analysis of sampled execution traces. [PLDI-24] [AISTATS-22] [AISTATS-19]

Random Variate Generation. Generating random variables is a fundamental operation in computer science that enables probabilistic programming and randomized algorithms. I am interested in the computational and information-theoretic limits of random sampling, and developing new methods that are optimal (in entropy consumption and/or sampling error) while also extremely efficient in practice.
[POPL-20] [AISTATS-20]

Software + Applications. A central motivation of our lab is to develop performant, freely available software systems that make probabilistic inference more broadly accessible to everyday users while also helping domain-experts solve meaningful problems in the sciences, engineering, and public interest. [Nature Commun-24] [eLife-2019]

Probabilistic Computing Systems Lab

Gaurav Arya
Graduate Student, 2024–
Carnegie Mellon University
Interests: scientific computing, probabilistic programming

Thomas Draper
Graduate Student, 2024–
Carnegie Mellon University
Interests: probabilistic algorithms, information and computation

Alumni

Dr. Wonyeol Lee
Postdoctoral Associate, 2023–2024
→ Assistant Professor, POSTECH
PhD, Stanford University, 2023
Interests: continuous computation, program correctness

Jesse Michel
Visiting Graduate Student, 2024
Massachusetts Institute of Technology
Interests: differentiable programming, physical simulation

Publications

Google Scholar | dblp | ORCID | BibTeX

• Scalable Spatiotemporal Prediction with Bayesian Neural Fields
  Saad, Burnim, Carroll, Patton, Köster, Saurous, Hoffman
  Nature Communications 15(7942), 2024
  Editor's Highlight
  link | paper | code | BibTeX

• Programmable MCMC with Soundly Composed Guide Programs
  Pham, Wang, Saad, Hoffmann
  OOPSLA, Proc. ACM Program. Lang. 8(OOPSLA2), 2024
  paper | link | artifact | BibTeX

• Robust Resource Bounds with Static Analysis and Bayesian Inference
  Pham, Saad, Hoffmann
  PLDI, Proc. ACM Program. Lang. 8(PLDI), 2024
  paper | artifact | link | BibTeX

• GenSQL: A Probabilistic Programming System for Querying Generative
  Models of Database Tables
  Huot, Ghavami, Lew, Schaechtle, Freer, Shelby, Rinard, Saad, Mansinghka
  PLDI, Proc. ACM Program. Lang. 8(PLDI), 2024
  paper | artifact | link | code | BibTeX

• Learning Generative Population Models From Multiple Clinical Datasets
  via Probabilistic Programming
  Loula, Collins, Schaechtle, Tenenbaum, Weller, Saad, O'Donnell, Mansinghka
  AccMLBio, ICML Workshop on Efficient and Accessible Foundation Models for Biological Discovery, 2024
  paper | link | BibTeX

• Sequential Monte Carlo Learning for Time Series Structure Discovery
  Saad, Patton, Hoffman, Saurous, Mansinghka
  ICML, Proc. 40th International Conf. on Machine Learning, 2023
  paper | link | code | BibTeX

• Scalable Structure Learning, Inference, and Analysis with Probabilistic Programs
  Saad
  PhD Thesis, Massachusetts Institute of Technology, 2022
  MIT George M. Sprowls PhD Thesis Award
  

• Estimators of Entropy and Information via Inference in Probabilistic Models
  Saad, Cusumano-Towner, Mansinghka
  AISTATS, Proc. 25th International Conf. on Artificial Intelligence and Statistics, 2022
  paper | link | arXiv | code | BibTeX

• Bayesian AutoML for Databases via the InferenceQL Probabilistic Programming System
  Schaechtle, Freer, Shelby, Saad, Mansinghka
  AutoML, 1st International Conf. on Automated Machine Learning (Workshop), 2022
  paper | link | BibTeX

• SPPL: Probabilistic Programming with Fast Exact Symbolic Inference
  Saad, Rinard, Mansinghka
  PLDI, Proc. 42nd International Conf. on Programming Design and Implementation, 2021
  paper | artifact | link | arXiv | code | BibTeX

• Hierarchical Infinite Relational Model
  Saad, Mansinghka
  UAI, Proc. 37th Conf. on Uncertainty in Artificial Intelligence, 2021
  Oral Presentation
  paper | link | arXiv | code | BibTeX

• The Fast Loaded Dice Roller: A Near Optimal Exact Sampler for Discrete Probability Distributions
  Saad, Freer, Rinard, Mansinghka
  AISTATS, Proc. 24th International Conf. on Artificial Intelligence and Statistics, 2020
  paper | supplement | link | arXiv | code | BibTeX

• Optimal Approximate Sampling from Discrete Probability Distributions
  Saad, Freer, Rinard, Mansinghka
  POPL, Proc. ACM Program. Lang. 4(POPL), 2020
  paper | supplement | artifact | link | code | arXiv | BibTeX

• A Family of Exact Goodness-of-Fit Tests for High-dimensional Discrete Distributions
  Saad, Freer, Ackerman, Mansinghka
  AISTATS, Proc. 23rd International Conf. on Artificial Intelligence and Statistics, 2019
  paper | supplement | link | arXiv | BibTeX

• Bayesian Synthesis of Probabilistic Programs for Automatic Data Modeling
  Saad, Cusumano-Towner, Schaechtle, Rinard, Mansinghka
  POPL, Proc. ACM Program. Lang. 3(POPL), 2019
  paper | supplement | link | arXiv | BibTeX

• Gen: A General Purpose Probabilistic Programming System with Programmable Inference
  Cusumano-Towner, Saad, Lew, Mansinghka
  PLDI, Proc. 40th International Conf. on Programming Design and Implementation, 2019
  paper | link | code | BibTeX

• Elements of a Stochastic 3D Prediction Engine in Larval Zebrafish Prey Capture
  Bolton, Haesemeyer, Jordi, Schaechtle, Saad, Mansinghka, Tenenbaum, Engert
  eLife 8:e51975, 2019
  paper | BibTeX

• Temporally-Reweighted Chinese Restaurant Process Mixtures
  for Clustering, Imputing, and Forecasting Multivariate Time Series
  Saad, Mansinghka
  AISTATS, Proc. 21st International Conf. on Artificial Intelligence and Statistics, 2018
  paper | supplement | link | code | BibTeX

• Goodness-of-Fit Tests for High-dimensional Discrete Distributions
  with Application to Convergence Diagnostics in Approximate Bayesian Inference
  Saad, Freer, Ackerman, Mansinghka
  AABI, 1st Symposium on Advances in Approximate Bayesian Inference, 2018
  paper | link | BibTeX

• Detecting Dependencies in Sparse, Multivariate Databases
  Using Probabilistic Programming and Non-parametric Bayes
  Saad, Mansinghka
  AISTATS, Proc. 20th International Conf. on Artificial Intelligence and Statistics, 2017
  paper | supplement | link | arXiv | code | BibTeX

• Probabilistic Search for Structured Data via Probabilistic Programming
  and Nonparametric Bayes
  Saad, Casarsa, Mansinghka
  PROBPROG, 1st International Conf. for Probabilistic Programming, 2018
  arXiv | code | BibTeX

• Time Series Structure Discovery via Probabilistic Program Synthesis
  Schaechtle*, Saad*, Radul, Mansinghka
  PROBPROG, 1st International Conf. for Probabilistic Programming, 2018
  arXiv | BibTeX

• A Probabilistic Programming Approach to Probabilistic Data Analysis
  Saad, Mansinghka
  NIPS, Proc. 30th Conf. on Neural Information Processing Systems, 2016
  paper | link | code | BibTeX

• Probabilistic Data Analysis with Probabilistic Programming
  Saad, Mansinghka
  arXiv, Technical Report arXiv:1608.05347, 2016
  Extended version of NIPS 2016.
  arXiv | BibTeX

• Probabilistic Data Analysis with Probabilistic Programming
  Saad
  MEng Thesis, Massachusetts Institute of Technology, 2016
  MIT Charles & Jennifer Johnson MEng Thesis Award, 1st Place
  

Teaching

Carnegie Mellon University

Massachusetts Institute of Technology

Software

GitHub: https://github.com/probsys

• BayesNF: Bayesian neural fields for spatiotemporal data modeling
  https://github.com/google/bayesnf

• GenSQL: Probabilistic query language for generative models of database tables
  https://github.com/OpenGen/GenSQL.query

• AutoGP.jl: Automated Bayesian model discovery for time series data
  https://github.com/probsys/AutoGP.jl

• EEVI: Estimating entropy and mutual information in probabilistic programs
  https://github.com/probsys/eevi

• HIRM: Bayesian nonparametric structure learning for relational systems
  https://github.com/probsys/hierarchical-irm

• SPPL: Probabilistic programming language with fast exact symbolic inference
  https://github.com/probsys/sppl

• FLDR: Fast exact sampler for discrete probability distributions
  https://github.com/probsys/fast-loaded-dice-roller
  See also: code generator by Peter Occil.

• OPTAS: Optimal limited-precision sampler for discrete distributions
  https://github.com/probsys/optimal-approximate-sampling

• TRCRPM: Bayesian method for clustering, imputing, and forecasting
  https://github.com/probcomp/trcrpm

• Gen.jl: General-purpose probabilistic programming with programmable inference
  https://gen.dev

• CGPM: Library of composable probabilistic generative models
  https://github.com/probcomp/cgpm
  

• BayesDB: Probabilistic programming for data analysis on sqlite
  https://github.com/probcomp/bayeslite

Sublime Text Users: Productivity plugins (9000+ users):
AddRemoveFolder; RemoveLineBreaks; ViewSetting.

Talks

• Scalable Spatiotemporal Prediction with Bayesian Neural Fields
  UAI 2024 Workshop on Tractable Probabilistic Modeling, Barcelona, Spain
  

• Automated Gaussian Processes and Sequential Monte Carlo
  Learning Bayesian Statistics Podcast, Episode #104, 2024, Virtual
  

• Keynote: Domain-Specific Probabilistic Programs for Time Series Modeling
  Google Forecasting Summit 2023, Mountain View, CA
  

• Programmable Systems for Probabilistic Modeling and Inference
  SCS Faculty Lightning Talks 2023, Virtual
  

• Scalable Structure Learning and Inference via Probabilistic Programming
  MIT Thesis Defense, 2022, Virtual
  MIT Thesis Defense, 2022, Cambridge, MA
  

• Scalable Structure Learning and Inference for Domain-Specific Probabilistic Programs
  LAFI 2022, Philadelphia, PA
  

• SPPL: Probabilistic Programming with Fast Exact Symbolic Inference
  PROBPROG 2021, Virtual
  SPLASH 2021, Chicago, IL
  MIT PLSE Seminars 2021, Virtual
  PLDI 2021 [extended] [lightning], Virtual

• Fairer and Faster AI Using Probabilistic Programming Languages
  MIT Horizon 2021, Virtual

• Hierarchical Infinite Relational Model
  UAI 2021 (Oral Presentation), Virtual

• Fast Loaded Dice Roller
  AISTATS 2020, Virtual

• Optimal Approximate Sampling from Discrete Probability Distributions
  POPL 2020, New Orleans, LA

• Bayesian Synthesis of Probabilistic Programs for Automatic Data Modeling
  CICS Seminars 2019, Notre Dame, IN
  POPL 2019, Cascais, Portugal
  PROBPROG 2018, Boston MA

Press

• MIT researchers introduce generative AI for databases. MIT News, Jul. 2024

• Estimating the informativeness of data. MIT News, Apr. 2022

• Exact symbolic AI for faster, better assessment of AI fairness. MIT News, Aug. 2021

• How and why computers roll loaded dice. Quanta Magazine, Jul. 2020

• How Rolling Loaded Dice Will Change the Future of AI. Intel Press Release, Jul. 2020

• Algorithm quickly simulates a roll of loaded dice. MIT News, Jan. 2020

• MIT Debuts Gen, a Julia-Based Language for Artificial Intelligence. InfoQ, Jul. 2019

• New AI programming language goes beyond deep learning. MIT News, Jun. 2019

• Democratizing data science. MIT News, Jan. 2019

• MIT lets AI ``synthesize'' computer programs to aid data scientists. ZDNet, Jan. 2019