Large language models are being asked to generate longer and longer
outputs. A reasoning model may think for tens of thousands of tokens
before answering a math problem
(DeepSeek-R1), and a coding agent may
run for hours. To generate each of those tokens, the model has to look
ba…
LLMs are advanced AI for tasks like text and code generation. As LLMs scale towards being distributed, efficiently serving LLM requires multiple hardware units to work together. To manage how LLMs process user requests across these resources, developers use different strategies —…
Modern machine learning (ML) kernels evolve rapidly with new optimizations and hardware features. ML compilers must accommodate richer memory hierarchies, more forms of thread cooperation, and backends with different execution models, while retaining the control needed for peak p…
The allocation of scarce donor organs constitutes one of the most consequential algorithmic problems in healthcare. Organ transplantation remains in many cases the only viable option for critically ill patients. Unfortunately, demand far outweighs existing supply: in the United …
Long-context understanding is a critical frontier for large language models (LLMs). Many of the most valuable applications, such as code generation, processing entire books, and modeling DNA sequences, either require long-context understanding to work well or become dramatically …
This post is based on our write-up Model self-identification could be subliminally transferred. Our implementation is provided in the accompanying repository.
Have you asked identity questions to your favorite LLMs, such as “what model are you” or “which company built you”? The a…
It is 2040, and you’re a hungry, busy PhD student in your office who has to attend a talk in thirty minutes. You place a DoorDash order from your favorite restaurant. An autonomous robot picks up your order from a restaurant in a mall. It then drives itself from the mall to your …
Randomness is a fundamental resource used in computer science for a variety of applications, including stochastic simulation, statistical modeling, AI, machine learning, and cryptography.
A prominent application of random sampling is text generation with large language models, wh…
Ask a modern image generator for a picture of an arctic fox, and behind the scenes it will start from pure random noise and repeatedly nudge that noise toward an image — often running a large neural network 50, 100, or even 250 times in sequence before a single picture emerges. T…
Suppose a hospital runs a statistical analysis over millions of patient records to determine whether a drug is effective. The computation takes days of cluster time, and the raw data cannot be shared due to privacy regulations. A regulatory agency needs to trust the result—but …
