Coffee and breakfast served in the Amy Gutmann Hall Lobby. Refreshments provided by Philadelphia Catering.

Symposium opens in Amy Gutmann Hall Auditorium

Title: Neurosymbolic AI at AWS

Abstract: This talk will discuss work at AWS to bring the power of agentic/generative AI with the safety and correctness of formal reasoning.

Refreshments provided by Philadelphia Catering.

Surbhi Goel, “Reliable Learning via Abstention”

Abstract: When a learning system is deployed, the data it encounters may no longer resemble its training data, and without assumptions on the shift, reliable prediction is impossible in general. This talk studies a simple response: allow the learner to abstain when it lacks sufficient evidence, with the guarantee that whenever it does predict, it is usually correct. I will show how abstention enables reliability under arbitrary distribution shift in settings ranging from classification to sequential decision-making, and discuss the statistical and computational questions that arise.

Hamed Hassani, “Rethinking Uncertainty for Trustworthy Human-AI Interaction”

Abstract: Interaction with AI systems is now happening at a massive scale. People engage in multi-round exchanges with AI to guide real-world decisions, and increasingly, AI agents act on our behalf, drawing on humans and tools for support. Yet despite the prevalence of these interactions, we still lack fundamental principles that should govern them: what makes an interaction trustworthy, and how can we ensure it reliably improves the resulting decisions? In this talk, I will discuss how rethinking uncertainty as a property of the interaction, rather than of the AI system alone, can help answer these questions. For human-AI collaboration, this means reasoning about how AI changes the decisions a person would make on their own. For agentic AI, it means reasoning about when support from humans or tools can meaningfully improve an agent’s decisions. I will show how this perspective leads to principled frameworks for designing reliable interactions across different humans, AI models, and forms of support.

Osbert Bastani, “A Programming Language for LLM Agents”

Abstract: Large language models (LLMs) often call external tools to solve tasks. One effective strategy is for LLMs to write code, enabling them to use complex control flow such as conditionals and loops. We propose Quasar, a novel programming language designed for more effective code actions. This language separates internal code capturing program logic from external calls to tools that interact with the world. New features can then easily be implemented by (1) annotating external calls with the effects relevant to that feature, and (2) modifying the execution of the internal code to track this information. To illustrate its utility, we implement several useful features on top of Quasar to enhance code actions: access control with batched user queries to improve security, autoparallelization of external calls to reduce latency, and conformal prediction for uncertainty quantification to mitigate hallucinations.

“Omniprediction with Long-term Constraints,” Jiuyao Lu, 2026 Fellow

“Frequency-Guide Posterior Sampling for Diffusion-Based Image Restoration,” Darshan Thaker, 2024 Fellow

“Learning to Wait in Sequential Decision Making,” Chris Watson, 2025 Fellow

With posters presented by: 

Kwan Ho Ryan Chan

Seewon Choi

Cassandra Goldberg

Helen Jin

Mayank Keoliya

Jiuyao Lu

Josh Magnus Ludan

Nicolas Menand

Alex Shypula

Alaia Solko-Breslin

Darshan Thaker

Stephanie Wang

Chris Watson

Honam Wong

Zixuan Yi

Lunch provided by Philadelphia Catering.

Title: When the Adversary Is the Environment: Attacks and Defenses for LLM Agents

Abstract: The rise of LLM agents marks a shift in the attack paradigm from conversational models that primarily generate content to agentic systems that can plan, invoke tools, and modify persistent state. As a result, the security question is no longer simply whether an LLM can be induced to produce unsafe output, but whether untrusted content can manipulate an agent’s actions, cross privilege boundaries, or compromise state that influences future decisions. Drawing on two of our works, this talk presents two attacks on web agents that turn the environment itself into an attack surface, where adversarial manipulation of webpage content can alter how an agent perceives and acts on the world. The first exploits untrusted environmental observations to induce covert exfiltration of private user data to attacker-controlled channels while preserving successful completion of the user’s intended task. The second formulates the attack as a black-box optimization problem, learning adversarial manipulations of the web environment from feedback provided by the target agent to induce attacker-specified actions. Standard prompt-based defenses provide limited protection against both attacks. The talk then places these attacks within the broader landscape of agentic attacks and defenses, highlighting how securing agents increasingly becomes a systems problem that calls for safeguards at the level of architecture and execution, and closes with open questions in defense design, evaluation, and multi-agent settings.

Aaron Roth, “Agreement and Alignment for Human-AI Collaborative Decision Making”

Abstract: As AI models become increasingly powerful, it is an attractive proposition to use them in important decision making pipelines, in collaboration with human decision makers. But how should a human being and a machine learning model collaborate to reach decisions that are better than either of them could achieve on their own? If the human and the AI model were perfect Bayesians, operating in a setting with a commonly known and correctly specified prior, Aumann’s classical agreement theorem would give us one answer: they could engage in conversation about the task at hand, and their conversation would be guaranteed to converge to (accuracy-improving) agreement. This classical result however would require making many implausible assumptions, both about the knowledge and computational power of both parties. We show how to recover similar (and more general) results using only computationally and statistically tractable assumptions, which substantially relax full Bayesian rationality. In the second part of the talk, we go on to consider a more difficult problem: that the AI model might be acting at least in part to advance the interests of its designer, rather than the interests of its user, which might be in tension. We show how market competition between different AI providers can mitigate this problem assuming only a mild “market alignment” assumption — that the user’s utility function lies in the convex hull of the AI providers utility functions — even when no single provider is well aligned. In particular, we show that in all Nash equilibria of the AI providers under this market alignment condition, the user is able to advance her own goals as well as she could have in collaboration with a perfectly aligned AI model. 

This talk describes the results of three papers, which are joint works with Natalie Collina, Ira Globus-Harris, Surbhi Goel, Varun Gupta, Emily Ryu, and Mirah Shi: 

Tractable Agreement Protocols: https://arxiv.org/abs/2411.19791 (STOC 2025) 

Collaborative Prediction: Tractable Information Aggregation via Agreement: https://arxiv.org/abs/2504.06075 (SODA 2026)

Emergent Alignment from Competition: https://arxiv.org/abs/2509.15090

Eric Wong, “Understanding Mechanisms of Alignment”

Abstract: LLMs are full of strange and unpredictable behaviors. They can be manipulated with strange characters and forget instructions after a single turn. How can this happen? In this talk, we will model such problems with mechanistic theory: an analytical approach that studies small transformers to understand these issues in the case of perfect reasoners. We will then discuss how these results generalize empirically to large language models and how these insights can guide practice.

Mayur Naik, “MathDuels: A Self-Play Benchmark That Grows”

Abstract: As frontier language models attain near-ceiling performance on static mathematical benchmarks, existing evaluations are increasingly unable to differentiate model capabilities, largely because they cast models solely as solvers of fixed problem sets. I will present MathDuels, a self-play benchmark in which models occupy dual roles: each authors math problems under adversarial prompting and solves problems authored by every other participant. A generation pipeline produces and filters the problems, and a psychometric model jointly estimates solver abilities and problem difficulties, from which author quality is derived. Experiments across 19 frontier models reveal that authoring and solving capabilities are partially decoupled, and that dual-role evaluation exposes capability separations invisible in single-role benchmarks. As newer models enter the arena, they produce problems that defeat previously dominant solvers, so the benchmark’s difficulty co-evolves with participant strength rather than saturating at a fixed ceiling. A public leaderboard at mathduels.ai is updated as new models are released.

“Efficient and Sound Probabilistic Verification for AI Agents,” Alaia Solko-Breslin, 2024 Fellow

“Evaluating the Diversity and Quality of LLM Generated Content,” Alex Shypula, 2025 Fellow

“Locally Approximating the Top Eigenvector of Bounded Entry Matrices,” Nicolas Menand, 2025 Fellow

“Data Canvas: A Provenance-Guided Harness for Agentic Data Engineering,” Zixuan Yi, 2026 Fellow

With posters presented by:

Amogh Ananda Rao

Raghu Argal

Vivienne Chi

Nick-Marios Kokolakis

Shirley Li

Yuanheng Li

Buyun Liang

Nghia Nguyen

Neil Sehgal

Satvik Tripathi

Jiayi Xin

Ahhyun Yuh

Gaoxiang Zhao

Rhea Zhou

Wine and refreshments served by Philadelphia Catering.