Preprint, 2026

Language reshapes experiential learning in humans and machines

Ellen Su, Divya Srinivasan, Mark K. Ho, Todd M. Gureckis · New York University

Human decision making draws on both direct evidence, which is accrued incrementally over time, and natural language, which arrives rapidly and is often abstract in meaning. In deciding where to get coffee, for example, you may weigh your own past experiences against a friend’s recommendation that “Coffee Project is the best.” In this project, we are interested in how these partial, verbal hints reshape what we have learned from experience in a seamless manner to guide everyday decisions.

We augment the multi-armed bandit with natural-language hints about a task's structure, parameters, or policy, creating the Instructed Bandit Tasks (IBT). Across two large experiments, we ask how hints change people's choices and compare them to two models. A hierarchical Bayesian agent treats each hint as a constraint on its beliefs, and Centaur is a language model fine-tuned on human behaviour.

Explore the tasks, play the games alongside the models, and see the results.

Instructed Bandit Tasks

The IBT are eight bandit tasks. In each, the player presses one of five buttons to earn points and may be given a hint about the task. Each button is drawn here as a slot machine, and pulling it pays points from a distribution the player never sees. Watch a game play out, or click a machine to pull it yourself.

Possible hints

    Players

    People

    Participants played up to 15 games, each with a random task and hint. Pick a task and a hint, then press buttons to earn as many points as you can.

      Bayesian agent

      The agent holds beliefs about which of the eight tasks it is in and what each button is worth, and updates both after every reward. A hint keeps only the worlds in which it is true. The task is drawn at random and hidden, as it is for people. Press buttons and watch the beliefs sharpen.

      \[ \underbrace{P(\tau,\, \mu_{1:N},\, \theta_\tau \mid h_t)}_{\text{Posterior}} \;\propto\; \underbrace{P(\tau)}_{\text{Type prior}} \; \underbrace{P(\mu_{1:N},\, \theta_\tau \mid \tau)}_{\text{Parameter prior}} \; \underbrace{P(h_t \mid \tau,\, \mu_{1:N},\, \theta_\tau)}_{\text{Likelihood}} \; \underbrace{\exp\{\gamma\, f_{l}(\tau,\, \mu_{1:N})\}}_{\text{Hint weight}} \]

      Centaur

      Centaur is a language model fine-tuned on human behaviour. It reads the game as text, hint and history included, and answers with a button number. Run one of its recorded games.

      What Centaur reads

      
          

      What it answers

      Results

      1. Hints improve learning

      People learn each task from experience alone, and hints improve performance on most task types. Both models reproduce the benefit.

      2. Hints modulate performance differently in each task type

      The same hint was roughly twice as beneficial in the time delayed task as in the stationary one, and one strictly true hint in the patch foraging task reliably increased regret. Both models predicted which hints help most, but only Centaur predicted the interference.

      3. Providing hints after experience alters their effect

      People integrate hints differently depending on prior experience. Value hints helped only when introduced at the start of the game. Centaur largely reproduced this sensitivity to timing, whereas the Bayesian model, which treats a hint as accurate evidence whenever it arrives, did not.