←→ prev / next   ⇧←→ siblings   ↑ parent   ↓ child   1–9 n-th child   M map   B backup   K hide this

Test-time training and memory for robot policies

RoboTTT (Jiang et al., NVIDIA / Stanford / UT Austin, July 2026)
T²Mem (Liu et al., Stanford / NVIDIA / Michigan, September 2026)

Nir Pechuk · Continual learning reading group

How can we learn from our test-time experience?

Outer loop · slow weights $\theta$ and $W_0$ · gradient descent on $\mathcal{L}_{\text{outer}}$ over training sequences
Inner loop · fast weights $W_t$ · gradient descent on $\mathcal{L}_{\text{inner}}$, at train and test time
$W_0 \to W_1 \to \cdots \to W_t \to \cdots$
$$W_t \leftarrow W_{t-1} - \eta\,\nabla_W\,\mathcal{L}_{\text{inner}}\!\big(f_{W_{t-1}}(K_t),\,V_t\big)$$
(inner: what to remember)
$$O_t = f_{W_t}(Q_t)$$
(apply: what to retrieve)
$$\min_{\theta,\,W_0}\ \mathbb{E}\Big[\mathcal{L}_{\text{outer}}\big(\text{task};\ \theta,\ W_T(\theta, W_0)\big)\Big]$$
(outer: what memory is for)

Choosing $\mathcal{L}_{\text{inner}}$ and $\mathcal{L}_{\text{outer}}$ is choosing how we hope to use test-time experience.

Past talks: ways LLMs use test-time experience

LLM paperHow we use past experienceInner objectiveOuter objectiveRobotics challenge
TTT Layers / TitansRemember contextReconstruct the current token's features.Use input and memory for NTP.Non-Markovian or partially observed tasks.
In-Place TTTEffectively use context for NTPReconstruct the next token's features.
Few-Shot TTTLearn from examplesPredict the examples.–Few-shot learning and shifting tasks
FTTTLearn from feedbackPredict the feedback.Use feedback-prediction gradients to optimize solutions.Adapting from failures
TTT-DiscoverIncrease probability of breakthroughsMaximize $\mathbb{E}[\exp(\text{reward})]$.–Discovering a successful strategy through scoring

Highlighted: what today's papers look like mechanically.

In robotics, the feedback is the context is the examples.

one deployment, one stream of observations and actions a human video of the task the stages I already finished my own failed attempt an example context feedback the same self-supervised fast-weight update no labels, no reward, no separate mechanism

Preliminaries: a VLA with a flow-matching action head.

images ot instruction l proprio qt VLM one forward pass hidden states Φt keys and values for every expert layer action expert cross-attends to Φt proprio qt, noised chunk Aτ, flow time τ velocity vθ k denoising steps, τ: 0 → 1 after k steps At = [at … at+H−1], execute (part of) it, observe ot+1
$$\pi_\theta\big(A_t \mid o_t, q_t, l\big),\quad A_t = [a_t,\dots,a_{t+H-1}]$$
(a chunk of H actions)
$$\textcolor{#B4722A}{A^{\tau}} = \tau A + (1-\tau)\,\epsilon,\qquad \epsilon\sim\mathcal N(0, I)$$
(noising; τ = 1 is clean data)
$$\mathcal L = \mathbb E_{\tau,\epsilon}\,\big\|\textcolor{#B4722A}{v_\theta}\big(\textcolor{#B4722A}{A^{\tau},\tau},\textcolor{#2A7B76}{\Phi_t},q_t\big) - (A-\epsilon)\big\|^2$$
(flow-matching loss: predict the velocity)
$$\textcolor{#B4722A}{A^{\tau+\delta} = A^{\tau} + \delta\, v_\theta(\cdot)},\qquad \tau: 0\to 1 \text{ in } k \text{ steps}$$
(inference, k ≈ 4–10 steps)

RoboTTT

Make TTT-layers work for robot policies.

T²Mem

Apply robotic inductive biases.

Conclusions and my thoughts

RoboTTT. Idea: apply the TTT-layers objective to a robot policy, using gating so we can start from a pretrained policy.

  1. Mechanismgated fast weights in the action head
  2. Training recipesequence action forcing, TBPTT, 8K context
  3. Decoupling fast and slow updatesDAgger distillation, in-context human video
  4. Resultsby the use of past experience
  5. Two questionson the way to T²Mem
Project-page teaser (first 30 s; click for the full 2 min). GR00T N1.7 backbone, context scaled to 8K timesteps, constant inference cost. Video: research.nvidia.com/labs/gear/robottt

Mechanism: attention within a timestep, TTT across timesteps.

images ot instruction l proprio qt VLM frozen hidden states Φt action expert attention within a timestep TTT layerTTT layerTTT layer proprio qt, 16 register tokens Rt, noised chunk Aτ
$$\textcolor{#1849C6}{W_t} \leftarrow \textcolor{#1849C6}{W_{t-1}} - \eta\,\nabla_W\big\|f_{W_{t-1}}(K_t) - V_t\big\|^2$$
(update, same as TTT layers)
$$O_t = f_{\textcolor{#1849C6}{W_t}}(Q_t)$$
(apply)
$$O = \tanh(\alpha)\odot O_{\text{TTT}} + O_{\text{attn}}$$
(gate, α init 0.001)
  • Placement: one TTT layer after each of GR00T N1.7's 16 DiT layers. Fast model is a 2-layer MLP, about 10M params per layer.
  • Gate starts near zero, so the pretrained policy is untouched at step 0 and learns how much memory to use.

Training recipe: how to make this work at 8K timesteps.

Sequence action forcing

$$\mathcal{L}_{\text{fm}} = \tfrac{1}{T}\textstyle\sum_{t}\ \mathbb{E}_{\tau_t,\epsilon}\big\|v_\theta(\,\cdot\,;W_{t-1}) - (A_t-\epsilon)\big\|^2$$
$$A_t^{\tau_t} = \tau_t A_t + (1-\tau_t)\,\epsilon,\qquad \tau_t \text{ drawn per chunk}$$

One noise level per chunk, not per sequence. Shared noise makes whole sequences uniformly easy or hard; training is unstable without this.

Truncated BPTT

TBPTT: gradients cut at segment boundaries, fast weights carried across

Fast weights carry across segments, gradients are cut. GPU memory scales with segment length, not context length. $W_0$ still gets gradient through segment 1.

One training step, start to finish 1 · Sample a sequence T/H contiguous control steps, each with ot, qt, At … t = 11+H1+2H 2 · Noise every chunk at its own τt sequence action forcing: a sequence is never all-easy or all-hard τ = 0.8τ = 0.2τ = 0.5 3 · One forward pass over the whole sequence attention per step; inside each TTT layer the fast weights sweep forward, one inner gradient step per chunk, chunk tokens as the mini-batch W₀W₁W₂… 4 · One velocity prediction per chunk flow loss ℓt at that chunk's τt, conditioned on Wt−1; no k-step denoising in training vθ → ℓ1, ℓ2, ℓ3, … 5 · Average the losses, one outer step on the slow weights gradient flows back through the inner updates, so W₀ and η are meta-learned; TBPTT cuts that backward pass at segment boundaries while W carries across

Fast weights let us learn how to use heterogeneous data.

fast-weight update (inner loss, self-supervised): every timestep, whatever the data flow-matching loss on DAgger data robot's own actions: in memory, no loss human corrections: in memory and supervised flow-matching loss on in-context human video data human video: no action labels, in memory only paired robot trajectory of the same task: supervised

Mixed quality: DAgger distillation

  • Remember the entire trajectory, but only update slow weights on the correction behavior.
  • Learn when to respond to a failure and the mapping failure → correction, not just the corrections in isolation. They position it as "Algorithm Distillation through meta-learning."

Mixed modality: in-context human video

  • No action labels in a video, but the inner loss is self-supervised, so the video still updates $W$.
  • Prime the memory with the human video; at train time, use slow weight updates on a paired robot trajectory, and at test time, this will produce in-context learning.

Results, grouped by the use of past experience.

  1. Remembering contextmain evaluation, perturbations
  2. Context as a scaling axis128 to 8K pretraining context
  3. Learning from feedbackDAgger distillation
  4. Learning from examplesone-shot imitation from human video
  5. Ablationswhat the design decisions bought
Pup Go Car, 2 min
Circuit, 1 min, 80 configs (20 train / 60 test)
Gear Bot, 5 min, 10 stages

Click a clip for the full sped-up rollout. YAM bimanual, 4 RGB cameras. Baselines: GR00T N1.7 (single step), GR00T N1.7 Hist. (one history frame), GDN (same placement and parameter count, Gated DeltaNet instead of TTT). Sequence models post-trained at 1K context. Video: research.nvidia.com/labs/gear/robottt

Past experience for remembering context

Task completion score bars for RoboTTT, GR00T N1.7, GR00T N1.7 Hist., GDN on three tasks
Fig. 7. Rubric-based task completion, 20 trials per task (10 for Gear Bot).
fully successful trials, %recovered after a human removes a part, %
Pup Go CarCircuitGear Botrooftire
RoboTTT4565207590
GR00T N1.7151505055
GR00T N1.7 Hist.04001525
GDN154006590

Table 1 (20 trials, 10 for Gear Bot) and Table 3 (20 trials per condition, 30 min of perturbation data co-trained). Darker is better within a column.

Context length as a scaling axis

Average task completion vs pretraining context length from 128 to 8K: RoboTTT rises steadily, GDN flat
Fig. 8. Pretrain at each context length, post-train, evaluate closed-loop on all three tasks. Predates the DAgger training used in Fig. 7.

Past experience for learning from feedback: DAgger distillation

Bar chart of task completion after four kinds of DAgger fine-tuning, for RoboTTT, GDN, GR00T N1.7 and GR00T N1.7 Hist.
Fig. 10, Pup Go Car. 100 pooled DAgger trajectories (50 from RoboTTT, 50 from GR00T N1.7); four fine-tuning variants per model.
On-the-fly recovery (the event in Fig. 11): misses the roof screw, re-aligns, succeeds. No human in the loop. Click for the wheel case too. Video: research.nvidia.com/labs/gear/robottt

Past experience for learning from examples: one-shot imitation from human video

Human demo, reset, robot rollout on an unseen configuration. Same prompt in every clip; click for all four configurations. Video: research.nvidia.com/labs/gear/robottt
Circuit, unseen configurationCompletionSuccesses
RoboTTT65%6 / 10
GDN33%0 / 10

Table 2. GR00T N1.7 cannot condition on context; its Hist. variant's window is far shorter than a video.

Ablations: what the design decisions bought

Ablation bars on Pup Go Car: no sequence action forcing, TTT linear, state tokens, plus action tokens, plus register tokens, GR00T with register tokens
Fig. 12, Pup Go Car. GR00T N1.7 + register tokens is the matched-token control.

Two questions on the way to T²Mem

Why are we putting memory into the action head? Would it be more useful to remember observations?

  • RoboTTT keeps the VL tokens out of TTT "for computational efficiency"; 16 registers stand in for them.
  • Their own limitations section asks for robotics-oriented objectives for the TTT layers.

Robot streams are strongly correlated with few surprising moments. Do we need to memorize each time step equally?

  • The pitch is that TTT "retains salient features and discards redundant ones", but the update rule treats every step the same.
  • Titans already asked this for language with surprise-driven updates.

How do we make memory preserve information that matters for downstream robotic decisions?
T²Mem. Idea: tailor RoboTTT using inductive biases from robotics.

  1. Memory-dependent tasksRoboMME
  2. Memory between VLM and AEvisual features are more useful than those in the action head
  3. Surprise-gated updatesobservations are temporally correlated
  4. Alternating memory–policy learninglearn to remember, then learn to act
  5. Results: RoboMME · is the memory used?
T2Mem overview: memory-dependent tasks, single VLA policy with fast-weight memory interface, test-time rollout
Fig. 1. π0.5 backbone, memory between VLM and action expert, simulated RoboMME tasks.

What counts as a memory-dependent task

$$\sup_{\pi\in\Pi_{\text{hist}}} J(\pi)\ >\ \sup_{\pi\in\Pi_{\text{current}}} J(\pi)$$
(the current observation is not enough; RoboMME, 16 simulated tasks in four suites)
1 / 3 · VideoUnmask: recall a hidden object (Permanence). Click for the next task. Video: robomme.github.io
2 / 3 · PickXtimes: count repeated actions (Counting). Click for the next task. Video: robomme.github.io
3 / 3 · PatternLock: reproduce a demonstrated sequence (Imitation). Click to start over. Video: robomme.github.io

Why not remember the observations themselves?

images ot instruction l VLM one forward pass hidden states Φt direct path memory interface 16 learned queries over Φt → Ut fast weights Wt write K, V · read Q → Rt queries see Φt only, never actions or proprio action expert attends to Φt and Ũt proprio qt, noised chunk Aτ
$$U_t = \operatorname{Attn}\big(Q_{\text{if}}(E_t),\ K_{\text{VL}}(\textcolor{#2A7B76}{\Phi_t}),\ V_{\text{VL}}(\textcolor{#2A7B76}{\Phi_t})\big)$$
(16 learned queries over the VL tokens only)
$$Q_t = U_t\theta_q,\qquad R_t = f_{\textcolor{#1849C6}{W_t}}(Q_t)$$
(read)
$$K_t = U_t\theta_k,\qquad V_t = U_t\theta_v$$
(bindings to write)
$$\tilde U_t = U_t + \tanh(\alpha)\odot R_t$$
(gated fusion → keys and values for the action expert)
Ablation: memory on the VL path 84 and 71 percent vs the same memory inside the action expert 30 and 26 percent vs no memory 23 and 28 percent
Fig. 4(a), VideoUnmask and MoveCube: memory on the VL path vs the same memory inside the action expert (RoboTTT-style) vs none. Click to expand.

Do we need to memorize each time step equally?

weight space W₀ Wt Δ = Wt − W₀, the path so far gradient aligned with Δ → β ≈ 0.1, short step gradient orthogonal to Δ → β ≈ 1, full step descent direction −g step taken −β·η·g
$$a_t = \cos\angle\big(-g_t,\ W_t - W_0\big)$$
(alignment with the path so far)
$$r_t = \frac{\lVert f_{W_t}(K_t) - V_t\rVert}{\lVert V_t\rVert}$$
(relative residual; normalization dropped)
$$\beta_t = \operatorname{smoothstep}\!\Big(\tfrac{1}{2}(1 - a_t)(1 + r_t)\Big)\ \in [0.1,\,1]$$
(write scale; clipping dropped)
$$W_{t+1} = W_t - \eta_t\,\beta_t\,g_t$$
(gated update)
Early-memory key/value reconstruction error over 300 subsequent frames, adaptive vs fixed beta
Fig. 4(c). Replay 300 real frames after a shared demo prefix; measure error on the first bindings. 20 VideoUnmask Hard episodes.

Alternating memory–policy learning

Training stages: pretrained VLA, memory-free adaptation, then alternate memory learning (policy frozen) and memory-conditioned policy learning (memory frozen)
Fig. 3. Freezing refers to slow parameters; the fast state keeps updating in every stage.
Ablation: alternating 84 and 71 percent vs non-alternating 28 and 28 percent vs no memory 23 and 28 percent
Fig. 4(b). Same budget, same data, same initialization.

Results on RoboMME

CountingPermanenceReferenceImitation
BinFillPickXSwingXStopCubeVidUmskBtnUmskVidUmskSBtnUmskSPickHighLVidRepickVidPlcBtnVidPlcOrdMoveCubeInsertPegPatternLockRouteStickAvg
Human96.010080.078.090.092.092.090.092.092.098.090.090.098.084.086.090.5
GroundSG, oracle subgoals (privileged)85.810010049.798.895.099.280.283.397.310010087.815.697.055.684.1
MemER (keyframes + VLM subgoals)56.779.359.30.081.372.038.021.370.725.330.026.082.76.716.712.042.4
FrameSamp + Modul (best RoboMME)39.687.392.042.032.725.124.418.222.930.460.032.077.87.653.666.744.5
TTT + Modul (RoboMME's naive TTT)34.265.136.72.127.222.125.214.114.612.132.722.331.21.13.67.022.0
π0.5 (memory-free)30.042.935.66.720.422.218.76.711.30.431.125.826.01.62.94.717.9
Stage 1 (their memory-free adaptation)26.734.031.32.027.36.018.02.011.31.326.727.338.01.33.35.316.4
T²Mem60.791.390.770.088.059.329.320.050.738.735.330.080.736.050.078.756.8

Table 1, condensed (full table in backup). Success rate %, 50 episodes × 3 seeds per task. Blue: best among non-privileged methods. Baseline numbers are quoted from the RoboMME paper.

Does the policy actually use the memory, and what does it cost?

Interventions: disabling writes collapses success; conflicting memory is worse than empty; pre-occlusion writes match full demonstration
Fig. 5. Task-specific checkpoints. (a) writes disabled vs enabled, (b) correct / conflicting / empty demo memory, (c) which demo frames may write.
History A: blue cube hidden in the first location
History B: blue and green exchanged

The counterfactual pair behind (b): same instruction, same final scene, different history, different correct container. Expert demonstrations from the project page, not policy rollouts. Video: yzliu84.github.io/T2MEM-project

Conclusions and my thoughts

What this says about TTT itself

  • Adaptive updates seem like a hard-coded heuristic for just further nesting the optimization problem and having even more frequencies of weights that control deeper optimization problems; see Nested Learning.

Where else memory could live

  • We know memory as context, layer, and gating, but what about as a separate expert with cross-attention?

How I would integrate it into robotics

  • No new module or gating, In-Place TTT style.
  • We can also take the more tailored inner objective like In-Place TTT or any of the exploration papers.

Backup

  1. RoboTTT tasks and rubrics
  2. RoboTTT training details
  3. T²Mem step-size calibration
  4. T²Mem training and inference order
  5. T²Mem task walkthroughs
  6. Full RoboMME table

RoboTTT tasks and rubrics

Pup Go Car stages
Pup Go Car (2 min): roof on, screw, drill, hand-off, flip, wheel in, drill, hand-off. Rubric 0.05 → 1.0 in 14 steps; at most two wheel attempts.
Gear Bot stages
Gear Bot (5 min, 10 stages): gear + two wheels per side, two chassis flips, head, drive with the remote. +0.1 per stage.
Circuit configurations
Circuit (1 min, ~80 configs): two or three components in a given order, switch or button turned on if present. Equal credit per component; no partial credit for a wrong order.

RoboTTT training details (Appendix A)

Trajectory length histogram of the pretraining mixture
Fig. A.1. Pretraining mixture by episode length.

T²Mem: curvature-calibrated step size (Appendix A.1)

$$G = \lVert g_t\rVert^2,\qquad C = \frac{2\,\lVert J_f\, g_t\rVert^2}{H_m N d}$$
(gradient norm, curvature proxy; per head)
$$q = \operatorname{clip}_{[10^{-12},\,10^{3}]}\big(G / \max(C, 10^{-30})\big)$$
(trial step)
$$b = \frac{L_h(W_t - q g_t) - L_h(W_t) + G q}{q^2}$$
(measured curvature along $g_t$)
$$\ell_t = \min\!\Big(10^3,\ \begin{cases} G/\max(2b,10^{-30}) & b>0\\ q & b\le 0\end{cases}\Big),\qquad \eta_t = \tfrac{1}{2}\operatorname{softplus}(\rho)\,\ell_t$$
(step size; one learned $\rho$ per layer, init so the multiplier is 1)
$$W_{t+1} = W_t - w_t\,\eta_t\,\beta_t\,g_t$$
(with $\beta_t$ from the surprise gate)

$\beta_t$ and the calibration are stop-gradient. Outer gradients still flow through the fast-state recurrence, $g_t$, the projections and $\rho$. The "fixed $\beta=1$" control keeps this calibration.

T²Mem: training and inference order

Training: one alternating-phase update; Inference: one episode
Fig. 7. Reads precede writes at the same observation.
Stage 1Stage 2AStage 2B
Purposememory-free adaptationmemory learningmemory-conditioned policy
TrainsVLM LoRA, AEinterface, projections, $W_0$, step multipliers, gatesVLM LoRA, AE, proprio proj.
Budget20K steps500 updates / cycle500 updates / cycle
LR2e-57.5e-5 (gates 5e-4)3e-5
Batch64 frames32 sequences32 sequences

Table 3. 32 H200 for 48 hours in total. Stride 16, predict 64 actions, execute 32.

T²Mem task walkthroughs

VideoUnmask (RoboMME demo): color–location bindings visible before the containers come down, occluded after. Video: robomme.github.io
MoveCube frames
Fig. 9, MoveCube: reproduce the demonstrated method (push, pick-and-place, hook), not just the destination.
SwingXtimes frames
Fig. 10, SwingXtimes: count completed cycles, then stop.
Counterfactual histories
Fig. 11: identical observations after occlusion, different correct answers.

Full RoboMME table

BinFillPickXSwingXStopCubeVidUmskBtnUmskVidUmskSBtnUmskSPickHighLVidRepickVidPlcBtnVidPlcOrdMoveCubeInsertPegPatternLockRouteStickAvg
Human96.00100.080.0078.0090.0092.0092.0090.0092.0092.0098.0090.0090.0098.0084.0086.0090.50
SimpleSG · Oracle (privileged)85.7899.78100.044.6733.1122.0015.5615.5644.0027.7831.3326.0087.3310.0095.3355.1149.58
GroundSG · Oracle (privileged)85.78100.0100.049.6798.7895.0099.2280.2283.3397.33100.0100.087.7815.5697.0055.5684.08
SimpleSG · Gemini46.0063.0045.002.0029.009.0014.002.0020.0015.0026.0029.0061.004.007.000.0023.25
SimpleSG · QwenVL77.5695.335.110.4434.2219.3315.339.5617.1125.3333.3325.1182.003.7812.677.7829.00
GroundSG · Gemini26.0018.004.003.0036.0014.0013.000.009.0017.0012.007.0017.000.007.002.0011.56
GroundSG · QwenVL52.0092.677.330.0088.6724.0030.6714.0015.1125.3354.0031.7871.563.336.676.0032.70
TokenDrop · Context48.6785.1194.673.1133.7831.5626.2216.0020.6717.7831.1125.3381.334.0012.6720.0034.50
TokenDrop · Modul34.4483.5686.005.3328.2229.3328.4421.3321.3322.0059.5636.0062.007.1132.4451.5638.04
TokenDrop · Expert54.2287.5691.784.2226.6730.4418.4418.8919.3320.8936.4424.6787.562.2216.2218.2234.86
FrameSamp · Context41.2272.0073.6713.6726.8930.2220.8915.2217.6715.2230.0020.8977.221.2215.2219.6730.68
FrameSamp · Modul39.5687.3392.0042.0032.6725.1124.4418.2222.8930.4460.0032.0077.787.5653.5666.6744.51
FrameSamp · Expert57.3386.2294.6728.8931.7825.7822.8920.2219.1123.1130.0024.2283.112.0013.5617.1136.25
TTT · Context35.5662.8942.443.3329.7822.8918.4414.4420.4413.1134.2220.2232.441.111.563.5622.28
TTT · Modul34.2265.1136.672.1127.2222.1125.2214.1114.5612.1132.6722.3331.221.113.567.0021.96
TTT · Expert34.8963.7841.334.0031.7822.4419.5618.0012.229.5634.0022.8933.560.893.115.5622.35
RMT · Context32.4456.8933.565.7831.3310.8917.332.0014.003.7832.0029.1125.782.005.568.8919.46
RMT · Modul33.3360.7837.784.6731.1111.7817.782.4417.114.2232.0031.1124.672.213.788.0020.17
RMT · Expert35.7860.2236.005.5628.0017.1115.782.0011.780.2224.2222.6720.001.894.224.8918.15
π0.530.0042.8935.566.6720.4422.2218.676.6711.330.4431.1125.7826.001.562.894.6717.93
π0.5 w/ past actions26.6758.3326.674.6730.6723.6720.6716.0012.338.6724.0018.6734.001.004.005.6719.73
SAM2Act+40.0076.0025.330.0027.3332.0018.0026.6717.335.3324.6720.0029.330.000.000.0021.37
MemER56.6779.3359.330.0081.3372.0038.0021.3370.6725.3330.0026.0082.676.6716.6712.0042.38
Stage 1 (no memory)26.6734.0031.332.0027.336.0018.002.0011.331.3326.6727.3338.001.333.335.3316.38
T²Mem60.6791.3390.6770.0088.0059.3329.3320.0050.6738.6735.3330.0080.6736.0050.0078.6756.83

Table 1 of T²Mem; all baseline numbers quoted from the RoboMME paper. Context / Modul / Expert are three ways RoboMME integrates a memory representation: prepended tokens, adaptive LayerNorm modulation of the AE, or a separate memory transformer attended by the AE.

Missing slide. Press ↑ or M.