GradMem

https://arxiv.org/abs/2603.13875

titans learning to memorize at test time flashbacks

what this paper introduces/does

they want to use grad steps to use and update memory priors over an LLM to make it efficcient than KV cache over long horizion. the LLM stays frozen i think.

we have $C$ the context being passed , query $Q$ and target result $Y$ they try to compress $C$ into a small fixed size $M$ memory. since query is the actual prompt as to what to do with the information $C$ or it’s compressed version is just the base state of information that the model needs to give the correct result.

and $f_{\theta }$ is a causal language model with $\theta$ params $f_{\theta }(Y|X)$ denotes the probability assigned by the model to an output sequence $Y$ conditioned on input $X$. so the equation of a standard model is : $f_{\theta }(Y|[C;Q])$

they have a two parts,

1. WRITE which is the encoder converting $C$ to $M$ as $M={\mathcal{E}}_{\theta }(C)$

2. **READ decode using memory and query **

$$f_{\theta }(Y|M,Q)\triangleq f_{\theta }(Y|[M;Q])$$

the actual training