Notes

• Recursive Language Models

  An RLM is a system around a language model for reasoning over prompts that are too large or too complicated to handle reliably in one pass. The long prompt stays outside the model as an environment it can inspect.

• How a GPT-style Transformer Works

  A GPT-style transformer does two jobs over and over: it lets tokens share context through attention, then gives each token a small neural network to work on what it learned. Most of the architecture helps those two updates happen reliably, many layers deep.

• Latent Lookahead Training for Transformers

  Latent lookahead gives an autoregressive transformer extra internal computation before it commits to a difficult next token. The additional steps happen in continuous hidden-state space rather than as visible generated tokens.

• Gradient Episodic Memory

  GEM learns new tasks while trying not to damage performance on earlier ones. It stores a small episodic memory and treats gradients computed from those remembered examples as constraints.

• FastViTHD

  FastViTHD is the high-resolution hybrid vision encoder inside FastVLM. It converts a large image into a compact set of strong visual tokens before those tokens enter the language model.