TinyRecursiveModels: Redefining AI with "Less is More"

In an era dominated by ever-larger foundational models, the TinyRecursiveModels (TRM) project from Samsung SAILT Montreal offers a refreshing and impactful counter-narrative: "Less is More." This open-source initiative introduces a groundbreaking recursive reasoning approach that achieves remarkable scores on challenging AI benchmarks like ARC-AGI-1 (45%) and ARC-AGI-2 (8%) using an incredibly compact 7-million-parameter neural network.

Challenging the Status Quo

The central motivation behind TRM is to debunk the myth that success in complex AI tasks solely depends on deploying massive, expensive-to-train models. TRM illustrates that a small model, when designed with an efficient recursive reasoning mechanism, can rival the performance of much larger counterparts. This philosophy not only democratizes AI development by reducing computational barriers but also opens new avenues for research into intelligent systems.

How TRM Works: Simplified Recursive Reasoning

TRM simplifies the concept of recursive reasoning, stripping it of unnecessary complexity often seen in other models inspired by biological systems. Its core mechanism involves a tiny network that iteratively refines its predicted answer. Starting with an embedded input question, an initial embedded answer, and a latent state, TRM performs two key steps:

1. Recursive Latent Update: The model recursively updates its latent state multiple times, conditioned on the question, current answer, and existing latent state.
2. Answer Refinement: The updated latent state is then used to refine the current answer.

This iterative process allows TRM to progressively improve its solutions, effectively addressing past errors and minimizing overfitting, all within an extremely parameter-efficient framework.

Get Started with TinyRecursiveModels

The project provides comprehensive instructions for setting up and experimenting with TRM. Here’s what you need to begin:

• Environment: Python 3.10 and Cuda 12.6.0 (or similar versions).
• Dependencies: Install necessary libraries, including torch (ensure compatibility with your CUDA version) and other requirements via pip.

Dataset Preparation & Experiments

TRM supports various datasets, including:

• ARC-AGI-1 and ARC-AGI-2 (for which specific notes on training data are provided).
• Sudoku-Extreme.
• Maze-Hard.

Detailed commands are available to build these datasets and run experiments across different GPU setups, showcasing the model's versatility across logical reasoning and puzzle-solving tasks. Runtimes range from under 24 hours to approximately 3 days, depending on the task and hardware.

Citing This Work

If you find TinyRecursiveModels beneficial for your research or applications, please consider citing the accompanying paper, "Less is More: Recursive Reasoning with Tiny Networks," by Alexia Jolicoeur-Martineau (2025). The work also references the innovative Hierarchical Reasoning Model (HRM) that inspired its development.

TinyRecursiveModels stands as a testament to the power of thoughtful architectural design over brute-force scaling, offering a practical, open-source solution for advanced AI reasoning.