TOON: JSON for LLM Prompts at Half the Tokens

Introducing TOON: The Token-Oriented Object Notation for LLMs

In the rapidly evolving landscape of Large Language Models (LLMs), efficiency is key. As context windows grow and data inputs become larger, the cost associated with token usage can escalate. This is where Token-Oriented Object Notation (TOON) steps in, offering a clever, open-source solution to significantly reduce token consumption when passing structured data to LLMs.

Developed by Johann Schopplich, TOON is an innovative data format specifically engineered for LLM input, not as a general-purpose JSON replacement for APIs or storage. Its core strength lies in its ability to convey the same information as JSON with a 30-60% reduction in tokens, leading to more cost-effective and faster LLM interactions.

Why TOON Matters for LLM Prompts

Traditional JSON, while ubiquitous for data exchange, can be verbose and token-heavy, especially for repetitive data structures. TOON addresses this by leveraging a minimal syntax that removes redundant punctuation (braces, brackets, most quotes) and adopts an indentation-based structure similar to YAML. Its 'sweet spot' is uniform arrays of objects – data with multiple fields per row and consistent structures across items. For these scenarios, TOON utilizes a tabular format, optimized for token efficiency.

Consider a simple example:

JSON:

{
"users": [
{ "id": 1, "name": "Alice", "role": "admin" },
{ "id": 2, "name": "Bob", "role": "user" }
]
}

TOON:

tsers[2]{id,name,role}:
1,Alice,admin
2,Bob,user

The difference in token count, particularly across large datasets, is substantial.

Key Features and Benefits

• Token-Efficient: Achieve up to 60% token savings compared to JSON, optimizing costs and prompt lengths.
• LLM-Friendly Guardrails: Explicit lengths and field declarations simplify data parsing and validation for LLMs.
• Minimal Syntax: Reduces boilerplate punctuation, enhancing readability for humans and improving tokenizer efficiency.
• Indentation-Based Structure: Familiar to YAML users, using whitespace for nested objects.
• Tabular Arrays: Declares keys once and streams data as rows, ideal for uniform datasets.
• Performance Benchmarks: TOON consistently demonstrates superior token efficiency across various real-world datasets like GitHub repositories, daily analytics, and e-commerce orders, often with comparable or even better retrieval accuracy across different LLM models (gpt-5-nano, claude-haiku, gemini-2.5-flash, grok-4-fast).

How TOON Works and How to Use It

TOON is designed as a translation layer: you can use JSON programmatically and convert it to TOON for LLM input. It's available as an npm package (@byjohann/toon) and includes a command-line interface (CLI) for easy conversion between JSON and TOON formats.

Installation is straightforward:

npm install @byjohann/toon
# or with pnpm/yarn

Encoding data:

import { encode } from '@byjohann/toon'

const data = {
  users: [
    { id: 1, name: 'Alice', role: 'admin' },
    { id: 2, name: 'Bob', role: 'user' }
  ]
}

console.log(encode(data))
// Produces the TOON output shown above

You can also customize the output with options like delimiter (comma, tab, or pipe) and lengthMarker for enhanced token savings and clarity, especially when communicating specific formatting to the LLM.

TOON in LLM Prompts

When sending TOON to LLMs, the best practice is to show the format rather than just describing it. Wrap your encoded data in a fenced code block (e.g., ```toon). The explicit array lengths and field headers within TOON help models track structure, significantly reducing parsing errors.

For generating TOON using LLMs, be explicit in your prompt instructions. Define the expected header format and rules (e.g., 2-space indent, no trailing spaces). This guides the LLM to generate compliant and efficient TOON output.

Considerations and Limitations

While TOON shines with uniform arrays of objects, its efficiency can decrease for deeply nested or highly non-uniform data, where JSON might still be more suitable. It's crucial to understand that token savings can vary by the specific LLM and tokenizer being used (benchmarks are based on a GPT-style tokenizer).

However, for a vast array of LLM prompt engineering tasks – particularly those involving large, consistent datasets – TOON provides a powerful, open-source tool to enhance efficiency and optimize LLM interactions. With growing community support and implementations in various languages, TOON is quickly becoming a valuable asset for AI developers.

Explore the TOON GitHub repository to dive deeper into its specification, benchmarks, and ongoing development.