Format Overview

TOON (Tabular Object Oriented Notation) is a minimalist data serialization format optimized for LLM context windows. It represents arrays of objects with consistent schemas using a header-based structure.

Basic Structure

TOON.schema: field1 | field2 | field3
value1 | value2 | value3
value4 | value5 | value6

Syntax Rules

1. Schema Declaration

TOON.schema: <field_name_1> | <field_name_2> | ... | <field_name_N>

• Must be first line of document
• Prefix: TOON.schema: (case-sensitive)
• Field names separated by | (space-pipe-space)
• Field names: alphanumeric + underscore, no spaces
• Order defines column order for all rows

Valid Examples

TOON.schema: id | name | email
TOON.schema: user_id | first_name | age | is_active

Invalid Examples

toon.schema: id | name          ❌ Wrong case
TOON.schema:id|name|email        ❌ Missing spaces
TOON.schema: first name | email  ❌ Space in field name

2. Data Rows

<value_1> | <value_2> | ... | <value_N>

• Each row starts on new line after schema
• Values separated by | (space-pipe-space)
• Number of values must match number of schema fields
• Values are strings (no type annotation)
• Leading/trailing whitespace in values is significant

Example

TOON.schema: id | name | age
1 | Alice | 30
2 | Bob | 25

3. Value Escaping

When values contain the delimiter (|), escape it:

Escaping Rules

• Pipe character: | → \|
• Backslash: \ → \\
• Newline: \n → \\n

Example

TOON.schema: title | description
Product A | Size: small \| medium \| large
Product B | Use \\ for escaping

Data Types

TOON is intentionally type-agnostic. All values are stored as strings. Parsers should infer types contextually or provide type conversion utilities.

Type Representations

Numbers

123  |  45.67  |  -89  |  1.23e5

Booleans

true  |  false  |  True  |  False  |  1  |  0

Null/Empty

NULL  |  null  |  (empty string)  |  -  |  N/A

Dates

2025-11-13  |  11/13/2025  |  1699900800

Recommendation: Use consistent representations within each field. For example, always use ISO 8601 for dates, "true"/"false" for booleans.

Edge Cases

Empty Values

TOON.schema: id | name | email
1 | Alice | 
2 |  | bob@example.com

Empty strings are valid. First row has empty email, second row has empty name (single space counts as non-empty).

Whitespace Handling

TOON.schema: name | title
Alice | Senior Engineer
 Bob  |  Manager

Whitespace before/after delimiter is part of value. "Alice" vs " Bob " are different. Parsers should NOT auto-trim unless explicitly configured.

Single Row

TOON.schema: id | name
1 | Alice

Valid TOON document with 1 data row. No minimum row count required.

No Data Rows

TOON.schema: id | name

Invalid: Schema with zero data rows. At minimum, schema + 1 row required.

Column Mismatch

TOON.schema: id | name | email
1 | Alice
2 | Bob | bob@example.com | extra

Invalid: First row has 2 values (expected 3), second row has 4 values (expected 3). All rows must match schema length.

Validation Rules

Parser Requirements

✓ MUST validate

• First line starts with TOON.schema:
• Schema has at least 1 field
• At least 1 data row exists
• All rows have same number of values as schema

✓ SHOULD validate

• Field names are valid identifiers (alphanumeric + underscore)
• No duplicate field names in schema
• Escaped characters are properly formed

✓ MAY validate

• Type consistency within columns
• Field name conventions (snake_case, camelCase)
• Maximum line length

Conversion to/from JSON

JSON → TOON

Input (JSON)

[
  {"id": 1, "name": "Alice", "active": true},
  {"id": 2, "name": "Bob", "active": false}
]

Output (TOON)

TOON.schema: id | name | active
1 | Alice | true
2 | Bob | false

TOON → JSON

Input (TOON)

TOON.schema: user_id | balance
101 | 49.99
102 | 1250.00

Output (JSON)

[
  {"user_id": "101", "balance": "49.99"},
  {"user_id": "102", "balance": "1250.00"}
]

Note: Values remain strings. Type inference is parser-dependent.

MIME Type & File Extension

MIME Type

text/toon

(Proposed, not standardized)

File Extension

.toon

Complete Example

users.toon

TOON.schema: user_id | username | email | created_at | is_verified
1001 | alice_wonder | alice@example.com | 2025-01-15 | true
1002 | bob_builder | bob@domain.co | 2025-02-20 | false
1003 | charlie_dev | charlie@test.org | 2025-03-10 | true
1004 | diana_admin |  | 2025-04-05 | false
1005 | eve_tester | eve@qa.com | 2025-05-12 | true

TOON Format Specification