Skip to content
Merged
3 changes: 0 additions & 3 deletions Package.swift
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,12 @@ let package = Package(
.visionOS("1.0"),
],
products: [
// Products define the executables and libraries a package produces, making them visible to other packages.
.library(
name: "ToonFormat",
targets: ["ToonFormat"]
)
],
targets: [
// Targets are the basic building blocks of a package, defining a module or a test suite.
// Targets can depend on other targets in this package and products from dependencies.
.target(
name: "ToonFormat"
),
Expand Down
117 changes: 51 additions & 66 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

Compact, human-readable serialization format for LLM contexts with **30-60% token reduction** vs JSON. Combines YAML-like indentation with CSV-like tabular arrays. Full compatibility with the [official TOON specification](https://github.com/toon-format/spec).

**Key Features:** Minimal syntax • Tabular arrays for uniform data • Array length validation • Swift 6.0+ • Configurable delimiters • Key folding support.
**Key Features:** Minimal syntax • Tabular arrays for uniform data • Array length validation • Swift 6.0+ • Configurable delimiters • Key folding / Path expansion support • Linux compatible.

LLM tokens are expensive, and JSON is verbose.
TOON saves tokens while remaining human-readable by
Expand Down Expand Up @@ -37,6 +37,8 @@ see the [TOON specification](https://github.com/toon-format/spec).

## Features

### TOONEncoder

`TOONEncoder` conforms to **TOON specification version 3.0** (2025-11-24)
and implements the following features:

Expand All @@ -53,10 +55,25 @@ and implements the following features:
- [x] Configurable flatten depth to limit the depth of key folding
- [x] Collision avoidance so folded keys never collide with existing sibling keys

### TOONDecoder

`TOONDecoder` conforms to **TOON specification version 3.0** (2025-11-24)
and implements the following features:

- [x] Correct escape sequence parsing (`\\`, `\"`, `\n`, `\r`, `\t`)
- [x] Three delimiter types: comma (default), tab, pipe
- [x] Array length validation
- [x] Tabular format parsing with field headers
- [x] Inline format for primitive arrays
- [x] Expanded list format for nested structures
- [x] Path expansion to unfold dotted keys into nested objects (inverse of key folding)
- [x] Detailed error reporting with line numbers
- [x] Configurable decoding limits for security

## Requirements

- Swift 6.0+ / Xcode 16+
- iOS 13.0+ / macOS 10.15+ / watchOS 6.0+ / tvOS 13.0+ / visionOS 1.0+
- iOS 13.0+ / macOS 10.15+ / watchOS 6.0+ / tvOS 13.0+ / visionOS 1.0+ / Linux

## Installation

Expand All @@ -66,10 +83,16 @@ Add the following to your `Package.swift` file:

```swift
dependencies: [
.package(url: "https://github.com/toon-format/toon-swift.git", from: "0.2.0")
.package(url: "https://github.com/toon-format/toon-swift.git", from: "0.3.0")
]
```

Then add the dependency to your target:

```swift
.target(name: "YourTarget", dependencies: ["ToonFormat"])
```

Comment thread
mattt marked this conversation as resolved.
## Usage

### Quick Start
Expand All @@ -84,6 +107,7 @@ struct User: Codable {
let active: Bool
}

// Encoding
let user = User(
id: 123,
name: "Ada",
Expand All @@ -94,18 +118,18 @@ let user = User(
let encoder = TOONEncoder()
let data = try encoder.encode(user)
print(String(data: data, encoding: .utf8)!)
```

Output:
// id: 123
// name: Ada
// tags[2]: reading,gaming
// active: true

```
id: 123
name: Ada
tags[2]: reading,gaming
active: true
// Decoding
let decoder = TOONDecoder()
let decoded = try decoder.decode(User.self, from: data)
print(decoded.name) // "Ada"
```

### Custom Delimiters
#### Custom Delimiters

Use tab or pipe delimiters for additional token savings:

Expand Down Expand Up @@ -144,7 +168,7 @@ items[2|]{sku|name|qty|price}:
B2|Gadget|1|14.5
```

### Length Markers
#### Length Markers

Add a `#` prefix to array lengths for emphasis and readability:

Expand Down Expand Up @@ -172,7 +196,7 @@ items[#2]{sku,qty,price}:
B2,1,14.5
```

### Tabular Arrays
#### Tabular Arrays

Arrays of objects with identical primitive fields use an efficient tabular format:

Expand Down Expand Up @@ -200,7 +224,7 @@ items[2]{sku,qty,price}:
B2,1,14.5
```

### Arrays of Arrays
#### Arrays of Arrays

For arrays containing primitive inner arrays:

Expand All @@ -219,7 +243,7 @@ pairs[2]:
- [2]: 3,4
```

### Key Folding
#### Key Folding

Key folding collapses single-key nested objects into dotted paths, reducing indentation and token count:

Expand All @@ -242,6 +266,7 @@ let config = Config(
)

let encoder = TOONEncoder()
encoder.keyFolding = .safe
let data = try encoder.encode(config)
```

Expand All @@ -262,66 +287,26 @@ database.connection:
port: 5432
```

When enabled, key folding applies only when
all path segments are valid identifiers
(start with a letter or underscore and contain only alphanumerics or underscores),
each level in the chain is a single-key object,
and the folded path does not collide with an existing sibling key
(collision avoidance).
### Decoding Limits

#### Flatten Depth

To control how aggressively key folding collapses nested objects,
use `flattenDepth`:
Protect against malicious or malformed input:

```swift
struct Metrics: Codable {
struct Service: Codable {
struct CPU: Codable {
let usage: Double
}
let cpu: CPU
}
let service: Service
}

let value = Metrics(
service: .init(
cpu: .init(usage: 0.73)
)
let decoder = TOONDecoder()
decoder.limits = TOONDecoder.DecodingLimits(
maxInputSize: 1024 * 1024, // 1 MB
maxDepth: 64,
maxObjectKeys: 1000,
maxArrayLength: 10000
)

let encoder = TOONEncoder()
encoder.keyFolding = .safe
let data = try encoder.encode(value)
```

Output with unlimited `flattenDepth` (default):

```
service.cpu.usage: 0.73
```

Output with deep nesting and `flattenDepth = 2`:

```swift
encoder.flattenDepth = 2
```

```
service.cpu:
usage: 0.73
```

> [!TIP]
> Specifying a flatten depth less than 2 has no practical effect.

### Version Information

Check the supported TOON specification version:

```swift
print(TOONEncoder.specVersion) // "3.0"
print(toonSpecVersion) // "3.0"
```

## Contributing
Expand All @@ -339,7 +324,7 @@ This project follows the [Contributor Covenant Code of Conduct](CODE_OF_CONDUCT.

## Project Status

This library implements **TOON specification version 3.0** (2025-11-24) with full encoding support.
This library implements **TOON specification version 3.0** (2025-11-24) with full encoding and decoding support.

See [CONTRIBUTING.md](CONTRIBUTING.md) for detailed guidelines.

Expand Down
Loading