# Soma Language — for AI Agents > Soma is a declarative cell language where systems carry their proofs. > One file holds a whole service: contract (`face`), storage with invariants > (`memory`), a model-checked lifecycle (`state`), handlers, HTTP routes and > tests. `soma verify` proves the state machines; memory invariants reject > bad writes before they commit; LLM agents run inside lifecycles the > compiler proves terminate, with token budgets. Every compiler error > names its own fix, so the write → check → fix loop converges. Everything below is current for the soma binary named in https://soma-lang.dev/version.json and generated from the compiler or verified against it. Unknown URLs on this site return 404, never a page. ## Machine-readable resources - [llms-service.txt](https://soma-lang.dev/llms-service.txt): this file + [guarantees](https://soma-lang.dev/docs/guarantees.md) (proven / enforced / not covered) + [serving](https://soma-lang.dev/docs/serving.md) (routing, exposure, atomicity) + [operations](https://soma-lang.dev/docs/operations.md) (what kills the process, ports, exit codes, HTTP statuses, limits, Linux deploy) + every gotcha — what a service needs, ~100 KB - [llms-full.txt](https://soma-lang.dev/llms-full.txt): this file + full reference + every gotcha + every builtin, one fetch (~130 KB) - [docs/robotics.md](https://soma-lang.dev/docs/robotics.md): 100 robotics robustness scenarios, reproduction commands and limits; offline with `soma docs robotics` - [agent.md](https://soma-lang.dev/agent.md): drop-in AGENTS.md / CLAUDE.md block for a project that uses Soma - [skill/SKILL.md](https://soma-lang.dev/skill/SKILL.md): the same knowledge packaged as an agent skill - [builtins.json](https://soma-lang.dev/builtins.json): every builtin with its exact signature (from `soma describe --builtins --json`) — never guess one - [gotchas.json](https://soma-lang.dev/gotchas.json): the mistakes models make, each with the real diagnostic and the fix - [corpus/domains.json](https://soma-lang.dev/corpus/domains.json): the domains (~3 KB) → one domain's index (~20 KB, summaries, each program's verify verdict and soma.toml) - [corpus/index.json](https://soma-lang.dev/corpus/index.json): 300+ verified programs (id, title, features, url) — filter on `features`, then GET the `url` for the raw source. Per domain with summaries: `https://soma-lang.dev/corpus//index.json`. Plain listing: [index.md](https://soma-lang.dev/corpus/index.md) - [docs/reference.md](https://soma-lang.dev/docs/reference.md), [docs/builtins.md](https://soma-lang.dev/docs/builtins.md), [docs/gotchas.md](https://soma-lang.dev/docs/gotchas.md): raw markdown - [repo/index.json](https://soma-lang.dev/repo/index.json): the package registry - [status](https://soma-lang.dev/status): maturity, version, MIT license, known limits, security contact - Source: https://github.com/soma-dev-lang/soma ## Pick Soma when / skip it when Soma 2.8.10 corrects native code that diverged from the interpreter: an exact Int quotient next to `%` stayed a Float, Int-only builtins given a Float were truncated or failed in rustc (they are refused like the interpreter), a huge literal times a small local overflowed instead of promoting to BigInt, `to_string` of an exact quotient printed "2.0", `str_at` bounds and error wording differed, `buffer(-3)` died on "capacity overflow". Storage: Int indices past 64 bits acted on element 0 of a List slot, a Map key serde reserves broke its round trip, `set_budget` accepted a negative BigInt; `rows[i] = v`, `rows.delete(i)` and `m.len` on persistent slots no longer rewrite or scan the whole table, joins and self-assigned `with`/`without` are linear. The quadratic-concatenation lint ignores numeric accumulators, `(f)(x)` is refused like `f(a)(b)`, `"{0x1F}"` interpolates, `check --json` keeps its keys on a load failure, and a restarted cluster node no longer reports its own tables as orphans. Sixteen new Rust regressions cover these changes; every program of the repository keeps its `check`, `verify` and `test` verdicts. Scope: native `/` is typed Float, so an exact-quotient division by zero answers `inf`/`NaN` natively where the interpreter raises. Details: https://soma-lang.dev/CHANGELOG.md. Soma 2.8.9 distinguishes storage read failures from absent keys and empty collections. A failed read aborts the invocation and rolls back earlier local writes, including inside `try`. Legacy lists retain their values on append and cannot resurrect deleted items or hang during replacement. The JSON backend refuses malformed/unreadable files and preserves its old state on write failure. The HTTP storage adapter validates responses, propagates failures and preserves typed non-finite Floats. Both protocol demos are corrected. Thirty-six new Rust regressions cover these changes (including three Python HTTP scenarios). Scope: JSON storage has per-file replacement, not SQLite multi-slot transactions; HTTP providers have no remote transaction or exactly-once guarantee, and this release does not wire HTTP providers into `run`/`serve`. Details: https://soma-lang.dev/CHANGELOG.md. Soma 2.8.8 checks SQLite transaction boundaries and propagates refused writes from `remember()`, `next_id()` and transitions. Failed commits withhold queued events and replication; failed task boundaries stop execution. Auxiliary tables open before the transaction. List writes handle failures atomically, indexed reads handle gaps, and normal returns from scheduled ticks commit their writes. `next_id()` rejects counter exhaustion/corruption and migrates only its own cell's legacy counter. Twenty-nine new regression tests cover these paths. Compatibility: a lost transaction aborts the invocation even inside `try`; already committed task steps and external effects are not undone. Details: https://soma-lang.dev/CHANGELOG.md. Soma 2.8.7 fixes `if` expression scope, lambda capture inside interpolations and error details, and collection evaluation when interpolations assign locals. Static analyses now see calls containing quoted colons: recursion, invariant writes and token costs cannot disappear from the proof. `recall()` stays within its cell and preserves JSON-shaped Strings; `remember()` rejects functions and excessive encoded nesting before writing. Twenty-four integration regressions reproduce the defects on 2.8.6; a unit test covers legacy memory migration. Compatibility: parse remembered JSON text explicitly with `from_json()`; keep branch locals inside their branch. Details: https://soma-lang.dev/CHANGELOG.md. Soma 2.8.6 corrects duplicate/skipped argument evaluation, self-append bindings, piped map updates, ignored builtin mocks and inconsistent pipe dispatch. Calls and pipes share local-lambda and cell-handler resolution. Constructors coerce declared Float and nested record fields and reject incompatible shapes. Twenty-four new integration regressions cover these fixes. Compatibility and examples: https://soma-lang.dev/CHANGELOG.md. Soma 2.8.5 fixes typed input and storage boundaries: implicit Int-to-Float promotion rejects overflow, typed maps check underscore keys and variant payloads, and declared sum/record returns validate their values. Persistent writes reject functions hidden in variants and excessive encoded nesting. Local nth index limits are consistent; sleep requires an Int. Eighteen new integration regressions plus a date-count unit test cover these corrections. See https://soma-lang.dev/CHANGELOG.md. Soma 2.8.4 corrects finite means that overflowed, large or tiny standard deviations, mixed Int/Float medians and clamps. Means round after averaging; medians preserve the selected Int and propagate NaN. Clamp rejects nonnumeric operands and NaN bounds, compares exactly, and preserves the selected operand's type. Nine regressions plus 320 independent numeric reference vectors cover these fixes. See https://soma-lang.dev/CHANGELOG.md. Soma 2.8.3 adds 100 regression scenarios for robot-controller use: numeric boundaries, mission transitions, rollback, native execution and persistence. They do not establish hardware qualification, hard real-time behavior or distributed safety. Matrix: https://soma-lang.dev/docs/robotics.md. Soma 2.8.2 cluster corrections: typed Map updates after commit, qualified slot identities, full-mesh reconnect, logical versions, persisted tombstones and state resynchronization. Local reads; no consensus, physical sharding, exactly-once signals or distributed proof. Unsupported strong/causal configurations are refused. See https://soma-lang.dev/docs/cluster.md. Soma 2.8.1 corrections (2026-09-20): integer while optimization preserves outer operands, each assignment and raised errors. Int ratios round once without inf/inf; variance retains nearby large-number differences. chr rejects invalid Unicode; substring clamps negative ends; index_of uses exact structural equality. Numeric builtins reject invalid operand types. Huge bit counts cannot become zero; native random bounds cannot overflow. Generated native temporaries avoid operand and loop-counter collisions. The deprecated --jit flag is now a compatibility no-op; use [native]. Also included in Soma 2.8.1: Int/Float comparisons are exact across interpretation, bytecode and native code; numeric sorting and filtering do not round Ints through Float. NaN remains unordered. `distinct` and `distinct_by` use structural equality, including nested collections. `range(a, b, 0)` raises kind `range`; `top`/`bottom` need a nonnegative Int. HTTP query `%2B` stays `+`, and `?flag` supplies an empty String. New replay logs tag errors separately from returned values. These are source-build fixes; consult CHANGELOG.md and version.json for build provenance. Cost proofs reject arithmetic overflow and a shadowed `range` needs an explicit loop bound. Invalid mock clocks fail tests; `days_in_month` needs Int arguments. Native failures are recorded after the transaction completes; replay refuses malformed logs and native compilation failures. Pick it when the task has a lifecycle or a limit that must hold: payments, approvals, interlocks, quotas, escrow, agent workflows with a budget — and you want to hand back evidence (`soma verify` output) instead of "I believe this is correct". Also for a small self-contained HTTP service: one file, no framework, storage and tests included. Skip it when you need a large ecosystem, a mature package set, or raw interpreter speed outside numeric `[native]` handlers. Soma is experimental: verification is per cell; cross-cell composition is linted, not proven. ## The agent loop ``` soma check app.cell # static gates FIRST: contracts, undefined # interpolation vars, unknown/ambiguous calls soma verify app.cell # prove state machines + memory invariants soma test app.cell # run `cell test` assertions (assert/assert_fails) soma run app.cell sig arg1 arg2 soma serve app.cell -p 8080 soma describe app.cell --faces # contract summary of every cell soma describe --builtins --json # exact builtin signatures — never guess soma fix app.cell # auto-repair common check errors soma docs agent # this file, offline, from the binary soma docs guarantees|serving|operations # proofs, HTTP rules, what goes wrong — offline too soma example invariant state_machine # verified programs with those features; soma example # …and the source of one soma init myapp # app.cell + soma.toml + AGENTS.md ``` Work in that order. `check` is cheap and catches most mistakes before anything runs; a `verify` failure is an error, not a warning. ## Cell anatomy ```soma cell App { face { // the contract signal add(a: Int, b: Int) -> Int // return types live HERE promise all_persistent } memory { data: Map [persistent, consistent] invariant data >= 0 // checked BEFORE every write; // between two slots of the cell (a rule no single slot can state): // reserved: Map [persistent] // invariant (reserved ?? 0) <= (data ?? 0) // — checked on every write to EITHER slot; the written slot is its new // value, the other is read AT THE SAME KEY (a key it lacks reads as // (): default both sides with `?? 0`). verify reports it // runtime-checked: a rule between slots is not proven by induction — // by design, so verify prints it as a `·` note and `--strict` passes. // A delete is a write here too (dropping the other side's entry takes // it to ()). `?? 0` means the absent side reads 0: write the // right-hand slot FIRST (or seed both in one handler), else the first // write to the other is refused. `other.size` is that slot's entry // count; `other.get(key)` is refused (it is the value BEFORE the // write, so it would not guard that slot). // Between two CELLS nothing is proven (verify is per cell): it lists // `note: cross-cell: …` for each handler that transitions its machine // while acting on another cell's — enforce those with a `require`. // To refuse AND keep what you wrote (an audit row), `return // refusal("conflict", "…")` instead of raising: same body and status // a raised error would give, without the rollback. } // violating write rejected, slot unchanged state flow { // model-checked by `soma verify` initial: draft draft -> approved { guard { amount < 1000 } } // guard: reads the CALLING handler's locals draft -> rejected // approved/rejected: terminal (no outgoing edge) } on add(a: Int, b: Int) { return a + b } // NO return type on handlers on approve_request(id: String, amount: Int) { transition(id, "approved") } // binds the guard's `amount` on reject(id: String) { transition(id, "rejected") } on request(method: String, path: String, body: String) { match map("method", method, "path", path) { {method: "GET", path: "/"} -> html("

hi

") {method: "POST", path: "/approve/" + id} -> approve_request(id, 500) // ONE variable per pattern, at the end _ -> response(404, map("error", "not found")) } } } ``` ## Core syntax (current) ```soma let x = 42 x += 1 // no semicolons; newlines separate let v = -1 // negative literals work let s = "hello {name}" // interpolation; UNDEFINED vars are // check-time errors; an unescaped " inside {} ends the string let xs = [1, 2, 3] xs[0] = 99 // bracket index read AND write let m = map("k", 1) m["k"] = 2 // maps too; m.k and m["k"] both read let g = Game { bet: 10 } g.bet = 20 // records: literal + dot mutation g.board[0] = 7 xs[i][j] = v // nested lvalues let y = m["nope"] ?? 0 // null-coalescing; () is null for i in range(0, 10) { } range(9, -1, -1) // stepped/descending ranges sum(xs) avg(xs) min(xs) max(xs) product(xs) // reductions over lists contains(xs, x) index_of(xs, x) slice(xs, 1, -1) keys(m) values(m) entries(m) // membership, slices, map views sort_by(rows, "field") sort_by(rows, r => [0 - r.total, r.name]) // stable; list key = several keys xs = push(xs, x) concat(a, b) // push/concat RETURN a new list; `+` on numeric // lists adds element-wise — it does not concatenate xs.sort() xs.reverse() m2.det() // UFCS: any builtin as method match x { 1 -> "a" n if n > 5 -> "big" _ -> "other" } // arms use -> let f = p => p * 2 f(3) // lambdas use =>; first-class data |> filter(x => x.ok) |> sort_by("score", "desc") |> top(10) let r = try { risky() } if r.error != () { ... } // error handling ``` ## Vectorized (numpy/MATLAB style) ```soma let M = [1, 0, 0, 1].reshape(2, 2) // matrices = List>, first-class A * B // matmul A + B / A - B elementwise A + 10 A / 2 // scalar broadcast (both orders) v * v v + v // numeric vectors: + * / - elementwise A > 2 v >= 1.0 // comparison masks (0.0/1.0 Floats); Int vectors compare and + - * exactly, [6] / 3 is [2] (the `/` rule per element) A.T A.shape det(A) matrix("1 2; 3 4") concat(a, b) // EXPLICIT list concatenation (+ on non-numeric lists concats) ``` ## Sum types (exhaustive — compiler refuses a missing arm) ```soma cell type Pay { variants { Charged { tx: String } Declined { reason: String } Cash } } match r { Charged { tx } -> "paid {tx}" Declined { reason } -> "no: {reason}" Cash -> "cash" } // typed state machines: state order: OrderState { initial: Placed ... } // transition(id, Variant) is compile-checked against the type // to_json(Charged { tx: "t1" }) == '{"_type":"Pay","_variant":"Charged","tx":"t1"}' // (a tuple variant carries "_values"); from_json brings the variant back; HTTP // answers a returned variant the same way. Slots store variants as variants. // A variant with named fields reads them by name too: `c.tx`. // RECORD type = a `cell type` with ONE variant with named fields: // cell type Line { variants { Line { sku: String, qty: Int } } } // on add(lines: List) { … } // a JSON object / Map becomes a Line // — from HTTP, a tool call or the CLI — field by field; a missing, extra or // mistyped field is a `type` error naming it (400). Several variants: the // client cannot pick one (`_type`/`_variant` are refused). ``` ## Coming from Python / TypeScript ```soma map("k", v) map() // no {k: v} literal; record: Name { k: v } m.k m["size"] // a key; m.len / m.size / m.keys: the entry count / keys // when the map has no such key (a record: () ) — m["size"] is always the key; // a List / String has only .len / .size (.first / .last); any other field raises kind `type` [1, 2, 3] [] // list literals x == () x ?? 0 // null is (); no null / None / undefined; `??` only runs its right side on () 5 == "" // raises kind `type` (Int vs String is a bug); nested values of different types are just unequal substring(s, i, i + 1) // s[i] as a String; str_at(s, i) is a BYTE (Int) if c { a } else { b } // is an expression; no ternary; && || ! (no and/or/not) // a condition (if, while, &&, ||, !, require, guards, // filter/any/all predicates, invariants) is a Bool — `()` // counts as false; a String/Int/List is a type error x => x + 1 // lambdas take ONE parameter for x in xs { } for k in keys(m) { m[k] } for e in entries(m) { e.key e.value } // for over () runs 0 times, over a String its lines, over a number: type error slice(xs, 1, 3) slice(xs, -2) // no xs[a:b] xs = push(xs, x) // push / concat / sort / with RETURN new values on f(n: Int) [native] { … } // annotations go AFTER the parameter list; no @decorator fail("bad_kind", "bad kind: {k}") // throw new Error(msg); catch: let r = try { … } r.detail == msg parse_int(s) ?? parse_float(s) // Number(s); to_float("5") prints 5.0; round(x) is an Int, round(x, 2) a Float m["k"] = v m.get("k") // a LOCAL map has no .set — .set/.delete/.push are slot methods f(opts: Map) … opts.x ?? 10 // keyword arguments / defaults: pass a map, read with ?? Name { k: v } is_a(r, "Name") // struct / dataclass / interface: a record literal (no class) format("%5.2f|%-8s|%03d", x, s, n) // printf / f-string widths (%d %s %f %.Nf, - and 0 flags) 7 / 2 == 3.5 idiv(-7, 2) == -3 // `/` is exact; idiv truncates toward zero, % follows the dividend's sign floor_div(-7, 2) == -4 mod(-7, 2) == 1 // Python's // and % ``` A `class` with private fields is a `cell` with `memory` slots: one instance, no `new`, no `this` — `this.hits.get(u)` is `hits.get(u)`; `constructor(a, b)` is a `configure(a, b)` handler writing a `config` slot; subclasses are the variants of a `cell type`, overridden methods one exhaustive `match` each. A script is `on main() { print(…) }` run with `soma run --fresh app.cell main`: `soma run` prints the handler's returned value (return nothing from main), and slots persist in `.soma_data/` between runs unless `--fresh` (one database per DIRECTORY: when other programs' cells have data there, `--fresh` resets only this program's tables); `soma test` starts empty. `counts[k] += 1` on a missing key is `() + 1` — write `(counts.get(k) ?? 0) + 1`; `for r in rows { r.v = 2 }` changes a copy (check warns). Number literals: `1_000_000`, `0xFF`, `0b101`; strings: `\n \t \r \" \\ \u{1F600}`. `soma check` names the Soma form for each of these when you get it wrong. ## Tests `every` / `after` blocks do not run under `soma test`: call the sweeper directly (`assert _sweep(5) == ["o4"]`) with `mock now` for the clock. ```soma cell test MyTests { // a name and a `rules` block are required; it may also // hold its own helpers (`on _walk(n: Int) { … }`), // private to that test cell (shared ones go in a plain cell) rules { let orders = fixture() // shared by the rules below; a bare call // is not a rule: bind it (let _ = f()) assert total(orders) == 42 // == is structural on lists/maps assert_fails pay("x") // passes when the expression RAISES assert_fails pay("x") matching "invalid transition" // …for the right reason mock think "billing" // next think() reply; or a list to queue mock think error "timeout" // next think() fails mock approve false // next approve() answer (one per call) mock price_check 120 // next call of the handler of that name (a tool, an mock Notifier.send error "down" // http wrapper) — Cell.h: THAT cell's only — answers / raises instead; // error "not_found: x" raises kind not_found mock now 1700000000 // freeze now()/now_ms()/today() until the next mock now (1700000000.25 or `mock now_ms …` for sub-second) property "sum" forall n: Int in 0..100 ensures f(n) >= 0 // 0..100 = 0…99, every value (≤ 20k); ONE Int variable over a range; iterations share the test cell's slots (a writing body sees earlier iterations' writes) } } ``` `soma test` isolates storage (fresh per run), runs `[native]` handlers natively, and — with no LLM key configured — mocks `think()` (echo: reply = prompt). A mock answers the NEXT matching call, even one in a later rule (a queue like `mock think ["a", "b"]` spans rules); a mock still unused after a rule that raised is discarded (with a note). A mock naming no handler, a test helper named like a program handler or a builtin, and test cells that assert nothing are errors. Each `cell test` starts with fresh slots, machines, mocks and trace(). `soma test --json` gives one record per rule. `approve()` never answers on its own: unscripted it raises `approval_required` (also under serve; SOMA_APPROVE=always|never is the explicit unattended policy; `soma run` at a terminal asks). `assert` needs a Bool. A test cell calls the handlers of every cell unqualified, or as `Cell.handler(…)`; slot methods (`m.get(k)`, `xs.len`) work there too. A failure prints the asserted source, `file:line`, left/right, and the fields a value really has when `x.field` came back `()`. ## Things you would otherwise guess - A fresh id is implicitly in the initial state: `transition(id, "held")` on an id nobody created is legal; `has_state(id)` is true only once the id has TRANSITIONED — a record stored in a slot but never transitioned is "fresh" to it (test the slot for duplicates: `require m.get(id) == () else Dup`). - `transition()`'s first argument is an INSTANCE id (an order, a ticket) — a String or an Int (`42` and `"42"` are one instance; `()` is refused), not the machine's name; one cell has one machine, any number of instances. A cell without a machine may call transition() / get_status() when the program has exactly one machine; that machine's guards then read the CALLING handler's locals (a check error if it does not bind them). - A slot is a Map or a List (`n: Int` is a check error); a handler parameter may not take a slot's name (reads and writes would mix the two). Read a whole slot by its bare name (`history`, `counts`); write with `counts.set(k, v)` / `counts[k] = v` / `history.push(x)`; `history = …` is an error. An invariant on a Map slot applies to every value written (`value` / `key` name the value and key being written — only inside an invariant; `slot.get(key)` there is the value BEFORE the write: `invariant value >= (versions.get(key) ?? 0)` keeps versions monotone); it is a pure condition (on a List slot `key` is the element's Int index) — no handler call, think(), transition(), I/O or network, even inside a lambda or an interpolation (a check error). - `r.kind` of a refused write is `"invariant"`; of a bad `transition()`, `"invalid_transition"`; of `require … else Tag`, `"Tag"`. - Every parameter has a type (`x: Any` takes a value of any kind — check it with `type_of(x)`); element types too (`xs: List`). - Declared parameter types are enforced at runtime: `add(points: Int)` given `2.5` raises kind `type` (400) before the body runs — no `is_int` check needed; a missing value `()` is refused for a List parameter (a Map parameter accepts `()`: an absent record, such as an empty tree). - Values are copies: `let row = s[k] row.count += 1` changes `row` only — write it back (`s[k] = row`); a map read from a slot is a copy too (`cards.set(id, c)` after editing `c`). Records keep their field order. - In `cell test`, `request(...)` is an ordinary call: an error it does not catch RAISES (assert it with `assert_fails request(...) matching "not_found"`); the kind → HTTP status mapping is applied by `soma serve` only. Return `response(status, …)` yourself when a test must see the status. - `"""…"""` is raw for escapes (`\t` stays two characters) but `{x}` IS interpolated; write `{{` for a literal brace. - `[1, 2] + [10, 20]` is element-wise (`[11, 22]`; Floats when a value is one); `concat(a, b)` joins lists. - Persistent data lives in `.soma_data/` beside the program (serve and run); `soma test` starts empty. A slot gives back exactly what it accepted (an Int beyond 64 bits stays an Int); a value that does not fit the slot's declared value type (`Map` given a String) is refused with kind `type`. - On a List slot: `rows.push(x)`, `rows[i] = x`, `rows.delete(i)` (by index), `rows.get(i)`, `rows.len` — all invariant-checked. A Map slot's `.keys` / `.values` / `.entries` come SORTED by key (every backend); a LOCAL map (`keys(m)` on a `let`) and a record's own fields keep insertion order. `keys(record)` / `values(record)` on a record (`Line { … }`) are its declared fields, in order; `r.field = v`, `r.field += 1` and `xs[0].field = v` check the field's declared type. - A bare call to a name two cells define (`pack(id)` with `Orders.pack` and `Warehouse.pack`) is a check error from a third cell: qualify it (`Warehouse.pack(id)`). Inside `Warehouse` the bare call runs Warehouse's own `pack` (check warns); write `Orders.pack(id)` for the other. - `xs[i]` is the same everywhere (local list, List slot, string): negative from the end, kind `index` out of range, kind `type` for a non-Int; `nth(xs, i)` and `rows.get(i)` answer `()` out of range instead (a negative `i` still counts from the end: `nth(xs, -1)` is the last). A `let` named like a memory slot hides the slot (check warns). - `emit ev(data)` inside one process is a synchronous, atomic fan-out to every cell with `on ev(data)`, the emitting cell included (a handler does not receive its OWN event: `on ping` emitting `ping` does not re-enter itself — recurse with a call): a listener that raises fails the emitter, and the emitter's rollback takes the listeners' writes with it. Only across processes (the `[peers]` bus) is an emitted event an effect that is not rolled back. `soma check` warns when nobody handles the event. - A bound is proven only against LITERALS: repeat the number in the invariant and in the `require` (there are no constants; a handler returning it is not used by the prover). - `invariant rows.size <= 500` (NOT `len(rows)`, which in an invariant measures the value being written; a bare `size <= 500` that names no slot bounds EVERY slot of the memory) is proven when the handler's ONE write that can add an entry (a `push`, or a `set` of a key not known to exist) comes after `require len(rows) < 500 else Full`, is not in a loop or a lambda, and no handler it calls (or event it emits) adds to the slot too; a `set` after `require rows.get(k) != () else Missing` (or in the `else` of `if rows.get(k) == ()`) cannot grow the slot. - An invariant is checked AT each write: a `require` proves only the writes that come AFTER it. A `require` counts for the writes of its own block and the blocks nested in it — `if ok { require n < 3 else Full m.set(k, n + 1) }` is proven; a require inside a loop body narrows that iteration's `let` locals only. - A trailing `Map` parameter is optional: `on add(a: Int, opts: Map)` is callable as `add(1)` (`opts` is `map()`) — the keyword-argument idiom, read with `opts.step ?? 1`. HTTP headers reach `request` as a fifth `headers: Map` parameter (lower-case names). - HTTP out: `http_get/post/put/patch/delete(url, body?, map("timeout", ms, "headers", map(...)))` never raise on the network (a timeout, a refused connection, a non-2xx answer come back as values); an invalid or unknown option raises kind `type` (a program bug); non-2xx is `{error, kind, status, body}` (kind `http_status` — redirects are followed up to 5 hops; a 3xx left unfollowed (a 307/308 answering a POST) is http_status too, a 6th hop is kind `network` (too many redirects) — | `timeout` — also a body that stalls — | `too_large` over `max_bytes` | `refused` | `network`); default timeout 30 s; invalid option values are refused; a String body is sent as text/plain unless it is JSON; `mock http_post …` in tests. - `verify --strict` bounds a write only from numbers it can bound: literals, slots with an interval invariant, and values narrowed by `require`/`if`. A number read out of a record (`a.balance`) is unbounded until a `require` narrows it; the ⚠ line says why (`` `v` = a.balance — not bounded ``, or `` `v` is only known to lie in [0, ∞) — narrow it: `require v <= 100 else …` ``). - `ensure cond` at the end of a handler raises kind `ensure` (422) when the postcondition is false — the whole handler is rolled back. In a `[task]` handler it undoes the CURRENT step only: writes made before the last think() are already committed (check warns). - Dates: `parse_date("2026-03-01")` → map(year, month, day, weekday, epoch_day), `add_days`, `add_months`, `days_between`, `months_between`, `days_in_month`, `format_date(ts)` (a date argument may also be an Int Unix time in seconds, as now() returns — validate client input as a String); ISO strings compare with `<`. Dates are UTC and there are no time zones or time-of-day builtins: keep a fixed offset per place (`let local_min = floor_div(now_ms(), 60000) + offset_min` — now() is seconds) and parse "HH:MM" yourself (`split(t, ":")`); `every` / `after` run in any cell, `request` or not. - Money: Ints in cents (any size, exact); `div_round(n, d)` is HALF_UP division (`div_round(cents * bps, 120000)` = BigDecimal interest at scale 2); `format("%10.2f", …)` only for display, `money = "{idiv(c, 100)}.{…}"` for exact text. - Data: `read_csv(path)` → List: RFC 4180 (quoted `"a, b"`, `""`, multi-line cells); unquoted cells auto-typed (`1.00` → 1.0), quoted cells and `007` stay Strings; `read_csv(path, map("raw", true))` keeps all text (exact money); `map("delimiter", ";")` reads `;` files (other options are refused); `from_csv(text, opts?)` parses CSV text already in memory (an upload); `to_csv(rows)` is the text `write_csv` writes (a download). `write_csv` quotes what a reader would change (a String "12" stays a String; a List cell is its JSON text). One builtin call builds at most 100,000,000 characters / matrix cells (`pad_left`, `format` widths, `zeros`…) and a List of at most 10,000,000 (`range`); an Int holds at most 2^24 bits (a product, shl, product(), parse_int / to_int of text past it is refused before it is built); matmul does at most 10^9 multiply-adds, pow_mod bits(exp) × bits(m) ≤ 2^30 — past any of these it raises kind `range`. Map slot keys are text (`m.set(1, …)` and `m.set("1", …)` are one entry). `use helper` imports helper.cell beside the program, `use lib::m` lib/m.cell (a `use` inside an imported file resolves beside THAT file); imported handlers are callable by name; an imported file's test cells run only under `soma test lib/m.cell`. `agg`, `sort_by`, `top`, `median`, `pstdev`; `stdev` / `variance` are SAMPLE (n − 1) like Python's statistics, `pstdev` / `pvariance` population. `|> map`, `|> filter`, `xs[i]`, `len(xs)`, `nth(xs, i)`, `xs.len` and a List slot's `rows[i]` / `rows.len` cost O(1) per call (20k rows in 0.1 s); passing a big list to a handler COPIES it (values are copies) — loop inside the handler, or pass an index. - `think_json()` raises kind `json` when the model does not answer a JSON object (catch it with `try`); a ```json fence around the object is fine. A mocked `think` costs ~4 characters per token, so `set_budget` exhaustion (kind `budget`) is testable offline. Under `soma serve`, `trace()` keeps the last 1000 steps process-wide. ## Scheduled work ```soma every 30s { expire_due() } // under soma serve: runs at start-up, then every 30 s (units: ms s min h d — 500ms, 90min, 24h, 1d); // a daily job at a time of day: `every 1h` + a "last run" slot compared with today() after 10s { warm_cache() } // once, 10 s after start-up ``` A tick is a handler invocation: atomic, serialized with requests (and with other processes on the same `.soma_data`), rolled back and logged if it raises; the next tick still runs. The interval counts from the end of the previous tick. verify checks the `transition()` targets of these blocks too. ## Handlers are atomic With SQLite or in-memory storage, a top-level handler invocation is all-or-nothing: if it raises, its slot writes and transitions are rolled back. A failing `try { }` block's slot writes, transitions and pushes are rolled back to where it started (plain locals it assigned keep their new value), and the handler continues. Under `soma serve` handlers run one at a time (a [task] handler: each step between think()s), so read-modify-write is safe (300 parallel payments of 10 against a balance of 1000 pay exactly 100). `[immutable]` on a slot makes it append-only: a List slot only grows by `push`, a Map slot only gains new keys; overwriting or deleting an entry raises kind `invariant` (an audit log that cannot be truncated or rewritten). With SQLite persistent slots the handler IS one SQLite transaction: a process killed mid-handler (`kill -9`) leaves nothing of it on disk. `every` / `after` ticks are handler invocations too (same rollback). You do not write compensation code. Not rolled back: effects outside the process (http calls, think(), emitted events). Storage read failures abort the invocation even inside `try`. The legacy JSON backend only replaces individual files; it has no multi-slot transaction or cross-process isolation. The HTTP provider adapter has no remote transactions, and a failed write may have reached the remote service. ## Errors ```soma fail("not_found", "reservation {id}") // raise: a kind + a detail require amount > 0 else InvalidAmount // raise with kind InvalidAmount (a bare tag) require open < 3 else LoanLimit "{member} holds {open} loans" // kind + interpolated detail let r = try { reserve(id) } // r = {value, error, kind, detail} if r.kind == "not_found" { return response(404, map("error", r.detail)) } if r.error != () { fail(r) } // re-raise unchanged ``` Kinds raised by the runtime: `invalid_transition`, `guard_failed`, `invariant`, `ensure`, `division_by_zero`, `index`, `llm`, `budget`, `type`, `json` (a body that is not JSON), `stack_overflow`, `approval_required` (`approve()` with nobody to answer). Branch on `r.kind`, never on substrings of `r.error`; in tests `assert_fails f() matching "not_found"` matches the kind or the message. ## HTTP `response(status, body)` is a map `{_status, _body}` (assert `r._status == 404`); `html(..)`, `redirect(..)` likewise. A handler may also return a plain map or string (200). `soma run app.cell request GET /path ""` calls the router without a server; `soma serve app.cell -p 8080` serves it. Static files: `static/`. `soma serve` ALSO exposes every public handler of the request-owning cell at `///` (that is how forms post to `/add`). A path that `request` matches explicitly goes to `request`; prefix handlers that must not be reachable over HTTP with `_` (`on _debit(…)`); `start`/`init` are never endpoints. A handler that changes state (a slot write, a transition, an emit, a call into another cell) answers GET/HEAD with 405 — POST to it. A request body may not carry `_type`/`_variant`/`_values` anywhere in its JSON, even for a `body: String` handler (a client cannot forge a record or a variant). Only a map built by `response()` / `html()` / `redirect()` is an HTTP response: a returned map with `_status` keys from client data is plain JSON. Response header values with CR/LF are dropped. A handler that calls `from_json` on other client text (a ws message, a decoded field) builds whatever declared variant it names — shape-checked (a missing or mistyped field raises kind `type`), but validate WHICH variant. Realtime: `on ws(msg: String)` (port+1, same-origin browsers only), `publish(stream, data)`, `sse(...)` routes — pushes (publish and emit) leave AT COMMIT, never from a rolled-back handler; an `emit` target is not an HTTP endpoint (`soma docs serving`, Realtime). `NaN`/`inf` in a path or query stay Strings. `soma check` warns when a handler and a route share a name unless the route delegates to that handler. `body: String` is the raw text (`from_json(body)`); `body: Map` is the parsed JSON (a non-JSON body is a 400 before the handler runs, kind json; `soma run` parses its text argument the same way). In a test cell pass the Map itself — `request("POST", "/x", map("a", 1))`: a String argument there is a `type` error, not parsed. A raised error answers with its kind: not_found → 404, unauthorized → 401, rate_limited → 429, guard_failed → 403, invalid_transition → 409, invariant → 422, your own `fail("tag")` / `require … else Tag` → 400, as `{"error": message, "kind": tag}`. A plain return answers 200: a map/list as JSON, `()` as `null`, a String or number as `{"result": …}` (a String that is valid JSON text, like `to_json(x)`, is sent as that JSON). Path segments and query values are coerced to the parameter's declared type (`/decide/x/true` → Bool). Every response carries `Access-Control-Allow-Origin: *`. `request` may be declared in `face` or not; its return type is not checked (a route answers whatever it returns). `serve` binds 127.0.0.1 (`--host 0.0.0.0` to expose), refuses a taken port and a program that fails check, and survives a handler's runaway recursion (500 `stack_overflow`; the process stays up). ## Verification properties `soma verify` proves, for every cell with a state machine: reachability, deadlock-freedom, liveness (every state can reach a terminal — or, with no terminal state, return to the initial one: a reactive machine passes --strict), refinement (each handler's `transition()` targets are declared states — the SOURCE of a transition is the instance's runtime state, so a removed edge is caught at runtime as `invalid_transition`, not by verify; declare `[verify.before.X] requires` for the edges you care about), and termination of every handler whose recursion has a decreasing argument — a handler it cannot prove (`while true`, `rec(n + 1)`) is a ⚠ warning, not a proof, and the runtime depth guard is what stops it. `property … forall n in a..b` in tests walks EVERY value when the range has at most 20,000 values; a wider range is only sampled (50 values, fixed seed — not a proof). State more in a `soma.toml` beside the program (`[package]` is optional; an unknown key is an error). The properties apply to every machine of the cells in `cells = [...]` (all cells when absent): a property whose TARGET state one machine lacks is vacuously true there (but `before.X requires [Y]` with X present and Y absent FAILS: X is reachable without Y) — output names each machine `Cell.machine` when there are several; split programs, or scope `cells`, when two machines need different properties: ```toml [verify] cells = ["Expense"] # optional: which cells these apply to deadlock_free = true # (always on; the key is accepted) eventually = ["paid", "rejected"] # every path reaches one of these (AF) never = ["corrupt"] # unreachable states (AG not) always = ["a", "b", "c"] # the machine is only ever in one of these [verify.after.rejected] never = ["paid"] # `paid` is unreachable once `rejected` was reached eventually = ["archived"] # after `rejected`, every path reaches `archived` [verify.before.paid] requires = ["manager_approved"] # precedence: no path reaches `paid` without ONE of these requires_all = ["held", "picked"] # …without EACH of these ``` In the machine, `* -> failed` means from EVERY state — it also adds `paid -> failed`, so `paid` is no longer final (verify says so). Keep states final with `* -> failed except [paid, denied]`. A property naming a state no machine declares is an error, and verify ends with one verdict line (`VERIFY OK` / `VERIFY FAILED — …`). `soma verify` and `soma test` refuse a program that fails `soma check`. Invariants on computed values are proven by induction when interval reasoning suffices (`(counts.get(k) ?? 0) + 1` keeps `counts >= 0`; `let open = … require open < 3 else Full set(k, open + 1)` keeps `<= 3` — an unconditional `require` on a once-bound local counts); the rest is reported per conjunct as runtime-checked. `soma verify --strict` fails on every ⚠ too (a proof that degraded to runtime-checked, an unprovable termination) — the CI gate. A failed property prints a counter-example path and is an error. What verify does NOT prove: data-dependent rules inside handlers, and invariants on computed values (those are runtime-checked — reported as such, never as "proven"). Guards (`a -> b { guard { cond } }`) are enforced at runtime and read the locals of the handler that calls `transition()` — so EVERY handler whose `transition()` targets the guarded state must bind those names (check error otherwise), even one meant for another edge; the model keeps guarded edges. A guard is a pure condition: calling a handler, think(), next_id() or a slot write (`.set`/`.delete`/`.push`) inside it is a check error — compute the value in the handler before `transition()` and test that local. The calls in a `require` condition and in its `else` detail are part of the termination, cost and invariant analyses like any other statement. A `cost { tokens: N }` bounds the REPLY tokens (the sum of the max_tokens caps of the think() calls a handler can make — ×10 provider rounds in an agent with tools, or × `map("max_rounds", N)`); prompt tokens are not in it — `set_budget` / `tokens_used()` count prompt + reply, per request under serve and per rule in a test (return `tokens_used()` from the handler to assert on it); `tokens_remaining()` is -1 without a budget, never below 0 with one; a `set_budget` reached from a model's tool call (a delegated agent's own) only lowers what is left — it never lifts or resets the caller's. A cost bound that cannot be proven (a think() without a literal max_tokens) is a check ERROR. ## Agents (LLMs inside the cage) ```soma cell agent Researcher { face { signal research(topic: String) -> Map tool search(q: String) -> String [capability: "https://api.x.com/*"] "Search" } cost { tokens: 6000 } // PROVEN by soma check: 2000 × 3 rounds state job { initial: idle idle -> working working -> done * -> failed except [done] } on search(q: String) { return to_json(http_get("https://api.x.com/search?q={q}")) } on research(topic: String) { set_budget(8000) // token cap: a prompt that alone overruns what is left raises kind budget before it is sent; otherwise the provider round that crosses it completes and the NEXT round (a later tool round of the same think() too) raises kind budget transition(topic, "working") // one machine instance per topic let r = try { think("Research {topic}", map("max_tokens", 2000, "max_rounds", 3)) } if r.error != () { transition(topic, "failed") return map("error", r.kind) } transition(topic, "done") return map("facts", r.value, "spent", tokens_used()) } } // `on ask(q: String) [task] { … think(…) … }` runs as STEPS: each think() ends // the current step (its writes commit) and waits for the model OUTSIDE the // handler lock, so concurrent requests overlap their model calls (200 × 2 s // waits in ~9 s instead of 400 s); a failure rolls back the current step only // (writes before the last think() stay — a `try` cannot undo them either); // the prover does not carry a read or `require` across a think (another // request may write meanwhile): read, require and write AFTER the last think. // Only the ENTRY POINT decides: a [task] handler called from a plain handler // (`on request` routing to it, a listener, a tick) runs inside the caller's // atomic unit and holds the lock — mark the caller [task] too: // `on request(method: String, path: String, body: String) [task]`, // `every 1min [task] { … }`, `after 5s [task] { … }`. check warns on each // of these. SOMA_LLM_MOCK_LATENCY_MS gives a mock a latency to test it. A // plain handler stays one atomic unit. Unknown handler annotations // ([tsak]) are errors; known: [native], [task], [deterministic], [record]. // HORDES — one [task] handler over many inputs with a bounded pool: // let h = horde(Reviewer.review, docs, map("concurrency", 200, // "on_result", "_store", "on_done", "_summarize", "on_error", "_failed", // "max_attempts", 2)) // returns "Audit:h1" at once // on _store(v: Map) { verdicts.set(v.id, v) } // or _store(input, result) // horde_status(h) → {state, total, queued, running, done, failed, cancelled, tokens} // horde_results(h) → results in input order (() if not done); horde_cancel(h) // The target takes one parameter; mark it [task] (check warns when it // thinks: each task would hold the lock). Write the target and the options // AT THE CALL (a literal map with literal handler names — check refuses a // computed target or options: they decide what runs and what it costs); // name callbacks `_…` (a public one is an HTTP endpoint). Only the owner // cell (and test rules) can read or cancel a horde. The queue is stored in // soma.db under serve / run, even without a [persistent] slot: a restart // resumes it; with several servers on one .soma_data each horde runs in one // of them (a lease, taken over ~6 s after its server stops). A task's // result is recorded — and on_result called — in the same atomic unit as // its LAST step, so each result is written once even after kill -9; steps // before its last think() may run again. on_error(input, error: Map // {error, kind, detail}) after max_attempts (default 1); on_done(id) once, // when nothing is left (cancel included). concurrency default 8, max 1000. // Without workers (`soma test`) the horde runs right after the calling // handler commits — its next statements run first, as under serve — one // task per unit. A restart does not resume a horde whose target or // callbacks the new code renamed (it says so; the tasks wait). // BUDGET: map("budget_tokens", 2000000) is a hard ceiling for the whole horde — every think() of its // tasks and callbacks first RESERVES an upper bound (request bytes + // max_tokens), waits outside the lock for calls in flight to settle if it // does not fit yet, and is refused (kind budget) when it never will; the // horde then stops (state "exhausted"; tasks it stopped count as // cancelled, no on_error). A horde started inside a task or callback of a // budgeted horde runs under that budget too (nested hordes cannot spend // past it). on_done's think() also counts: keep headroom for it. A // provider that ignores max_tokens raises kind llm, charged what it // reported. The ceiling counts SETTLED calls: calls in flight // at a kill -9 were sent, then are sent again after the restart. soma verify prints each horde's bound: // the literal budget_tokens, else inputs × per-task × max_attempts for a // literal list/range (reply tokens), else unbounded — a horde in a cell // with `cost { tokens: N }` needs a literal budget_tokens to prove it. // ROUNDS (simulations): map("snapshot", world, "apply", "_apply", "seed", 7, // "instance", "id") — the target takes (input, snapshot): every agent of // the round sees the same world; `apply(result)` / `apply(input, result)` // runs at the END, in input order, once per task (not as replies arrive), // then on_done — which may start the next round (a horde() in on_done). // seed: random() in task i draws the same numbers on every run. instance: // the input field naming an agent; its remember()/recall() keys and its // conversation are its own and persist across rounds. 10 000 agents × 20 // rounds: 46 s, identical on every run. After a crash, steps before a // task's last think() may run again — write memory in the last step. // vote(Cell.handler, input, k) → {winner, count, k, unanimous, errors, // votes}: k agents on one input (in a [task] step under serve/run they run // at once, outside the lock); the most common answer wins. Provider limits // for every think() of // the process: soma.toml [agent] rpm = 500, tpm = 200000 (or SOMA_LLM_RPM / // SOMA_LLM_TPM): at most rpm requests / tpm tokens in ANY 60 s window; a // 429 pauses every caller. Measured: 10 000 tasks with a // 2 s mocked model, concurrency 500, in 41 s. // soma verify PROVES the lifecycle; `cost { tokens: N }` (also latency:, // usd:) is PROVEN when every think() has a literal max_tokens and runs a // known number of times — with tools, × the rounds (`max_rounds`, else 10). // `latency` counts WAITING: each think()'s `timeout` (a total, retries // included — the wait for a rate limit and a mock's latency too) × its // rounds, http timeouts, literal sleep()s, vote() × k; approve(), file/stdin // reads and computed sleeps make it advisory; CPU time is not in it. `usd` // is the REPLY tokens × the price the table knows for the cell's model // (prompts are not counted; a model the table does not know makes it // advisory) — an estimate of the provider's bill, not the bill. // The model may call only the face's `tool`s (never a private or other // handler), with typed arguments, within their capabilities (URL patterns with // `*`: "https://api.x.com/*"; a tool with no capability is unrestricted; a // tool that calls think() makes a `cost` bound unprovable, a tool doing I/O a // `latency` bound; I/O in a loop or lambda leaves a `tokens` bound provable) — the scope binds // MODEL calls only (your own code calling the tool handler is not // restricted), and a tool is not an HTTP endpoint; `mock think` scripts text // replies only (to test tool calls offline, point [agent] url at a fake // OpenAI-chat-compatible server: tools go out as `tools`, results come back // as `role: "tool"` messages, think_json sets response_format json_object, // usage.total_tokens feeds tokens_used(), usage.completion_tokens is held to // max_tokens); a tool call that raises is rolled // back. `map("max_rounds", N)` caps the tool-call rounds of one think(); // `map("tools_allowed", ["lookup"])` offers only those tools to that call // (classify with no refund tool in reach); a `*` in a capability stays in // its part of the URL (host or path), `..` segments and userinfo are // refused, and a tool's scope holds for the tools of any agent it calls; // a scoped tool reads/writes no file (read_file, load, include, …) and opens // no link/socket. An unknown option, or a tools_allowed name that is not a // tool, is an error. A reply longer than its max_tokens (a provider ignoring // the cap) raises kind `llm`; a tool that calls back its think() handler is a // termination ⚠ (the model drives that recursion). Each agent cell keeps its OWN multi-turn // context (clear_context() resets it); every/after ticks use the same // [agent] config as requests. `soma test` mocks think() only when no key is // configured — set SOMA_LLM_MOCK=echo for offline tests with a soma.toml key; // `mock h [[1, 2]]` scripts ONE List answer ([1, 2] is a queue of two). // approve(msg) is a gate a PERSON answers: tests script it (mock approve), // serve raises approval_required unless SOMA_APPROVE says otherwise (and // the handler rolls back) — for a web flow, commit a pending state in one // request and decide in another (POST /decide/…); approve() is for run/test. // think(prompt, map(..)) or think(prompt, system, map(..)): options go LAST. // Offline: `mock think …` in tests, SOMA_LLM_MOCK=echo | fixed:TEXT | // rules:mocks.json — a list of {"match": "substring of the prompt", // "cell": "AgentCell", "reply": "text" or a JSON value}: the first rule whose // match / cell fit answers, none: an echo — or // [agent] mock = "echo" in soma.toml. Record/replay: `soma run --record` (not serve) logs each top-level // call; `soma replay` re-runs them from EMPTY storage with the soma.toml // [agent], calls think() again (replies are not recorded) and compares the // RESULTS — a state difference shows only in a later result. ``` ## Top gotchas (full list: /docs/gotchas.md or /gotchas.json — all verified) 1. `f(args)` calls YOUR handler `f` when one takes that many arguments, the builtin `f` otherwise (`on list() { return list(1, 2) }` reaches the builtin). `soma check` warns when an argument count sends a call to a builtin you probably did not mean. Another cell's handler: `Ledger.post(x)` (or bare `post(x)` when the name is unique); `get_status(id)` answers the initial state for an unknown id — `has_state(id)` tells them apart. 2. `match` arms use `->` (`=>` is lambdas). Handlers don't declare return types (face signals do). 3. An unescaped `"` inside `{...}` ends the string — escape it (`{m[\"k\"]}`) or bind a let first. 4. `==` is structural on lists, maps and variants (`[] == []`, key order irrelevant). `null`/`None` do not exist: null is `()`, test with `x == ()` or default with `x ?? 0`. 5. `transition()` returns {id, from, to}; read state with get_status(id). Wrap fallible transitions in try { }. 6. Slot methods (.set/.get) are for `memory` slots; local maps use m[k]. 7. `assert_fails expr` needs an expression that RAISES (not a falsy bool). 8. serve routes only the cell that owns `request` — put routable signals there, delegating to domain cells. 9. Adjacent string literals don't concatenate (`return "a" "b"` is a check error). Language keywords (on, given, cell, agent, tool, use, require, promise, let, if, match, return, …) are not names — `soma check` says which one and how to rename it. Reserved HANDLER names (they would replace the builtin): transition, approve, fail, get_status, has_state, valid_transitions, think, think_json, set_budget, tokens_used. 10. Floats: compare with a tolerance, never ==. 11. `7 / 2` is `3.5` — everywhere, `[native]` included. `idiv(7, 2)` is the integer quotient (truncates toward zero, BigInt-exact). `pow` is a Float power; `ipow(3, 40)` is the exact Int power (no `**` operator). 12. One memory invariant names one slot; `size` invariants also guard `delete`. A cost bound is *proven* only when every `think()` runs a known number of times (literal `range`, or an annotated loop: `for [loop_bound(50)] x in xs { … }`, `while [loop_bound(50)] cond { … }` — checked: more iterations raise kind `loop_bound`); a declared bound that cannot be proven is a check ERROR. ## Packages ```toml [dependencies] matrix = "^0.2" # semver; registry at soma-lang.dev/repo local = { path = "../pkg" } # or git = "...", subdir = "packages/x" ``` `soma install` → .soma_env/packages/, commit-pinned in soma.lock with a sha256 of the files (sub-directories included): a package modified after install, or holding a file the lock does not list, is refused at import (`soma install` restores it); an imported cell that defines `request` or `ws`, or declares `every` / `after`, is a check warning (it would own HTTP routing / the WebSocket port). `use matrix` imports it. A package's API = its face; its proofs (cell test) re-run on YOUR toolchain: `soma test .soma_env/packages/matrix/matrix.cell`. ## Performance Interpreter is the reference semantics (linear-time `map`/`filter`/indexing; a 20k-row CSV aggregation runs in ~0.3 s interpreted). Annotate hot handlers `[native]` for Rust-codegen: measured at rustc -O parity (16.8 ns/op) on numeric loops; a native compile failure aborts the run (there is no silent fall-back to the interpreter). It needs `cargo` (rustup) on PATH; the first compile takes seconds and is cached in `.soma_cache/` beside the program. Arguments are type-checked at the call like any handler; `random(n)` / `random(lo, hi)` are Ints, `random()` a Float; buffer indexes are Ints from 0 (no negative indexes); an Int literal beyond 64 bits is refused (build it: `shl(1, 64) - 1`); `[loop_bound(N)]` is checked; a `[native]` handler's recursion deeper than 20,000 calls raises stack_overflow (interpreted handlers: 512 — walk deep structures with an explicit stack). Floats print with the shortest round-trip digits; print/to_string never use exponent form (`1e-7` prints `0.0000001`, a Float inside a printed map or list too; `format("%.3e", x)` for scientific notation); to_json — HTTP bodies included — uses JSON's shortest form (`1.18e21`). An SSE `data:` line and a WebSocket push carry the value as JSON on one line (a String payload is quoted, so a client cannot forge another event). Native math: sqrt, log, exp, pow, abs, min, max, floor, ceil, round, sin, cos, tan, atan, atan2, asin, acos, pi() (asin/acos outside [-1, 1] raise kind range, as interpreted). `soma check` verifies the native vocabulary (every handler, every error at once). Same semantics as the interpreter: `7 / 2` is 3.5, a division by zero is an ordinary `try`-catchable error, an Int overflow promotes to BigInt, and an Int slot refuses a non-exact `/` instead of truncating. One difference: native code is statically typed: a quotient returned or stored as an Int is the exact Int (`return a / b` gives -7 for -7 / 1; an exact quotient past 2^53 raises natively — use idiv(a, b) there), but inside an expression that needs its type — `to_string(a / b)` — it is a Float ("-7.0") — `soma check` warns; write idiv(a, b) for an integer quotient. A native local keeps the type of its `let` (assigning a String to a number local is a check error). What a `[native]` handler may use — nothing else: - parameters and locals: Int, Float, Bool, String (a List cannot cross the boundary in either direction: parse the input into buffers inside the handler, return a String) - arithmetic, comparisons, `if`/`while`/`for i in range(a, b)`, `return`, sqrt log exp pow abs min max floor ceil round sin cos random, idiv gcd pow_mod sqrt_int, band bor bxor bnot shl shr bit_len bit_test bit_set bit_clr bit_next, to_int to_float to_string, str_len str_at str_eq, sibling `[native]` handlers - arrays: `buffer(n) -> Buf` (n Ints, zeroed), `buf_get(b, i) -> Int`, `buf_set(b, i, v)`; `buffer_f(n)` / `buf_get_f` / `buf_set_f` for Floats - maps: `hashmap() -> HMap` (Int → Int), `hm_get(m, k)`, `hm_set(m, k, v)`, `hm_inc(m, k)`, `hm_len(m)`, `hm_has(m, k)` - strings: `strbuf() -> SBuf`, `sb_push(b, s)`, `sb_push_int(b, n)`, `sb_push_char(b, c)`, `sb_len(b)`, `sb_finish(b) -> String`; `regex_count / regex_replace / regex_match`; `read_file read_stdin write_str` (write_str flushes; an Int overflow re-runs the handler in BigInt mode, so text written BEFORE the overflow is repeated — keep write_str away from overflow-prone arithmetic, or return the text) - a Buf/HMap/SBuf lives in ONE variable of ONE handler: it cannot be re-bound (`let t = u u = v` — copy elements instead), returned, or passed to a sibling (pack results into a String with strbuf). A Buf holds 64-bit Ints: a value that overflows i64 while stored into it is a `range` error (scalars promote to BigInt; mask the value or keep it scalar). `for` takes `range(a, b)` only (a step → `while`). A literal `/ 0` does not compile. `soma check` reports all of these. - one FFI call costs ~2 µs: make the LOOP native, not just the per-element kernel (10^7 sibling calls inside a native loop: 20 ms; 2·10^6 calls from an interpreted loop: 3.5 s). Not available natively: maps/lists/records, think(), memory slots, `try`, `match`, string interpolation — call an interpreted handler for those. The buffer / hashmap / strbuf primitives exist ONLY in `[native]` handlers (`soma check` refuses them elsewhere); everything else in the list, `sin`, `regex_*`, `read_stdin` included, works in interpreted handlers too. ## Learn from working code - https://soma-lang.dev/corpus/index.json — 300+ programs, all passing check+test (algorithms, web, agents, state machines, finance, games, data, safety interlocks, medical, aerospace…). Before writing a cell, fetch one with the features you need and adapt it. - treasury/ airlock/ poker/ elevator/ hft/ delivery/ — complete apps where the safety property is a verified theorem - packages/matrix/ — the reference package (numpy-style linalg, 43 proofs) --- # Robotics robustness audit — 100 scenarios These scenarios exercise Soma behavior relevant to robot controllers. They do not qualify hardware, real-time performance or a lunar system. Each scenario is a separate Rust integration test in `compiler/tests/robotics_robustness.rs`. ## Results — Soma 2.8.3, 2026-09-20 - Robotics campaign: **100 passed, 0 failed, 0 ignored**. - Complete Rust suite: **523 passed, 0 failed, 0 ignored**. - Core CLI fixtures: **118 passed, 0 failed**. - Published corpus: **321 programs passed the website build checks**. Measured locally on macOS ARM64. Linux CI reruns the full suite on every push and pull request. This campaign confirmed the tested behavior without finding an additional runtime defect. The source of every scenario is in [robotics_robustness.rs](https://github.com/soma-dev-lang/soma/blob/v2.8.3/compiler/tests/robotics_robustness.rs). ## Scope and reproduction Added in Soma 2.8.3 on 2026-09-20. This is a regression campaign for the language, with inputs chosen for robot-controller use. The motor and sensor values are simulated data; no physical hardware is driven. Run all 100 independently named scenarios from the repository root: ```sh cargo test --release --locked --manifest-path compiler/Cargo.toml --test robotics_robustness ``` Run them with every existing Rust test: ```sh cargo test --release --locked --manifest-path compiler/Cargo.toml --no-fail-fast ``` IDs 001–080 cover numeric boundaries, sensor validation, mission transitions, storage invariants, rollback and collection/JSON boundaries. IDs 081–090 test the distinction between static rejection, proof, runtime checks and unproved distribution. IDs 091–095 run native handlers, including comparison with the interpreter. IDs 096–099 use fresh processes to check committed data and rollback persistence; 100 distinguishes recorded errors from returned data. `verify_fail` and `check_fail` mean the expected rejection is asserted, including its diagnostic. A refusal is a passing test, not a waived failure. The separate existing `compiler/tests/cluster.rs` suite covers actual network replicas, seed failure, reconnect, stale updates, restart, and partition recovery. The 100 scenarios below do not add consensus or distributed safety proofs. ## Limits These are finite examples, not a proof for every input. They do not measure worst-case execution time, memory exhaustion, radiation tolerance, sensor or actuator faults, abrupt power loss, or hardware fail-safe behavior. Restart tests exit processes normally after a commit or a rejected handler; they do not inject power loss during a disk write. Local transaction rollback cannot undo a command already sent to a physical actuator. The cluster remains experimental and eventually consistent, with local reads and no fenced scheduler. Use the [guarantees](https://soma-lang.dev/docs/guarantees.md) and [cluster limits](https://soma-lang.dev/docs/cluster.md) when assessing a design. | ID | Scenario | Check | |---|---|---| | 001 | encoder counts above float precision | runtime | | 002 | odometer integer promotion | runtime | | 003 | negative odometer promotion | runtime | | 004 | wide distance products | runtime | | 005 | large finite calibration ratio | runtime | | 006 | negative fixed point division | runtime | | 007 | subnormal measurement ratio | runtime | | 008 | nearby large sensor variance | runtime | | 009 | high dynamic range median | runtime | | 010 | exact message sequence sorting | runtime | | 011 | divide zero is catchable | runtime | | 012 | integer divide zero is catchable | runtime | | 013 | remainder zero is catchable | runtime | | 014 | nan rejected by sensor guard | runtime | | 015 | positive infinity rejected by sensor guard | runtime | | 016 | negative infinity rejected by sensor guard | runtime | | 017 | nan cannot become integer command | runtime | | 018 | huge integer cannot silently become infinity | runtime | | 019 | non numeric actuator math is rejected | runtime | | 020 | failed math does not poison later commands | runtime | | 021 | sensor guard accepts exact endpoints | runtime | | 022 | sensor guard rejects out of range | runtime | | 023 | missing telemetry is not zero | runtime | | 024 | false telemetry is not missing | runtime | | 025 | zero measurement survives coalescing | runtime | | 026 | control vector arithmetic | runtime | | 027 | vector shape mismatch is rejected | runtime | | 028 | large sequence dedup is exact | runtime | | 029 | sensor maps compare structurally | runtime | | 030 | no sensor fusion from empty set | runtime | | 031 | new rover is docked | runtime | | 032 | nominal mission round trip | runtime | | 033 | cannot drive before arming | runtime | | 034 | repeated arm is not a second command | runtime | | 035 | low power prevents arming | runtime | | 036 | abort is terminal | runtime | | 037 | abort stops driving motor | runtime | | 038 | different robots have independent states | runtime | | 039 | cannot skip to sampling | runtime | | 040 | unknown transition is rejected | runtime | | 041 | battery bounds are inclusive | runtime | | 042 | battery underflow preserves old value | runtime | | 043 | battery overflow preserves old value | runtime | | 044 | invalid battery type is rejected | runtime | | 045 | nan motor command is rejected | runtime | | 046 | infinite motor command is rejected | runtime | | 047 | motor bounds are inclusive | runtime | | 048 | command queue capacity is enforced | runtime | | 049 | immutable audit cannot be rewritten | runtime | | 050 | immutable audit cannot be deleted | runtime | | 051 | failed sample rolls back state and data | runtime | | 052 | failed batch rolls back every slot | runtime | | 053 | create then delete is undone | runtime | | 054 | delete rollback restores existing record | runtime | | 055 | repeated overwrites restore original | runtime | | 056 | failed immutable append does not burn id | runtime | | 057 | try is a nested savepoint | runtime | | 058 | error kind and detail survive catch | runtime | | 059 | short circuit and avoids invalid read | runtime | | 060 | short circuit or avoids invalid read | runtime | | 061 | negative list index reads last | runtime | | 062 | out of range list index raises | runtime | | 063 | fractional list index is not truncated | runtime | | 064 | invalid range step is rejected | runtime | | 065 | long command stream can stop early | runtime | | 066 | queue elements obey declared type | runtime | | 067 | map cannot silently accept list push | runtime | | 068 | json preserves large command id | runtime | | 069 | json overflow is rejected | runtime | | 070 | untrusted json cannot forge unknown variants | runtime | | 071 | empty string value is present | runtime | | 072 | null value is distinct from missing key | runtime | | 073 | null cannot be an accidental key | runtime | | 074 | private storage keys are reserved | runtime | | 075 | key and value views stay consistent | runtime | | 076 | missing delete has no side effect | runtime | | 077 | text command ids are not coerced | runtime | | 078 | nested map update does not alias another value | runtime | | 079 | unicode command payload roundtrips | runtime | | 080 | integer clock difference stays exact | runtime | | 081 | unbounded recursion is not a proof | verify_fail | | 082 | guarded counter induction is provable | verify_ok | | 083 | invalid state target is static error | check_fail | | 084 | unsafe literal write cannot verify | verify_fail | | 085 | dynamic state target cannot pass strict | verify_fail | | 086 | replica count does not prove consensus | verify_fail | | 087 | eventual cluster is not a strict safety proof | verify_fail | | 088 | missing handler is rejected | check_fail | | 089 | proven decreasing recursion terminates | verify_ok | | 090 | typed boundary mismatch is reported | runtime | | 091 | native encoder add promotion | runtime | | 092 | native distance multiply promotion | runtime | | 093 | native negative integer division | runtime | | 094 | native zero divisor is catchable | runtime | | 095 | native wide bit shift preserves encoder bits | runtime | | 096 | committed telemetry survives process restart | process | | 097 | failed handler leaves no persistent writes | process | | 098 | failed transition leaves no persistent state | process | | 099 | immutable audit survives process restart | process | | 100 | record replay preserves failure classification | process | --- # Experimental cluster runtime The replication protocol described here is available since Soma 2.8.2 and is included in the current [Soma 2.8.10 release](https://github.com/soma-dev-lang/soma/releases/tag/v2.8.10). ## What is implemented A `scale` block with `shard: records` and `consistency: eventual` selects a mutable Map for full replication. Despite the syntax name, entries are not physically partitioned: each connected replica eventually holds the selected slots. All cells are supported; `Cell.slot` identities keep same-named slots separate. Other slots stay local. A resource-only `scale { memory: "32Mi" }` does not enable the cluster or open a bus port. Updates retain their stored types, including large integers, nested values, empty strings and null. A failing handler or `try` block publishes no rolled back writes. Remote writes are serialized with local handlers and checked against the declared slot type. Replication cannot target an unselected slot; legacy `_cluster_*` events cannot directly mutate storage. Each key carries a `(Lamport counter, originating node ID)` version. The higher version wins, including on a deletion. Reordered and duplicate updates do not overwrite a newer version. This is a deterministic conflict rule, **not wall-clock last-writer order**. Concurrent read-modify-write operations can lose an application's increments. There are no cross-node transactions, multi-key snapshots, linearizable reads, quorum acknowledgements or consensus. `get`, `has`, `keys`, `values` and `len` read the local replica. Discovery builds direct links between members, so leaves can communicate after a seed stops. Each outgoing link retries failed connections with a 1–30 second backoff. State is exchanged on connection and roughly every three seconds thereafter, including deletion markers. Persistent slots require SQLite: data and versions commit in the same transaction. Each replica must have its own project/data directory. Ephemeral data and its versions are lost when the process stops. Keep the whole `.soma_data` directory in backups; deleting cluster metadata can resurrect stale values. ## Start two nodes Save this as `app.cell` in **two distinct directories**, `node-a` and `node-b`: ```soma cell Replicated { memory { records: Map [persistent, consistent] } scale { replicas: 2 shard: records consistency: eventual tolerance: 1 } on put(key: String, value: String) { records.set(key, value) } on get(key: String) { return records.get(key) } } ``` Run in separate terminals: ```sh cd node-a SOMA_NODE_ID=127.0.0.1:8082 soma serve app.cell -p 8080 ``` ```sh cd node-b SOMA_NODE_ID=127.0.0.1:8092 soma serve app.cell -p 8090 --join 127.0.0.1:8082 ``` The bus uses HTTP port + 2. Choose a fixed HTTP port in 1–65533 and avoid HTTP, dashboard and bus port overlap. Across machines, bind the appropriate interface with `--host` and set `SOMA_NODE_ID` to an address reachable by peers. Seeds may also come from `SOMA_SEEDS` or `[cluster] seeds` in `soma.toml`. A successful protocol acknowledgement establishes membership; an unreachable or incompatible seed is retried, never reported as joined. The cluster bus has no authentication or TLS: use a trusted private network. Do not connect it to untrusted clients or mix unrelated applications. ## Scheduling and signals After membership settles, `every` checks the lowest live node ID at each tick. Missing heartbeats expire after 15 seconds, checked every three seconds. The surviving node can then take over. This is **advisory leadership**: discovery delays and partitions can produce simultaneous ticks. There is no fencing, consensus lease or exactly-once scheduler. Committed `emit` events reach connected cluster peers once per direct link. They are fire-and-forget and are not part of state resynchronization; use application IDs, deduplication and an outbox for reliable workflows. Do not also declare the same nodes in `[peers]`, which creates a second event path. ## Verification and upgrade `strong` and `causal` are rejected: the runtime does not implement their protocols. Replicated Lists, immutable slots and memory invariants are also refused. `replicas` and `tolerance` are declarations, not provisioning or fault-tolerance proofs. `soma verify --strict` fails on the explicit warning that distribution is unproved; a successful non-strict run is not a cluster proof. Protocol v2 is incompatible with 2.8.1 and earlier cluster traffic. Stop all nodes and upgrade them together, retaining each data directory. Existing local entries acquire replication metadata at startup. Do not run older binaries or use `soma run` to mutate a live replica's store. This implementation uses a full mesh, periodic full-state exchange and retained tombstones, with at most 256 discovered peer addresses per node. An encoded update must fit the 16 MiB bus line limit; larger local writes are refused and rolled back. It is intended for small experimental clusters. It has no demonstrated large-cluster throughput or bound on tombstone growth. ## Regression coverage `compiler/tests/cluster.rs` starts real server processes with distinct stores. It covers initial sync, types, rollback, same-named slots, event duplication, concurrent writes during a TCP partition and convergence after healing, seed failure, stale replica restart, tombstones, reordered updates, private slot boundaries and scheduler takeover. Unit tests check hash-ring balance/minimal movement and monotonic heartbeat expiry. These are regression tests, not a model-checked proof of arbitrary partitions. --- # PART 2 — Language reference (SOMA_REFERENCE.md) # Soma Language Reference — for AI Agents > Give this file to an AI agent as context when asking it to write Soma code. > The authoritative builtin list is `soma docs builtins` (published as /docs/builtins.md and /builtins.json). ## Quick rules - No semicolons. Newlines separate statements. - No `function`/`def`. Use `on handler_name(params) { }`. - No `null`. Use `()` for null/unit. - Lists: `[1, 2, 3]` and `list(1, 2, 3)` both work. - No `{key: val}`. Use `map("key", val, "key2", val2)`. - No `import`. Use `use lib::module`. - No `console.log`. Use `print(value)`. - No `===`. Use `==`. - Strings: `"hello {name}"` (interpolation with `{}`). - Multi-line strings: `"""..."""` (backslashes are literal, quotes work inside; `{x}` IS interpolated — write `{{` for a literal brace, so a template that a later `render(tpl, "name", …)` fills is written `"""hi {{name}}"""`; a bare `{name}` with no such variable is a check error). - Integer division: `7 / 2 = 3.5` (auto-promotes to float when non-exact). Same in `[native]` handlers: `7 / 2 = 3.5`, an exact quotient is an Int (BigInt-exact). Native code is statically typed, so a slot that can only hold an Int (an Int variable, an index) refuses a non-exact quotient with a runtime error — it never truncates. `idiv(a, b)` is the integer quotient on every backend (truncates toward zero, BigInt-exact). ## Cell structure ```soma cell AppName { memory { data: Map [persistent, consistent] // → SQLite cache: Map [ephemeral, local] // → in-memory } state workflow { initial: draft draft -> review review -> approved review -> rejected * -> cancelled } every 30s { // runs periodically } after 5s { // runs once after delay (one-shot timer) } on handler_name(param1: Type, param2: Type) { // handler body return value } on request(method: String, path: String, body: String) { match path { "/" -> html(dashboard()) "/api/data" -> get_data() _ -> response(404, map("error", "not found")) } } } ``` ## Types | Type | Example | Notes | |------|---------|-------| | Int | `42`, `-1`, `0` | arbitrary precision (i64 fast path, BigInt beyond — a slot gives it back as an Int) | | Float | `3.14`, `1.5e3` | 64-bit, scientific notation | | String | `"hello {name}"` | interpolation with `{}` | | Bool | `true`, `false` | | | List | `list(1, 2, 3)` or `[1, 2, 3]` | ordered | | Map | `map("key", val)` | key-value pairs, MUST have even args | | Unit | `()` | null equivalent | | Duration | `5s`, `1min`, `500ms`, `1h` | converts to milliseconds | | Record | `User { name: "Alice", age: 30 }` | typed map with `_type` field | | Any | `on log(x: Any)` | a parameter or slot value of any kind (check it with `type_of(x)`); every parameter needs a type — `x: Any` when it can be anything | Mixed Int/Float comparisons preserve the exact integer value: an Int beyond 2^53 is not rounded to Float before comparison. NaN is unordered (`<`, `>`, `<=`, `>=` and `==` are false; `!=` is true). Sorting places NaN after finite numbers in ascending order. Structural equality applies inside lists/maps; `distinct` and `distinct_by` use it and retain values containing NaN. ## Variables ```soma let x = 42 x = x + 1 // reassignment x += 10 // compound assignment x -= 5 // compound subtraction x *= 2 // compound multiplication x /= 3 // compound division let name = "world" let greeting = "hello {name}" // interpolation ``` ## Control flow ```soma if condition { // ... } else if other { // ... } else { // ... } while condition { if done { break } if skip { continue } } for item in list(1, 2, 3) { print(item) } for i in range(0, 10) { // 0 to 9 } match value { "a" -> expr1 "b" -> { stmts; expr2 } 42 -> expr3 () -> expr4 // match null "x" || "y" -> expr5 // or-pattern name -> use(name) // variable binding (captures value) "/api/" + rest -> api(rest) // string prefix pattern {method: "GET", path} -> get(path) // map destructuring _ -> default_expr // wildcard } // Map destructuring with nested patterns match request { {method: "GET", path: "/"} -> home() {method: "POST", path: "/api/" + resource} -> create(resource) {method: "DELETE", path: "/api/" + resource} -> delete(resource) _ -> response(404, map("error", "not found")) } // Guard clauses match score { n if n >= 90 -> "A" n if n >= 80 -> "B" n if n >= 70 -> "C" _ -> "F" } // Range patterns match http_status { 200..299 -> "success" 300..399 -> "redirect" 400..499 -> "client error" 500..599 -> "server error" _ -> "unknown" } // If/match as expressions let x = if cond { a } else { b } let y = match status { "on" -> true _ -> false } ``` ## Functions (handlers) ```soma on add(a: Int, b: Int) { return a + b } on _private_helper() { // underscore prefix = not exposed as HTTP endpoint return "internal" } // Call: let result = add(1, 2) ``` ## Lambdas ```soma let doubled = list(1, 2, 3) |> map(x => x * 2) let evens = data |> filter(x => x % 2 == 0) let found = data |> find(x => x.id == target) let has = data |> any(x => x.active) let ok = data |> all(x => x.valid) let n = data |> count(x => x.score > 80) // Block lambda let enriched = data |> map(s => { let score = s.x * 2 + s.y s |> with("score", score) }) // Reduce let sum = list(1, 2, 3) |> reduce(0, p => p.acc + p.val) ``` ## Collections ```soma // List let items = list(1, 2, 3) // or: let items = [1, 2, 3] let items = push(items, 4) // append let pairs = enumerate(items) // list of {index, value} maps let first = items[0] // bracket index (or nth(items, 0)) items[0] = 99 // index assignment (in place) let two = with(items, 1, 88) // functional: copy with element 1 replaced let rev = reverse(items) let r = range(0, 10) // [0..9] let down = range(9, -1, -1) // [9,8,..,0] — step may be negative let evens = range(0, 10, 2) // [0,2,4,6,8]; a zero step raises kind range let sorted = sort(items) // ascending let sorted = sort(items, "desc") // descending let n = len(items) // Map let m = map("name", "Alice", "age", 30) let name = m.name // field access let age = m["age"] // bracket index (or m.get("age")) m["email"] = "a@b.com" // index assignment let keys = m.keys() // or keys(m); m.keys without parentheses also answers let vals = m.values() let updated = m |> with("city", "NYC") let smaller = without(m, "age") // copy minus one key let merged = merge(m, map("x", 1)) // right side wins on conflicts // String — bracket index returns the 1-char string at that position let s = "hello" let h = s[0] // "h" ; out-of-bounds raises ``` ## Pipe operators ```soma // Higher-order (with lambdas) data |> map(s => s.name) data |> filter(s => s.score > 50) data |> find(s => s.id == target) data |> any(s => s.active) data |> all(s => s.valid) data |> count(s => s.score > 80) data |> reduce(0, p => p.acc + p.val) // Field-based data |> filter_by("price", ">", 100) // operators: > >= < <= == != data |> sort_by("score", "desc") data |> top(10) data |> bottom(5) data |> group_by("dept") data |> distinct("category") // unique values // Aggregates data |> sum_by("qty") // sum of a field data |> avg_by("qty") // average (Int if whole, else Float) data |> min_by("qty") // row with smallest field value data |> max_by("qty") // row with largest field value data |> count_by("status", "open") // count rows where field == value data |> pluck("name") // list of one field's values data |> select("id", "name") // project each row to listed fields data |> agg("dept", "qty:sum", "qty:avg") // group + aggregate; ops: sum avg min max count // Utilities data |> flatten() data |> reverse() data |> zip(other) list("a", "b", "c") |> join(", ") // "a, b, c" ``` ## String builtins ```soma len("hello") // 5 (chars, not bytes) concat("foo", "bar") // "foobar" contains("hello", "ell") // true starts_with("hello", "he") // true ends_with("hello.txt", ".txt") // true replace("hello", "l", "r") // "herro" split("a,b,c", ",") // ["a", "b", "c"] trim(" hi ") // "hi" uppercase("hello") // "HELLO" lowercase("HELLO") // "hello" substring("hello", 1, 3) // "el" index_of("hello", "ll") // 2 escape_html("x") // "<b>x</b>" to_json(map("a", 1)) // {"a":1} (compact, like JSON.stringify; print(m) shows {"a": 1}) from_json("{\"a\": 1}") // map ``` ## Math builtins ```soma abs(-5) // 5 round(3.7) // 4 floor(3.7) // 3 ceil(3.2) // 4 min(3, 7) // 3 max(3, 7) // 7 idiv(7, 2) // 3 — integer division, truncates toward zero ln(2.718) // ~1.0 — natural log (alias of log) clamp(15, 0, 10) // 10 pow(2, 10) // 1024.0 sqrt(16.0) // 4.0 log(2.718) // ~1.0 exp(1.0) // ~2.718 log10(100.0) // 2.0 random() // float 0.0..1.0 random(100) // int 0..99 random(10, 20) // int 10..19 ``` ## Type conversion ```soma to_int("42") // 42 to_int("abc") // () — returns null, not 0 to_int(3.7) // 3 to_float(42) // 42.0 to_string(42) // "42" type_of(42) // "Int" is_type(rec, "User") // true if rec's _type field is "User" (alias: is_a) ``` ## Error handling ```soma let result = try { risky_operation() } // {value, error, kind, detail} if result.error != () { print("Error: {result.detail}") // detail = the message alone (Error.message); error = "kind: detail" return response(500, map("error", result.detail, "kind", result.kind)) } let value = result.value // Short form: `try { … }?` re-raises the error (same kind; the handler is rolled back) or gives the value let value = try { risky_operation() }? // Equivalent to: if result has error, return error map; else unwrap value // Agent builtins let answer = think("What is 2+2?") // call LLM (configured in soma.toml) let data = think_json("Return as JSON: ...") // LLM returns Map, not String // Bounded think/http — enables compile-time budget proofs let answer = think("prompt", map("max_tokens", 500, "timeout", 10000)) let data = http_get(url, map("max_bytes", 65536, "timeout", 5000)) delegate("Writer", "write", facts, topic) // call another agent's handler remember("key", value) // persistent agent memory let val = recall("key") // recall from agent memory set_budget(5000) // hard token cap let t = tokens_used() // tokens consumed let left = tokens_remaining() // budget minus used (-1 = unlimited) let log = trace() // execution log clear_trace() // reset the execution log clear_context() // reset multi-turn LLM conversation approve("publish article") // human-in-the-loop gate ``` ## Matrices A matrix is a `List>` (a list of equal-length rows) and is a first-class value: bracket-indexed, method-callable, and operated on with arithmetic operators. ```soma let M = [1, 0, 0, 1].reshape(2, 2) // flat list → 2×2 matrix let A = [1, 2, 3, 4, 5, 6].reshape(2, 3) let s = A.shape // [2, 3] (parens-free pseudo-field) let t = A.T // transpose (also A.transpose()) let x = A[1][2] // element access (row 1, col 2) let n = matrix("1 2; 3 4") // MATLAB-style literal // vectorized operators (numpy/MATLAB style): let prod = A * A.T // matmul (both matrices) let scaled = 2 * A // scalar broadcast: * / + - both orders let shifted = A + 10 // matrix + scalar let halves = A / 2 let summed = A + A // elementwise (equal shapes) let v = list(1.0, 2.0, 3.0) let v2 = v * 2 // vector broadcast: * / + - let sq = v * v // vector elementwise: + * / - let vsum = v + v // elementwise add (numeric vectors) let mask = A > 2 // comparison mask → 0/1 matrix let vm = v >= 2.0 // 0.0/1.0 vector (a mask: multiply with it) // list CONCATENATION is explicit: concat(a, b). Non-numeric lists // (strings, records) keep + = concat. let d = det([4, 3, 6, 3].reshape(2, 2)) // -6.0 // any builtin is also a method (UFCS): xs.sum(), xs.sort(), xs.reverse(), // m.det(), m.transpose(), m.matmul(other) ``` Builtins: `reshape(values, r, c)`, `transpose`, `shape`, `matmul`, `det`, `diag_sum` (trace), `identity(n)` / `eye(n)`, `scale(m, k)`, `mat`, `rows`, `cols`, `diag`, `zeros`, `ones`, plus the quant suite (`svd_lowrank`, `regress_sgd`, `clean_covariance`, `var_historical`, …). ## Agent configuration (soma.toml) ```toml [agent] provider = "ollama" # ollama (free, local) model = "gemma3:12b" # Or OpenAI: # provider = "openai" # model = "gpt-4o-mini" # key = "sk-..." # or use SOMA_LLM_KEY env var # Or Anthropic: # provider = "anthropic" # model = "claude-opus-4-6" # key = "sk-ant-..." # or use SOMA_LLM_KEY env var # Or custom endpoint: # url = "https://your-api.com/v1/chat/completions" # model = "your-model" # key = "your-key" ``` Ollama needs no key. OpenAI/Anthropic keys can go in soma.toml or `SOMA_LLM_KEY` env var. Env vars always override soma.toml. ```soma // Postconditions: ensure (checked at point of execution) on withdraw(balance: Int, amount: Int) { let result = balance - amount ensure result >= 0 // fails with error if false return result } // try catches: division by zero, type errors (NOT a stack overflow: runaway recursion fails the handler), invalid // transitions, require/invariant/ensure failures, fail(). An undefined // function is a `soma check` error; an undefined variable inside a `try` // is a check warning and raises kind undefined_variable (catchable). ``` ## Storage ```soma memory { accounts: Map [persistent, consistent] // → SQLite; records as values cache: Map [ephemeral, local] // → in-memory rows: List [persistent] // an append log: push / rows[i] = v / rows.delete(i) balance: Map [persistent] invariant balance >= 0 && balance <= 1000 // checked BEFORE every .set()/.push() commits invariant size <= 10000 // entry-count bound (all slots in this section) } // Invariant bindings: and `value` = the value being written, // `key` = the key, `size` = entry count after the write. An invariant // that names slots guards only those; one using just value/key/size // guards every slot in its section. A violating write raises a // try-catchable error and the slot is UNCHANGED. `soma verify` proves // literal, clamp(), require-narrowed and read-modify-write forms by // induction (docs/guarantees.md); what it cannot prove is runtime-checked // and reported as ⚠ (a failure under --strict). // Invariants may call builtins only. // In handlers: accounts.set("a1", map("owner", "ada", "cents", 100)) let a = accounts.get("a1") // returns () if missing a.cents = a.cents + 5 // edit the copy, write it back: accounts.set("a1", a) // (or accounts["a1"].cents = 105 in one step) accounts.delete("a1") let keys = accounts.keys // list of keys let vals = accounts.values // list of values let n = accounts.len // count // The value type is enforced on every write: Map refuses a // String or 1.5 (kind `type`); an Int written to a Float slot becomes a // Float; Map takes only Pay variants. Ints of any size, // Floats (NaN, inf), (), variants and nested maps/lists round-trip exactly. // No to_json needed for values: a map stored is a map read back. ``` ## State machines ```soma state order { initial: pending pending -> validated { guard { amount > 0 } } validated -> sent sent -> filled sent -> rejected filled -> settled * -> cancelled // from any state } // In handlers: transition("order_id", "validated") // move state let status = get_status("order_id") // current state let valid = valid_transitions("order_id") // available transitions ``` ## Sum types ```soma // `cell type` + `variants` = tagged union. Struct, tuple, or bare variants. cell type PaymentResult { variants { Charged { transaction_id: String, amount: Int } // struct variant Declined(String) // tuple variant Pending // unit variant } } // Construct directly — bare variants take no parentheses: on charge(amount: Int) { if amount > 0 { return Charged { transaction_id: "tx-1", amount: amount } } if amount == 0 { return Pending } return Declined("non-positive amount") } // match is EXHAUSTIVE — missing a variant is a compile error: on describe(r: Map) { return match r { Charged { transaction_id, amount } -> "ok {transaction_id} ${amount}" Declined(reason) -> "rejected: {reason}" Pending -> "..." } } ``` ```soma // Typed state machine: states must be variants of the sum type, // and transition() takes a variant (typo = compile error): cell type TodoStatus { variants { Pending InProgress Done Cancelled } } cell TodoList { state todo: TodoStatus { initial: Pending Pending -> InProgress InProgress -> Done * -> Cancelled } on start(id: String) { transition(id, InProgress) // variant, not string return get_status(id) // "InProgress" } } ``` ## HTTP server ```soma // Run: soma serve app.cell // HTTP on :8080 (127.0.0.1 unless --host); WS on :8081 only with `on ws`; // signal bus on :8082 only with emit / scale / --join // Dashboard: http://localhost:8080/__soma/ (state machines, budget, verification) on request(method: String, path: String, body: String) { match path { "/" -> html(render_page()) "/api/data" -> get_all_data() _ -> response(404, map("error", "not found")) } } // Response types: html("

Hello

") // text/html map("key", "value") // application/json (auto) response(201, map("id", 1)) // custom status code redirect("/other") // 302 redirect sse("trade", "update") // SSE event stream ``` ## HTTP client, WebSocket, signal bus ```soma let resp = http_get(url, map("timeout", 2000)) // GET; JSON bodies auto-parse to Map/List let resp = http_post(url, body, map("timeout", 2000)) // a Map/List body goes as JSON // also http_put / http_patch (url, body, opts?) and http_delete(url, opts?) // opts: timeout (ms, default 30000), max_bytes, headers: map("Authorization", "Bearer …") let ws = ws_connect("ws://host:9001") // open WebSocket (send-only) → {status, url} ws_send(message) // send text on the open WebSocket subscribe("ws://host:9001/stream") // read-only WS: incoming {"event", "data"} → on event(data) when the program emits that event or soma.toml [bus] accept lists it; other text → on ws(msg) link("host:8082") // TCP signal-bus link: emits reach peer, peer EVENTs → handlers publish("stream-name", data) // push to SSE subscribers on a runtime-chosen stream ``` The http builtins never raise. A 2xx answer is its body; anything else is `{error, kind, status, body}`: kind `http_status` (with the upstream status and its body, parsed when JSON), `timeout`, `refused` or `network`. Branch on `resp.kind` / `resp.status`, not on the text. Under `soma serve` the call holds the handler lock for its whole duration (handlers are serialized): keep timeouts short. In tests, `mock http_post map(...)` scripts the next call, `mock http_get error "timeout: slow"` a failure (`status_404: …` gives status 404); an unscripted real call prints a note. A List scripts several calls in order (`mock http_get [1, 2]` answers 1 then 2): to answer ONE call with a JSON array, nest it (`mock http_get [[1, 2]]`). ## Events: `emit` In one process, `emit trade(data)` calls every cell that declares `on trade(data: Map)`, synchronously, inside the emitter's transaction (a listener that raises fails the emitter; the emitter's rollback undoes the listeners' writes). `soma check` warns when no cell handles the event. Across processes the same statement goes over the signal bus (`[peers]`), where it is fire-and-forget: ```soma // soma.toml // [peers] // exchange = "localhost:8082" // Send (goes to all peers): signal order(map("ticker", "BTC", "qty", 1)) // or: emit trade(fill_data) // Receive (auto-dispatched from bus): on trade(data: Map) { record_fill(data) } ``` ## File I/O ```soma let content = read_file("data.txt") write_file("output.txt", content) let rows = read_csv("data.csv") // list of maps, auto-typed write_csv("out.csv", rows) // list of maps → CSV (headers from first row) let tpl = load("page.html") // read file (aliases: include, load_template) let s = load("page.html", "k", v) // read + replace {k} placeholders with v let files = read_files("dir", 100) // first N files → list of {path, content} let files = par_read_files("dir", 100) // parallel version (threaded) let counts = word_count(text) // map of word → count (lowercased; also takes a list) let counts = par_word_count(files) // parallel version (threaded) ``` ## Templates ```soma let s = render(tpl, "name", "Ada") // replace {name} with "Ada" in template string let s = render_each(rows, row_tpl) // render template once per map in list ``` ## Time ```soma let ts = now() // unix timestamp (seconds) let ms = now_ms() // milliseconds let today = today() // "2026-03-29" let formatted = format_date(ts) // "2026-03-29" sleep(100) // pause 100 milliseconds ``` ## Verification ```soma // soma.toml // [verify] // cells = ["Order"] # optional: which machines these apply to // deadlock_free = true // eventually = ["settled", "cancelled"] // never = ["invalid"] // always = ["open", "closed"] # the machine is only ever in one of these // // [verify.after.sent] // eventually = ["filled", "rejected"] // [verify.before.filled] // requires = ["sent"] # requires_all = [...] for several // (an unknown key is an error; --strict turns every ⚠ into a failure) // Run: soma verify app.cell ``` ## Face contracts (compile-time checked) ```soma cell API { face { signal create(name: String) -> Map // MUST have matching handler signal delete(id: String) // MUST have matching handler promise all_persistent // structural check promise "human-readable description" // a note (not checked) } // Missing handler for 'delete' → compile error } ``` ## Tests ```soma cell test MathTests { rules { assert 1 + 1 == 2 assert len("hello") == 5 assert round(3.7) == 4 } } // Run: soma test file.cell ``` ## Multi-file projects ``` project/ app.cell // main file lib/ helpers.cell // use lib::helpers scoring.cell // use lib::scoring soma.toml // config + peers + verify ``` ```soma // app.cell use lib::helpers use lib::scoring cell App { on run() { let result = helper_function() // from helpers.cell } } ``` ## Common patterns ```soma // CRUD web app cell App { memory { items: Map [persistent, consistent] } on request(method: String, path: String, body: String) { if method == "POST" && path == "/api/items" { let data = from_json(body) let id = to_string(next_id()) items.set(id, to_json(data |> with("id", id))) return data |> with("id", id) } match path { "/api/items" -> items.values |> map(s => from_json(s)) _ -> response(404, map("error", "not found")) } } } // Data pipeline cell Pipeline { on run() { let data = read_csv("input.csv") let result = data |> filter(s => s.score > 50) |> sort_by("score", "desc") |> top(10) print(result) } } // Real-time with state machine cell OrderSystem { memory { orders: Map [persistent, consistent] } state order { initial: pending pending -> validated validated -> shipped * -> cancelled } every 30s { check_stale_orders() } on create(data: Map) { let id = to_string(next_id()) orders.set(id, to_json(data |> with("id", id) |> with("status", "pending"))) return map("id", id) } on advance(id: String, target: String) { return transition(id, target) } } ``` ## Linear algebra & risk (`linalg`) Quantum-inspired sublinear linear algebra (Tang et al.) plus Bouchaud-Potters covariance cleaning and risk metrics. Every builtin takes an options Map carrying explicit sample / iteration / dimension bounds — `soma check` reads them and proves closed-form runtime. ### Matrix constructors ```soma // Most readable for hand-written matrices let A = matrix("1 2 3; 4 5 6") // 2×3, MATLAB-style; ';' rows, ws/',' entries // When you have row vectors in hand let B = rows(list(1.0, 2.0), list(3.0, 4.0)) // Column-major constructor (transposes) let C = cols(list(1.0, 2.0), list(3.0, 4.0)) // Reshape a flat list let D = mat(2, 3, list(1.0, 2.0, 3.0, 4.0, 5.0, 6.0)) // Standard constructors let I = eye(3) // 3×3 identity let Z = zeros(2, 4) // 2×4 zeros let O = ones(3, 3) // 3×3 ones let G = diag(list(1.0, 2.0, 3.0)) // 3×3 diagonal ``` ### Sublinear sampling (Tang) Build the BST-backed `Sampled` handle once; pay O(log n) per sample thereafter. ```soma let A = to_sampled(dense, map("max_rows", 1000, "max_cols", 50)) // A is a handle: { __sampled__, rows, cols, fro_norm, kind: "sampled" } let s = sample_row(A) // O(log m) ℓ²-norm row sample let isr = importance_sample_rows(A, map("samples", 50)) let svd = svd_lowrank(A, map( "row_samples", 100, "col_samples", 50, "rank", 10, "max_dim", 1000 )) let fit = regress_sgd(A, b, map( "eps", 0.01, "lambda", 0.1, "max_iter", 10000, "max_dim", 1000 )) drop_sampled(A) // free the registry entry ``` All four algorithms accept either a sampled handle or a dense `List>` transparently. ### Covariance cleaning (Bouchaud-Potters) Replace the noise-bulk eigenvalues of a sample covariance with their RMT-shrunk versions. Drastically improves out-of-sample portfolio optimization in any regime where N (assets) is comparable to T (observations). ```soma let cov = clean_covariance(returns, map( "method", "rie", // "rie" | "clip" | "raw" "eta", 0.1, // Stieltjes regularizer "center", true, "max_assets", 500, "max_obs", 1000 )) // cov.matrix is the cleaned N×N matrix; cov.eigenvalues_clean is the spectrum. ``` ### Market impact (Bouchaud square-root law) ```soma let imp = impact_sqrt(qty, daily_volume, sigma, map("Y", 1.0)) // imp.bps is the expected slippage in basis points. ensure imp.bps <= max_slippage_bps // compile-time pattern, runtime check ``` ### Risk metrics ```soma let var95 = var_historical(returns, map("alpha", 0.95, "max_obs", 250)) let es95 = expected_shortfall_historical(returns, map("alpha", 0.95)) let varg = var_gaussian(returns, map("alpha", 0.95)) // for comparison let q = quantile(returns, 0.05) // empirical quantile ``` Historical estimators make no distributional assumption — the Bouchaud-Potters baseline for fat-tailed markets. ### Verified pre-trade pattern ```soma on submit(qty: Float, vol: Float, sigma: Float, hist: List) { let imp = impact_sqrt(qty, vol, sigma, map()) ensure imp.bps <= 30.0 // 30bps slippage cap let var = var_historical(hist, map("alpha", 0.99, "max_obs", 250)) ensure var <= 0.05 // 99%-VaR cap of 5% emit place_order(qty) } ``` Wrap the `submit` call in `try { ... }` from the caller to catch the ensure-failure as a structured error and reject the order. See `examples/risk_check.cell` for a complete demo with budget proof. --- # PART 3 — Verified wrong→right pairs (AGENT_GOTCHAS.md) # Soma for agents: verified wrong → right The mistakes an LLM makes writing Soma, each with the **actual compiler error** and the fix. Every pair below is verified against the current `soma` binary. This is the self-correction corpus: when `soma check` / `soma run` emits one of these errors, apply the paired fix. A rule of thumb: **`soma check` catches most of these before you run.** Write the file, run `soma check app.cell`, fix what it reports, repeat. --- ## 1. Quotes inside `{...}` interpolation ```soma // WRONG — an unescaped `"` ends the string: `{len(` is left dangling return "len: {len("hi")}" ``` ```soma // RIGHT — escape it (`"len: {len(\"hi\")}"` prints len: 2), or, clearer, // bind the value first and interpolate the variable let n = len("hi") return "len: {n}" ``` ## 2. `match` arms use `->`, not `=>` ```soma return match x { 1 => "a" _ => "b" } // error: match arms use '->', not '=>' ``` ```soma return match x { 1 -> "a" _ -> "b" } // `_` is the catch-all arm ``` `=>` is **lambda** syntax (`p => p + 1`). `->` is match arms and signal return types. Don't cross them. ## 3. Handlers don't declare return types ```soma on add(a: Int, b: Int) -> Int { return a + b } // error: handlers do not declare return types — put '-> Int' on the // signal declaration inside face { } ``` ```soma face { signal add(a: Int, b: Int) -> Int } on add(a: Int, b: Int) { return a + b } ``` ## 4. Adjacent string literals do NOT concatenate ```soma return "hello " "world" // error: in G.hello: a string literal follows `return` and is never // evaluated — adjacent string literals do not concatenate ``` ```soma return "hello world" // one literal let name = "world" return "hello {name}" // or interpolate ``` (`soma check` also warns on any other unreachable statement after `return` / `break` / `continue`.) ## 5. `==` IS structural on lists, maps and variants ```soma [1, 2] == [1, 2] // true map("US", 1, "EU", 2) == map("EU", 2, "US", 1) // true — key order is irrelevant [] == [] // true ``` `<`, `>` on lists or maps is an error (compare a field or `len()`), and values of different kinds are an error to compare (`1 == "1"`), not `false`. ## 6. Float equality needs a tolerance ```soma return 0.1 + 0.2 == 0.3 // false — floating point ``` ```soma return abs((0.1 + 0.2) - 0.3) < 0.0001 ``` ## 7. `transition()` returns a map, not the target string ```soma on advance(id: String) { return transition(id, "next") // returns {id, from, to}, not "next" } ``` ```soma on advance(id: String) { transition(id, "next") return get_status(id) // the new state as a string } ``` Guard fallible transitions with `try`: ```soma let r = try { transition(id, "next") } if r.error != () { return map("error", r.error) } ``` ## 8. `is_a` does not recognize sum-type VARIANTS — match them ```soma let b = Box { w: 3 } // Box is a `variants` constructor return is_a(b, "Box") // false — variants aren't tagged records ``` ```soma // extract the kind with an exhaustive match handler on kind(s: Map) { return match s { Box { w } -> "Box" // ... every variant } } ``` (`is_a` / `is_type` DO work on record literals: `is_a(Game { x: 1 }, "Game")` is `true`.) ## 9. There is no `cell type X { fields { ... } }` Records are plain map-shaped values. Construct them with a literal: ```soma let g = Game { bet: 10, pot: 0 } // a field-accessible value (a Map) g.bet = 20 // mutate fields in place let b = g.bet return is_a(g, "Game") // true — record literals carry _type ``` Use `cell type X { variants { ... } }` only for *sum types* (tagged unions). ## 10. `soma serve` routes only the cell that owns `request` ```soma // other cells' signals are NOT auto-routed as HTTP endpoints ``` ```soma // put every routable signal on the request-owning cell, delegating // to domain cells: cell Api { face { signal request(...) -> String signal place(...) -> Map } on place(...) { return place_order(...) } // delegate to Orders on request(method: String, path: String, body: String) { ... } } ``` --- ## These USED to be limitations and now WORK — use them freely Older Soma code worked around these; the current language supports them directly. Prefer the direct form. ```soma // bracket indexing (read + write) on lists, maps, strings let x = xs[2] xs[2] = 99 let v = m["key"] m["key"] = 1 let c = s[0] // negative literals let n = -1 // not `0 - 1` // descending / stepped ranges for r in range(10, 0, -1) { } // not build-then-reverse // numeric reductions over a list sum(xs) product(xs) avg(xs) min(xs) max(xs) // UFCS — any builtin is a method xs.sum() xs.sort() xs.reverse() m.det() m.transpose() // nested record/list mutation g.board[0] = 99 g.meta.turn = 5 xs[i][j] = v // matrices are first-class, with vectorized (numpy-style) operators let M = [1,0,0,1].reshape(2,2) let P = A * B let t = M.T det(M) A + 10 A / 2 1 - A // scalar broadcast on matrices v * 2 v - 1 v * v v + v // vector broadcast + elementwise (+ * / -) A > 2 v >= 2.0 // comparison masks (0/1) // list CONCATENATION is concat(a, b); non-numeric lists keep + = concat // `with` is functional copy-update for maps AND lists let m2 = with(m, "k", 9) let l2 = with(xs, 0, 9) ``` --- ## The agent loop, in commands ``` soma check app.cell # contracts, interpolation, dispatch — fix these first soma verify app.cell # PROVE state machines + memory invariants soma test app.cell # run `cell test` assertions (assert / assert_fails) soma run app.cell sig a # execute a handler soma serve app.cell -p 8080 soma describe app.cell --faces # token-cheap contract summary of every cell soma describe --builtins --json # the exact builtin signatures (never guess) ``` When unsure of a builtin's signature, run `soma describe --builtins` — do not guess. When unsure of a cell's API, run `soma describe --faces`. --- ## More verified footguns (found generating 168 programs) ## 11. Your handler shadows a builtin of the same name — when the argument count matches `f(args)` calls the program's handler `f` if one takes that many arguments, the builtin `f` otherwise. User code shadows the library, as everywhere else. ```soma on merge(a: Int, b: Int) { return a + b + 1000 } on use_it() { return merge(1, 2) } // 1003 — your handler on list() { return list(1, 2) } // [1, 2] — 2 ≠ 0 arguments: the builtin ``` `soma check` warns on the two confusing cases: ```soma on merge(a: Int, b: Int, c: Int) { … } on use_it() { return merge(m1, m2) } // warning: call to 'merge' with 2 argument(s) … resolves to the BUILTIN merge(): // the handler G.merge takes [3] on list() { let items = list() … } // warning: inside G.list, `list(…)` with 0 argument(s) calls the handler ITSELF // (recursion), not the builtin list(). For an empty list write [] ``` Method calls (`xs.count(p)`) always go to builtins. ## 12. `assert_fails` needs an expression that RAISES, not a falsy bool ```soma assert_fails 1 == 2 // FAILS the test: 1==2 is just `false`, no error ``` ```soma assert_fails xs[99] // passes: out-of-bounds RAISES assert_fails transition(id, "illegal") matching "invalid_transition" // passes for THAT reason // (with several state machines, call a handler of the owning cell instead) assert !(1 == 2) // for a falsy predicate, use plain assert + ! ``` ## 13. Slot methods work on declared `memory` slots, not local maps ```soma let seen = map() seen.set("k", 1) // error: `.set()` is a memory-slot method and 'seen' is a local — a local map is written with brackets: `seen[k] = v` ``` ```soma let seen = map() seen["k"] = 1 // local maps use bracket indexing let v = seen["k"] ?? 0 ``` `.get`/`.set`/`.has`/`.delete`/`.keys` are for `memory { slot: ... }` slots. ## 14. `on` is a reserved keyword It can't be a parameter name or a map field read as `.on`. Use `enabled`, `active`, etc. ## 15. No semicolons; statements are newline-separated ```soma { a = 1; b = 2 } // error: unexpected character ';' — Soma has no semicolons, one statement per line ``` ```soma { a = 1 b = 2 } ``` ## 16. `given` is reserved (like `on`) It's a face-declaration keyword — can't be a state name, param, or identifier. `error: expected identifier, found Given`. Use `granted`, `input`, etc. ## 17. What a test cell's `rules { }` accepts `assert`, `assert_fails` (optionally `… matching "text"`), `let name = expr` (a fixture for the rules below), `mock think "reply"` / `mock think ["a", "b"]` / `mock think error "timeout"`, `mock http_get map(...)` / `mock Cell.handler v`, and `property`. No bare statements: put logic in a handler and call it. A List given to any `mock` is a QUEUE — one reply per call, in order — so a handler or `http_get` that answers a JSON array is mocked with a nested list: ```soma mock http_get [1, 2] // two calls: the first gets 1, the second 2 mock http_get [[1, 2]] // one call gets the list [1, 2] ``` `soma test` notes a scripted reply that no call consumed. ## 18. An invariant between two slots defaults both sides An invariant is checked on every write to either slot it names: the written slot is its new value, the other one is read at the SAME key — and a key it has no entry for reads as `()`, which no comparison accepts. ```soma memory { a: Map b: Map invariant a + b <= 100 } // error: memory invariant between slots (a, b) — the other slot is read at the // SAME key, and a key it has no entry for reads as (), which no // comparison accepts: default the sides (`(reserved ?? 0) <= (stock ?? 0)`) ``` ```soma memory { stock: Map reserved: Map invariant (reserved ?? 0) <= (stock ?? 0) // runtime-checked on both slots } ``` A rule between two slots stays runtime-checked (`verify` cannot prove it by induction). `size` invariants are enforced on `delete` too: `invariant size >= 1` rejects removing the last entry. ## 19. `7 / 2 = 3.5` everywhere — say `idiv` when you mean the integer quotient `/` on two Ints is 3.5 (an Int only when exact) in the interpreter AND in `[native]` handlers. Native code is statically typed, so where the quotient must be an Int it is checked instead of truncated: ```soma on mid(lo: Int, hi: Int) [native] { let m = lo m = (lo + hi) / 2 // m is an Int slot return m } // mid(1, 2) → error: Int / Int is not exact here, and this spot can only // hold an Int (7 / 2 is 3.5) — write idiv(a, b) ... ``` ```soma on mid(lo: Int, hi: Int) [native] { return idiv(lo + hi, 2) } // 1, everywhere ``` `soma check` warns on Int / Int in native handlers; `soma fix f.cell --native-idiv` rewrites them (for code written when native `/` truncated). A `[native]` division by zero is an ordinary, `try`-catchable runtime error. ## 20. A cost bound is only *proven* when every `think()` count is known `think()` reached through a loop over a list, a lambda (`map(xs, x => think(..))`) or a recursive helper makes the bound unprovable — and an unprovable declared bound is a `soma check` **error** (the message reads `cost: 'tokens' bound is advisory — …`, exit 1), not a note: a bound nobody can prove is a lie in the program's own words. Give the loop a literal `range(0, N)` or `[loop_bound(N)]` to get `bound proven` back, or remove the `cost` block. Calls to sibling handlers are composed: `for i in range(0, 3) { helper() }` costs 3 × helper. ## 21. Habits from Python / TypeScript that `soma check` now redirects ```soma if x == null { } // error: 'null' does not exist — Soma's null is `()`; also `x ?? default` xs.includes(v) // error: no method 'includes' — in Soma: contains(xs, v) rows.push(o) // warning: result discarded — push returns a NEW list: rows = push(rows, o) [a[0]] + rest // warning: `+` ADDS numeric lists element-wise — concat(a, b) / push(xs, x) let t = lefft + 1 // error: undefined variable 'lefft' (did you mean 'left'?) ``` Everyday collection builtins: `contains(list|map|string, x)`, `slice(xs, start, end?)` (negative indexes count from the end), `keys(m)` / `values(m)` / `entries(m)`, `sort_by(rows, "field")` or `sort_by(rows, r => [0 - r.total, r.name], "desc"?)` (stable; a list key sorts on several keys), `round(x, digits)`. ## 22. Transition guards see the calling handler's locals ```soma state expense { initial: approved approved -> paid { guard { amount < 10000 } } } on pay(id: String) { let amount = amounts.get(id) ?? 0 // the guard reads THIS `amount` transition(id, "paid") // raises "guard failed…" when false } ``` A guard sees: the locals of the handler calling `transition()`, the cell's memory slots, `_id`, `_from`, `_to`. `soma check` rejects a guard that reads a name its calling handler never binds. Guards are enforced at runtime; the model checker keeps the edge (an over-approximation, so safety results hold). ## 23. `soma.toml` is validated A `soma.toml` that does not parse — or has an unknown key under `[verify]` — is an error for every command (it used to be ignored silently, so `[verify]` properties were never checked). `[package]` is optional. ## 24. Handlers are atomic; errors have kinds A handler that raises leaves nothing behind: its writes and transitions are rolled back (a failing `try { }` block too, to where it began). Do not write compensation code. Raise with `fail("kind", "detail")` or `require cond else Tag`; catch with `let r = try { … }` and branch on `r.kind` (`"not_found"`, `"invalid_transition"`, `"guard_failed"`, `"invariant"`, …); `fail(r)` re-raises. ## 25. `* -> failed` leaves EVERY state, final ones included ```soma state s { initial: a a -> paid * -> failed } // paid -> failed exists: paid is not final state s { initial: a a -> paid * -> failed except [paid] } // paid stays final ``` `soma verify` warns about the first form and prints the second. ## 26. A route in `request` and a handler of the same name `soma serve` exposes every public handler at `//`. With `on hold(id, qty)` and a route `"/hold/" + id`, the explicit route wins — but other `/hold/…` shapes still reach the handler. Prefix internal handlers with `_`. `soma check` warns. --- # PART 4 — Every builtin (SOMA_BUILTINS.md, generated from the compiler) # Soma Builtins > GENERATED by `soma docs builtins` — do not edit by hand. > The source of truth is `compiler/src/interpreter/builtins/registry.rs`. 248 builtins. ✗ marks the nondeterministic set (random_token, random, now, now_ms, today) — calls to these are tracked by `soma replay` as potential sources of replay divergence. `deterministic` is membership in that replay set, not a purity claim: think/http_*/read_*/next_id are NOT in the log: replay calls them again (think needs a key or SOMA_LLM_MOCK, files are re-read) — pass their results in as handler arguments to replay exactly. The `native` section is usable inside `[native]` handlers only. ## string | Builtin | Signature | Description | |---|---|---| | `concat` | `concat(a, b) -> String \| concat(a: List, b: List) -> List` | Concatenate strings, or join two lists (numeric list `+` is elementwise, so this is THE list concat). | | `pad_left` | `pad_left(s, width: Int, fill?: String) -> String` | Left-pad to `width` characters: pad_left("7", 4, "0") = "0007". Default fill is a space. | | `pad_right` | `pad_right(s, width: Int, fill?: String) -> String` | Right-pad to `width` characters. | | `split` | `split(s: String, delim: String) -> List` | Split a string on a delimiter into a list of substrings. | | `replace` | `replace(s: String, old: String, new: String) -> String` | Replace every occurrence of `old` with `new`. | | `contains` | `contains(haystack: String, needle: String) -> Bool \| contains(list: List, x) -> Bool \| contains(m: Map, key) -> Bool` | Substring test; list membership (structural equality); map key membership. | | `starts_with` | `starts_with(s: String, prefix: String) -> Bool` | True if `s` begins with `prefix`. | | `ends_with` | `ends_with(s: String, suffix: String) -> Bool` | True if `s` ends with `suffix`. | | `lowercase` | `lowercase(s: String) -> String` | Lowercase the string (non-strings are stringified first). | | `uppercase` | `uppercase(s: String) -> String` | Uppercase the string (non-strings are stringified first). | | `trim` | `trim(s: String) -> String \| trim(s: String, chars: String) -> String` | Strip leading and trailing whitespace — or any of the characters in `chars` (Go's strings.Trim(s, cutset)). | | `sha256` | `sha256(s: String) -> String` | SHA-256 of the text, as 64 hex characters (store `sha256(salt + password)` or better an hmac_sha256 with a server secret, never the password). | | `hmac_sha256` | `hmac_sha256(key: String, message: String) -> String` | HMAC-SHA-256 as hex: sign a session cookie or a webhook payload with a server secret. | | `random_token` ✗ | `random_token(bytes?: Int) -> String` | Cryptographically secure random bytes from the OS, as hex (default 32 bytes = 64 characters): session tokens, API keys, salts. random() is NOT for secrets. | | `secure_eq` | `secure_eq(a: String, b: String) -> Bool` | Constant-time equality for secrets (tokens, signatures): `==` returns early and leaks a prefix by timing. | | `format` | `format(fmt: String, args...) -> String` | printf subset: %d %s %f %.2f %8.2f %e %.3e %3d %-8s %05d %% — %e is C-style scientific (6.022000e+23); widths, precision (rounded half away from zero on the decimal text), left-align with '-', zero-pad with '0'. | | `fields` | `fields(s: String) -> List` | Split on any run of whitespace, no empty pieces (Go's strings.Fields; split(s, " ") keeps empties). | | `index_of` | `index_of(s: String, sub: String) -> Int \| index_of(xs: List, x) -> Int` | Character index of the first occurrence of `sub` in a String, or the position of the first element equal to `x` in a List; -1 if absent. | | `substring` | `substring(s: String, start: Int, end: Int) -> String` | Character-based slice [start, end) — both indexes are clamped to 0..len(s), end exclusive. A negative end gives an empty string; use slice() for indexes relative to the end. | | `escape_html` | `escape_html(s: String) -> String` | Escape &, <, >, double and single quotes for safe HTML embedding. | | `str_len` | `str_len(s: String) -> Int` | Byte length of a string (cf. len(), which counts characters). | | `str_at` | `str_at(s: String, i: Int) -> Int` | Byte value at index `i`; errors if out of range. | | `str_eq` | `str_eq(a: String, b: String) -> Bool` | Exact string equality (fast path for [native] code). | | `chr` | `chr(n: Int) -> String` | The character with code point n: chr(65) == "A". Raises kind range outside 0..0x10FFFF or for surrogate code points. | | `ord` | `ord(s: String) -> Int` | Code point of the first character: ord("A") == 65. | | `regex_count` | `regex_count(text: String, pattern: String) -> Int` | Number of non-overlapping matches (Rust regex syntax). Same in [native] (pattern must be a literal there). | | `regex_match` | `regex_match(text: String, pattern: String) -> Int` | 1 when the pattern matches anywhere in text, else 0. | | `regex_replace` | `regex_replace(text: String, pattern: String, replacement: String) -> String` | Replace every match; $1 refers to the first capture group. | ## types | Builtin | Signature | Description | |---|---|---| | `len` | `len(x: String\|List\|Map) -> Int` | Characters of a string, elements of a list, or entries of a map. | | `to_string` | `to_string(x) -> String` | Render any value with its display formatting. | | `to_int` | `to_int(x) -> Int` | Convert to Int (floats truncate, strings parse, BigInt-exact); returns () on failure. | | `to_float` | `to_float(x) -> Float` | Convert to Float; returns () if a string fails to parse. | | `to_json` | `to_json(x) -> String` | Serialize a value as JSON (strings escaped, NaN/inf become null). | | `from_json` | `from_json(s: String) -> Any` | Parse a JSON string into a Map/List/scalar; maps and lists pass through. Invalid JSON RAISES (kind "json") — wrap LLM output in try { from_json(s) }. | | `type_of` | `type_of(x) -> String` | Type name: "Int" (any size), "Float", "String", "Bool", "List", "Map", "Function", "Variant", or "Unit". | | `is_type` | `is_type(value: Map, type_name: String) -> Bool` | True if a record's `_type` field equals `type_name`. | | `is_a` | `is_a(value: Map, type_name: String) -> Bool` | Alias of is_type. | | `fail` | `fail(kind: String, detail?) -> never \| fail(r: TryResult) -> never` | Raise a domain error. `try { f() }` yields {value, error, kind, detail}: branch on r.kind ("not_found", "invalid_transition", "guard_failed", "invariant", a `require … else Tag` tag, …); fail(r) re-raises a caught error unchanged. | ## math | Builtin | Signature | Description | |---|---|---| | `to_fixed` | `to_fixed(x: Float, digits: Int) -> String` | x with exactly `digits` decimals ("%.2f"), rounded half away from zero on the decimal text: to_fixed(1.005, 2) = "1.01". | | `div_round` | `div_round(n: Int, d: Int) -> Int` | Exact integer division rounded to the nearest, half away from zero (BigDecimal HALF_UP): div_round(10125 * 600, 120000) == 51, div_round(-7, 2) == -4. Money in cents stays exact. | | `floor_div` | `floor_div(a: Int, b: Int) -> Int` | Division rounded toward -∞ (Ruby/Python `//`): floor_div(-150, 100) = -2. `idiv` truncates toward zero; `/` is exact. | | `mod` | `mod(a: Int, b: Int) -> Int` | Modulo with the DIVISOR's sign (Ruby/Python `%`): mod(-150, 100) = 50. The `%` operator keeps the dividend's sign (C/Rust): -150 % 100 = -50. | | `divmod` | `divmod(a: Int, b: Int) -> [q, r]` | [floor_div(a, b), mod(a, b)] — q * b + r == a with 0 <= r < \|b\|. | | `abs` | `abs(x: Int\|Float) -> Int\|Float` | Absolute value, arbitrary precision (abs(-9223372036854775808) is 9223372036854775808). | | `round` | `round(x: Float) -> Int \| round(x: Float, digits: Int) -> Float` | Round half away from zero to the nearest integer, or keep `digits` decimals: round(2.345, 2) = 2.35. | | `floor` | `floor(x: Float) -> Int` | Largest integer <= x. | | `ceil` | `ceil(x: Float) -> Int` | Smallest integer >= x. | | `sqrt` | `sqrt(x: Int\|Float) -> Float` | Square root. | | `sin` | `sin(x: Int\|Float) -> Float` | Sine (radians). Also cos, tan, atan, atan2(y, x). | | `cos` | `cos(x: Int\|Float) -> Float` | Cosine (radians). | | `tan` | `tan(x: Int\|Float) -> Float` | Tangent (radians). | | `atan` | `atan(x: Int\|Float) -> Float` | Arc tangent. | | `atan2` | `atan2(y: Float, x: Float) -> Float` | Arc tangent of y/x, quadrant-aware. | | `asin` | `asin(x: Float) -> Float` | Arc sine (x in [-1, 1], else kind range). | | `acos` | `acos(x: Float) -> Float` | Arc cosine (x in [-1, 1], else kind range). | | `pi` | `pi() -> Float` | The constant π. | | `log` | `log(x: Int\|Float) -> Float` | Natural logarithm. | | `ln` | `ln(x: Int\|Float) -> Float` | Alias of log (natural logarithm). | | `exp` | `exp(x: Int\|Float) -> Float` | e raised to the power x. | | `log10` | `log10(x: Int\|Float) -> Float` | Base-10 logarithm. | | `pow` | `pow(base: Int\|Float, exp: Int\|Float) -> Float` | base raised to exp (always a Float; `ipow` for an exact Int power). | | `ipow` | `ipow(base: Int, exp: Int) -> Int` | Exact Int power (exp ≥ 0; kind range past the Int size limit). `to_int(pow(3, 40))` is off by 33, `ipow(3, 40)` is exact. | | `min` | `min(a, b) -> Int\|Float \| min(list: List) -> Int\|Float` | Smaller of two numbers, or the minimum of a list (Float if any element is). An empty list gives () (no minimum exists). | | `max` | `max(a, b) -> Int\|Float \| max(list: List) -> Int\|Float` | Larger of two numbers, or the maximum of a list (Float if any element is). An empty list gives () (no maximum exists). | | `sum` | `sum(list: List) -> Int\|Float` | Sum of a list of numbers (Int-exact unless any element is a Float); 0 when empty. Floats are added left to right without compensation (NumPy's pairwise / Python's fsum can differ in the last bits). | | `product` | `product(list: List) -> Int\|Float` | Product of a list of numbers; 1 when empty. | | `avg` | `avg(list: List) -> Int\|Float` | Mean of a list of numbers: finite operands are averaged exactly before Float rounding, avoiding intermediate overflow and cancellation. An exact mean of Ints stays an Int; () when empty. | | `parse_int` | `parse_int(s: String, base: Int?) -> Int \| ()` | Strict integer parse: () unless the WHOLE string is an integer ("1.5", "12abc", "" → ()). parse_int("ff", 16) = 255 (base 2..36, no 0x prefix). to_int() is lenient and truncates. | | `parse_float` | `parse_float(s: String) -> Float \| ()` | Strict float parse: () unless the whole string is a finite number. | | `idiv` | `idiv(a: Int, b: Int) -> Int` | Integer division truncating toward zero; errors on division by zero. | | `clamp` | `clamp(v, lo, hi) -> Int\|Float` | Constrain numeric v to [lo, hi] using exact Int/Float comparisons; return the selected operand with its type. Reject nonnumeric operands, reversed bounds and NaN bounds; a NaN value stays NaN. | | `random` ✗ | `random() -> Float \| random(max: Int) -> Int \| random(min: Int, max: Int) -> Int` | Time-seeded PRNG: float in [0,1), or int in [0,max) / [min,max). There is no seed: for reproducible runs write your own generator (an LCG over Ints), and for secrets use random_token(). | | `gcd` | `gcd(a: Int, b: Int) -> Int` | Greatest common divisor (Euclid, absolute values). | | `sqrt_int` | `sqrt_int(n: Int) -> Int` | Integer square root; errors on negative input. | | `pow_mod` | `pow_mod(base: Int, exp: Int, m: Int) -> Int` | Modular exponentiation base^exp mod m; errors if m is zero. | | `band` | `band(a: Int, b: Int) -> Int` | Bitwise AND. | | `bor` | `bor(a: Int, b: Int) -> Int` | Bitwise OR. | | `bxor` | `bxor(a: Int, b: Int) -> Int` | Bitwise XOR. | | `bnot` | `bnot(a: Int) -> Int` | Bitwise NOT. | | `shl` | `shl(a: Int, n: Int) -> Int` | Exact left shift (a * 2^n), arbitrary precision like every Int op. For a 64-bit wrapping shift (xorshift), mask: band(shl(x, 13), M) with M = shl(1, 64) - 1 bound once (a literal beyond 64 bits is not allowed in [native]); values past 2^63 run [native] code in BigInt mode — prefer 32-bit xorshift masks for speed. | | `shr` | `shr(a: Int, n: Int) -> Int` | Arithmetic shift right by a nonnegative Int count; arbitrarily large counts give 0 (nonnegative a) or -1 (negative a). | | `bit_test` | `bit_test(a: Int, i: Int) -> Int` | 1 if bit i of a is set, else 0. | | `bit_set` | `bit_set(a: Int, i: Int) -> Int` | a with bit i set. | | `bit_clr` | `bit_clr(a: Int, i: Int) -> Int` | a with bit i cleared. | | `bit_next` | `bit_next(a: Int, i: Int) -> Int` | Index of the lowest set bit at or above i, or -1 if none. | | `bit_len` | `bit_len(a: Int) -> Int` | Exact number of significant bits in the magnitude, including BigInt. | | `median` | `median(xs: List) -> Int \| Float` | Order Int/Float values exactly and preserve the middle value (mean of the two middles for even n). An Int middle stays exact; any NaN input makes the result NaN. | | `pstdev` | `pstdev(xs: List) -> Float` | Population standard deviation (divide by n). Preserve finite and subnormal deviations even when their variance is outside Float range. | | `stddev` | `stddev(xs: List) -> Float` | Same as pstdev (population). | | `stdev` | `stdev(xs: List) -> Float` | SAMPLE standard deviation (divide by n - 1); needs two values. Preserve finite and subnormal deviations even when their variance is outside Float range. | | `variance` | `variance(xs: List) -> Float` | SAMPLE variance (divide by n - 1) — statistics.variance; pvariance is the population form. | | `pvariance` | `pvariance(xs: List) -> Float` | Population variance (divide by n) — statistics.pvariance. | ## collection | Builtin | Signature | Description | |---|---|---| | `list` | `list(items...) -> List` | Build a list of the arguments; list(list(1, 2), 3) is [[1, 2], 3]. Use push(xs, x) to append. | | `map` | `map(key, value, ...) -> Map \| list \|> map(x => expr) -> List` | Build a map from key-value pairs (even arg count), or — with a lambda — transform each list element. | | `push` | `push(list: List, items...) -> List` | Return a new list with the items appended (the original is unchanged). | | `nth` | `nth(list: List, i: Int) -> Any` | Element at index i, or () when out of bounds. | | `reverse` | `reverse(list: List) -> List` | Return the list in reverse order. | | `range` | `range(start: Int, end: Int, step?: Int) -> List` | Integers from start toward end (exclusive); bounds and step are 64-bit Ints. A negative step counts down; zero raises kind range. Direct for-loops do not materialize the list, including stepped ranges. | | `sort` | `sort(list: List, order?: "desc") -> List` | Sort scalars ascending (or "desc"); errors on incomparable element types. | | `flatten` | `flatten(list: List) -> List` | Flatten one level of nested lists. | | `zip` | `zip(a: List, b: List) -> List<{left, right}>` | Pair elements positionally; stops at the shorter list. | | `enumerate` | `enumerate(list: List) -> List<{index, value}>` | Attach a 0-based index to each element. | | `with` | `with(m: Map, key, value, ...) -> Map \| with(list: List, i: Int, value) -> List` | Copy of the map with every key-value pair inserted, or copy of the list with element i replaced. An incomplete pair raises kind type; large Int map keys are stringified, while list indices must fit signed 64 bits. | | `without` | `without(m: Map, keys...) -> Map` | Return a copy of the map with the given keys removed. | | `merge` | `merge(a: Map, b: Map) -> Map` | Copy of `a` with all entries of `b` inserted (b wins on conflict). | | `join` | `join(list: List, sep: String) -> String \| join(left: List, right: List, key) -> List` | Join list elements into a string — or, with two lists, an inner data join on `key`. | | `slice` | `slice(xs: List\|String, start: Int, end?: Int) -> List\|String` | Sub-list / substring, end exclusive; negative indexes count from the end (slice(xs, -2) = last two). Clamped, never raises. | | `keys` | `keys(m: Map) -> List` | Keys of a map VALUE, in insertion order. (Memory slots: slot.keys().) | | `values` | `values(m: Map) -> List` | Values of a map VALUE, in insertion order. (Memory slots: slot.values().) | | `entries` | `entries(m: Map) -> List<{key, value}>` | Key/value records of a map VALUE: for e in entries(m) { e.key e.value }. | ## pipeline | Builtin | Signature | Description | |---|---|---| | `filter_by` | `filter_by(rows: List, field, op: ">"\|">="\|"<"\|"<="\|"=="\|"!=", value) -> List` | Keep rows whose `field` compares true against `value` (op defaults to == with 3 args). | | `sort_by` | `sort_by(rows: List, field, order?: "desc") -> List \| sort_by(list, x => key, order?: "desc") -> List` | Stable sort by a field (numbers by value, strings lexicographically) or by a key function; a list key sorts on several keys: sort_by(rows, r => [0 - r.total, r.name]). | | `top` | `top(rows: List, n: Int) -> List` | First n elements, capped at the list length. n must be a nonnegative Int; negative raises kind range, wrong types raise kind type. | | `bottom` | `bottom(rows: List, n: Int) -> List` | Last n elements, capped at the list length. n must be a nonnegative Int; negative raises kind range, wrong types raise kind type. | | `sum_by` | `sum_by(rows: List, field) -> Int \| Float` | Sum of a field across rows: exact Int when every value is an Int, else a Float; a numeric String ("5") counts as its number. | | `avg_by` | `avg_by(rows: List, field) -> Int\|Float` | Mean of a field; Int when whole, () on an empty list. | | `min_by` | `min_by(rows: List, field) -> Map` | Row with the smallest integer value of `field`, or (). | | `max_by` | `max_by(rows: List, field) -> Map` | Row with the largest integer value of `field`, or (). | | `pluck` | `pluck(rows: List, field) -> List` | Extract one field from every row (missing fields become ()). | | `group_by` | `group_by(rows: List, field) -> Map` | Group rows into a map keyed by the field's stringified value: 1 and "1" (true and "true") share a group, a row without the field goes to "unknown" and a () value to "null" — normalise the field first when those differ in your data. | | `distinct` | `distinct(rows: List, field?) -> List` | Unique elements — or, with `field`, the unique VALUES of that field (distinct_by keeps the rows). | | `distinct_by` | `distinct_by(rows: List, field: String) -> List` | The first row per distinct value of `field` (lodash uniqBy / dedup by id). Alias: unique_by. | | `count_by` | `count_by(rows: List, field, value) -> Int` | Number of rows whose `field` stringifies equal to `value`. | | `select` | `select(rows: List, fields...) -> List` | Project each row down to the named fields. | | `agg` | `agg(rows: List, group_field, "col:func"...) -> List` | Group + aggregate: func is sum\|avg\|min\|max\|count; every group also gets a `count`. Groups are keyed like group_by (the field's text). | | `inner_join` | `inner_join(left: List, right: List, key) -> List` | Merge rows whose `key` matches in both lists (left fields win). | | `left_join` | `left_join(left: List, right: List, key) -> List` | Keep every left row, merging matching right-row fields when found. | ## lambda | Builtin | Signature | Description | |---|---|---| | `filter` | `filter(list: List, x => Bool) -> List` | Keep elements where the lambda returns true (it answers a Bool; () counts as false). | | `find` | `find(list: List, x => Bool) -> Any` | First element where the lambda returns true, or (). | | `any` | `any(list: List, x => Bool) -> Bool` | True if the lambda returns true for at least one element. | | `all` | `all(list: List, x => Bool) -> Bool` | True if the lambda returns true for every element (true on empty). | | `count` | `count(list: List, x => Bool) -> Int` | Number of elements where the lambda returns true. | | `reduce` | `reduce(list: List, initial, p => expr) -> Any` | Fold the list; the lambda receives {acc, val} and returns the next acc. | ## io | Builtin | Signature | Description | |---|---|---| | `print` | `print(args...) -> ()` | Print arguments space-separated, then a newline. | | `read_file` | `read_file(path: String) -> String \| {error}` | Read a file as a string; returns {error: ...} on failure. | | `write_file` | `write_file(path: String, content) -> Bool \| {error}` | Write content (stringified) to a file; true on success. | | `read_csv` | `read_csv(path: String, opts: Map?) -> List \| {error}` | Parse an RFC 4180 CSV (quoted fields, "" escapes, multi-line quoted cells, CRLF) with a header row into maps. Unquoted cells are auto-typed Int/Float/String; a quoted cell and a leading-zero id (007) stay Strings; a short row is padded with "", extra fields are dropped. map("raw", true) keeps every cell as text (exact money: "1.00"); map("delimiter", ";") reads a `;`-separated file; any other option is refused. | | `to_csv` | `to_csv(rows: List) -> String` | The CSV text write_csv would write (header from the first row; quoting that from_csv reads back exactly) — for a download without a temp file. | | `from_csv` | `from_csv(text: String, opts: Map?) -> List` | read_csv on CSV text already in memory (an uploaded body): same header, typing, raw and delimiter rules. | | `write_csv` | `write_csv(path: String, rows: List) -> Bool \| {error}` | Write rows as CSV using the first row's keys as the header. A cell read_csv would split, trim or re-type is quoted (separators, quotes, newlines, edge spaces, and a String that reads as a number: "12" comes back "12"); a List/Map is its JSON text, () an empty cell; true comes back as the String "true". | | `read_files` | `read_files(dir: String, count: Int) -> List<{path, content}>` | Read the first `count` files of a directory, in file-name order, with their content (a file that is not UTF-8 text is skipped). There is no listing, move or delete: remember the names you processed in a slot. | | `par_read_files` | `par_read_files(dir: String, count: Int) -> List<{path, content}>` | Thread-parallel variant of read_files. | | `word_count` | `word_count(text: String \| docs: List) -> Map` | Lowercased word frequency of a string or of {content} docs (Rust-speed). | | `par_word_count` | `par_word_count(docs: List) -> Map` | Thread-parallel variant of word_count over a list. | | `read_stdin` | `read_stdin() -> String` | The whole standard input (for `soma run` filters). | | `write_str` | `write_str(s: String) -> Int` | Write s to stdout without a newline; returns the byte count. | ## template | Builtin | Signature | Description | |---|---|---| | `load_template` | `load_template(path: String, key, value, ...) -> String` | Read a file and substitute each {key} placeholder with its value. | | `load` | `load(path: String, key, value, ...) -> String` | Alias of load_template. | | `include` | `include(path: String, key, value, ...) -> String` | Alias of load_template. | | `render` | `render(template: String, key, value, ...) -> String` | Substitute {key} placeholders in an in-memory template string. | | `render_each` | `render_each(rows: List, template: String) -> String` | Render the template once per row, substituting {field} from each map. | ## web | Builtin | Signature | Description | |---|---|---| | `html` | `html(body) -> Response \| html(status: Int, body, header_key, header_value, ...) -> Response` | text/html response; auto-injects HTMX on full pages that use hx- attributes. | | `response` | `response(status: Int, body, header_key, header_value, ...) -> Response` | Response with explicit status, body, and optional headers. | | `redirect` | `redirect(url: String) -> Response` | 302 redirect to `url`. | | `sse` | `sse(streams...) -> Response` | Open a Server-Sent-Events connection that receives only the named streams (no name: every stream). | | `publish` | `publish(stream: String, data) -> ()` | Push data to a runtime-chosen SSE stream name on the event bus. | ## http | Builtin | Signature | Description | |---|---|---| | `http_get` | `http_get(url: String, opts?: {timeout, max_bytes, headers}) -> Map\|List\|String` | GET a URL. 2xx: the body (JSON parsed). Never raises on the network or a status: otherwise {error, kind, status, body}; an invalid or unknown option raises kind type — kind http_status (status + the upstream body), timeout, refused or network. timeout defaults to 30000 ms. | | `http_post` | `http_post(url: String, body, opts?: {timeout, max_bytes, headers}) -> Map\|List\|String` | POST body (a Map/List is sent as JSON, a String as is). Same result shape and default timeout as http_get. Also http_put, http_patch, http_delete(url, opts?). | | `http_put` | `http_put(url: String, body, opts?) -> Map\|List\|String` | PUT; same shape as http_post. | | `http_patch` | `http_patch(url: String, body, opts?) -> Map\|List\|String` | PATCH; same shape as http_post. | | `http_delete` | `http_delete(url: String, opts?) -> Map\|List\|String` | DELETE; same shape as http_get. | | `ws_connect` | `ws_connect(url: String) -> Map` | Open a WebSocket connection; incoming messages dispatch as signals. | | `ws_send` | `ws_send(msg) -> ()` | Send a message on the current WebSocket connection; errors if not connected. | | `link` | `link(addr: "host:port") -> ()` | Open a TCP signal-bus link to a peer node. | | `subscribe` | `subscribe(url: String) -> ()` | Subscribe to a remote event stream; an {"event", "data"} message runs on event(data) only for an event this program emits or soma.toml [bus] accept lists (else refused); other text runs on ws(msg). | | `refusal` | `refusal(kind: String, detail?: String) -> Map` | An HTTP response — status from the kind (`conflict` → 409, `not_found` → 404, `unauthorized` → 401, your own tag → 400) and body {error, kind, detail}, exactly what RAISING that error would answer. Return it when the handler must refuse AND keep what it wrote (an audit row): raising rolls the writes back, returning does not. It is a response envelope like response(): read the fields through `_body` (`r._body.kind`), and return it as the handler's value. | ## time | Builtin | Signature | Description | |---|---|---| | `parse_date` | `parse_date(s: "YYYY-MM-DD") -> {year, month, day, weekday, epoch_day}` | Strict ISO date to its parts (weekday 1 = Monday); raises kind "date" otherwise. | | `add_days` | `add_days(date: String, n: Int) -> String` | The ISO date n days later (negative n goes back), across month and year ends. | | `add_months` | `add_months(date: String, n: Int) -> String` | Same day n months later, clamped to the month's length (Ruby's Date >> n): add_months("2026-01-31", 1) = "2026-02-28". | | `days_between` | `days_between(a: String, b: String) -> Int` | Days from a to b (negative when b is earlier). | | `months_between` | `months_between(a: String, b: String) -> Int` | Whole months from a to b ("YYYY-MM-DD"), day-of-month aware like java.time MONTHS.between: 2026-01-15 → 2026-04-14 is 2, → 2026-04-20 is 3. | | `days_in_month` | `days_in_month(year: Int, month: Int) -> Int` | 28–31, leap years included. | | `now` ✗ | `now() -> Int` | Current Unix timestamp in seconds. | | `now_ms` ✗ | `now_ms() -> Int` | Current Unix timestamp in milliseconds. | | `today` ✗ | `today() -> String` | Today's date as "YYYY-MM-DD" (UTC). | | `format_date` | `format_date(ts: Int) -> String` | Format a Unix-seconds timestamp as "YYYY-MM-DD" (UTC). | | `sleep` | `sleep(ms: Int) -> ()` | Block the current handler for `ms` milliseconds. Requires one Int (other types raise kind type); values outside 0 to 86400000 raise kind range. Under serve it holds the handler lock the whole time. | ## state | Builtin | Signature | Description | |---|---|---| | `next_id` | `next_id() -> Int` | Monotonic per-cell counter, in its own table (persistent under run/serve; per test cell in tests); journaled — a refused request burns no id; the ids drawn inside a `try` that fails are kept (they may have escaped into a local), so ids are unique, not always dense. Exhaustion at 2^63 - 1 raises range; a malformed or negative stored counter raises storage. Legacy migration is confined to the current cell. | | `transition` | `transition(id, target_state: String) -> {id, from, to}` | Move instance `id` to `target_state` (read the new state with get_status(id)); raises kind "invalid_transition" with the valid targets, or "guard_failed". Rolled back if the handler later fails. | | `get_status` | `get_status(id) -> String` | Current state of instance `id` — the INITIAL state when `id` was never transitioned (an unknown id looks like a fresh instance; use has_state(id) to tell them apart). | | `has_state` | `has_state(id) -> Bool` | True when instance `id` was transitioned at least once (a recorded state exists). get_status(id) alone cannot distinguish an unknown id from a fresh one. | | `valid_transitions` | `valid_transitions(id) -> List` | States reachable from instance `id`'s current state. | ## memory | Builtin | Signature | Description | |---|---|---| | `remember` | `remember(key, value) -> ()` | Persist typed data in this cell's agent memory; rejects functions and storage encodings deeper than 100 levels. | | `recall` | `recall(key: String) -> Any` | The value this cell remember()ed under the key, or (); preserves JSON-shaped Strings and never reads another cell's memory. | | `append` | `slot.append(value) -> ()` | Memory-slot method: append a value to a list-backed slot (alias: slot.push). | ## agent | Builtin | Signature | Description | |---|---|---| | `think` | `think(prompt: String, system?: String, opts?: {max_tokens, timeout, max_rounds, tools_allowed, requires}) -> String` | Call the configured LLM with tool-calling, multi-turn context, and budget enforcement. A think() offers the model every tool of the cell's face, or only those named in map("tools_allowed", ["lookup"]) (a call to another is refused and told to the model); `requires` lists model capabilities checked by `soma check`. A timeout is not retried (429/5xx are, up to 3 times). | | `think_json` | `think_json(prompt: String, system?: String, opts?: {max_tokens, timeout, max_rounds, tools_allowed, requires}) -> Map` | Like think(), but parses the response as JSON into a Map (tools are offered too: pass max_rounds 1 to keep the cost bound at one round). There is no schema option — check the fields yourself. | | `horde` | `horde(handler, inputs: List, opts?: {concurrency, max_attempts, budget_tokens, seed, snapshot, instance, on_result, apply, on_done, on_error}) -> String` | Run `handler` (mark it [task]) once per input with a bounded pool of workers; returns the horde id (`Cell:h1`) at once. The queue is persisted with the program's data and resumes after a restart; each result is recorded — and on_result(result) / on_result(input, result) called — with the handler's last step, so once. budget_tokens: a hard ceiling (every think() reserves its bound first; state `exhausted`). on_error(input, error: {error, kind, detail}) after max_attempts (default 1); on_done(id) once at the end. Rounds: snapshot (handler takes (input, snapshot)), apply (in input order at the end), seed (reproducible random()), instance (per-agent memory). concurrency default 8, at most 1000. Without workers (soma test) it runs right after the caller commits. | | `vote` | `vote(handler, input, k: Int) -> Map` | k agents (1..25) answer the same input; {winner, count, k, unanimous, errors, votes}. The most common answer wins, ties to the earliest voter. In a [task] step under serve / run the k calls run at once outside the lock, else in turn; each voter starts a fresh model context; a voter that raises counts in `errors` (all raising: the first error). Voters' writes are theirs — keep them read-only. | | `horde_status` | `horde_status(id: String) -> Map` | {state: running \| cancelling \| exhausting \| done \| cancelled \| exhausted, total, queued, running, done, failed, cancelled, tokens, budget_tokens?} of a horde (its id names its cell). | | `horde_results` | `horde_results(id: String) -> List` | The results of a horde in input order; () for a task not done (failed, cancelled or still queued). | | `horde_cancel` | `horde_cancel(id: String) -> Bool` | Stop a horde: no new task starts, running ones finish, on_done is called. false when it was not running. | | `delegate` | `delegate(cell: String, signal: String, args...) -> Any` | Invoke another cell's handler and return its result. | | `set_budget` | `set_budget(max_tokens: Int) -> ()` | Hard cap on LLM tokens; think() fails once exhausted. | | `tokens_used` | `tokens_used() -> Int` | LLM tokens consumed since the budget was set. | | `tokens_remaining` | `tokens_remaining() -> Int` | Tokens left in the budget, or -1 if unlimited. | | `trace` | `trace() -> List` | Structured execution log: every think(), tool call, and approval. | | `clear_trace` | `clear_trace() -> ()` | Empty the agent trace log. | | `clear_context` | `clear_context() -> ()` | Reset the multi-turn LLM conversation history. | | `approve` | `approve(action: String) -> Bool` | Human-in-the-loop gate. Answered by `mock approve true\|false` in tests, by SOMA_APPROVE=always\|never, or by a person at the terminal under `soma run`; otherwise (e.g. under soma serve) it RAISES kind "approval_required" — it never approves on its own. | ## linalg | Builtin | Signature | Description | |---|---|---| | `matrix` | `matrix("1 2; 3 4") -> List>` | MATLAB-style matrix literal: ';' separates rows, whitespace/',' separates entries. | | `mat` | `mat(rows: Int, cols: Int, values: List) -> List>` | Reshape a flat list into an r×c matrix; errors if the count mismatches. | | `reshape` | `reshape(values, rows: Int, cols: Int) -> Matrix` | Lay a flat list or matrix out row-major as rows×cols; also m.reshape(r,c). | | `transpose` | `transpose(m: Matrix) -> Matrix` | Transpose; also m.transpose(). | | `shape` | `shape(m) -> List` | [rows, cols] for a matrix, [n] for a vector; also m.shape(). | | `matmul` | `matmul(a: Matrix, b: Matrix) -> Matrix` | Matrix product (also the `*` operator on two matrices); inner dims must agree. | | `det` | `det(m: Matrix) -> Float` | Determinant of a square matrix (LU with partial pivoting). | | `diag_sum` | `diag_sum(m: Matrix) -> Float` | Matrix trace: sum of the diagonal. | | `identity` | `identity(n: Int) -> Matrix` | n×n identity matrix (alias of eye). | | `scale` | `scale(m: Matrix, k) -> Matrix` | Scalar-multiply every entry (also `k * m`). | | `flatten_mat` | `flatten_mat(m: Matrix) -> List` | Flatten a matrix to a row-major vector. | | `rows` | `rows(r1: List, r2: List, ...) -> List>` | Build a matrix from row vectors. | | `cols` | `cols(c1: List, c2: List, ...) -> List>` | Build a matrix from column vectors (transposes). | | `eye` | `eye(n: Int) -> List>` | n×n identity matrix. | | `zeros` | `zeros(r: Int, c: Int) -> List>` | r×c matrix of zeros. | | `ones` | `ones(r: Int, c: Int) -> List>` | r×c matrix of ones. | | `diag` | `diag(values: List) -> List>` | Square diagonal matrix from a list. | | `to_sampled` | `to_sampled(A: List>, opts?: {max_rows, max_cols}) -> Map` | Build a BST-backed length-squared sampling handle (Tang); O(log n) per sample after. | | `sample_row` | `sample_row(A) -> Map` | Draw one row index by ℓ²-norm importance sampling (time-seeded PRNG). | | `drop_sampled` | `drop_sampled(handle: Map) -> Bool` | Free a to_sampled() registry entry; true if it existed. | | `importance_sample_rows` | `importance_sample_rows(A, opts: {samples}) -> Map` | Sample rows by squared-norm importance (time-seeded PRNG). | | `svd_lowrank` | `svd_lowrank(A, opts: {row_samples, col_samples, rank, max_dim}) -> Map` | Sublinear randomized low-rank SVD with declared sampling bounds. | | `regress_sgd` | `regress_sgd(A, b: List, opts: {eps, lambda, max_iter, max_dim}) -> Map` | Ridge regression via stochastic gradient descent with declared bounds. | | `clean_covariance` | `clean_covariance(returns: List>, opts: {method: "rie"\|"clip"\|"raw", eta, center, max_assets, max_obs}) -> Map` | RMT (Bouchaud-Potters) covariance cleaning: rows are observations (T), columns assets (N); the sample covariance divides by T (population — numpy cov uses T-1); center (default true) subtracts each column mean; clip replaces the eigenvalues inside the Marchenko-Pastur bulk by their mean; .matrix is the cleaned N×N, .eigenvalues the cleaned spectrum (no eigenvectors). max_obs / max_assets past the data raise kind range. | | `impact_sqrt` | `impact_sqrt(qty: Float, daily_volume: Float, sigma: Float, opts?: {Y}) -> Map` | Bouchaud square-root market-impact law; .bps is expected slippage. | | `quantile` | `quantile(values: List, q: Float) -> Float` | q-th quantile with linear interpolation between the two nearest sorted values (numpy's default): quantile(xs, 0.5) == median(xs). | | `var_historical` | `var_historical(returns: List, opts?: {alpha, max_obs}) -> Float` | Historical Value-at-Risk as a POSITIVE loss: -quantile(returns, 1 - alpha) (returns positive for gains); alpha in (0, 1), else kind range; more observations than max_obs raise kind range. | | `expected_shortfall_historical` | `expected_shortfall_historical(returns: List, opts?: {alpha, max_obs}) -> Float` | Historical expected shortfall (CVaR) as a positive loss: minus the mean of the returns at or below the (1 - alpha) quantile; alpha in (0, 1); max_obs as for var_historical. | | `var_gaussian` | `var_gaussian(returns: List, opts?: {alpha, mu, sigma}) -> Float` | Gaussian VaR assuming N(mu, sigma^2); moments inferred unless overridden. | ## native | Builtin | Signature | Description | |---|---|---| | `buffer` | `buffer(n: Int) -> Buf [native] only` | Array of n Ints, zeroed. Random access with buf_get / buf_set. Not available in interpreted handlers. | | `buf_get` | `buf_get(b: Buf, i: Int) -> Int [native] only` | Read b[i]. | | `buf_set` | `buf_set(b: Buf, i: Int, v: Int) -> () [native] only` | Write b[i] = v. | | `buffer_f` | `buffer_f(n: Int) -> BufF [native] only` | Array of n Floats, zeroed (buf_get_f / buf_set_f). | | `buf_get_f` | `buf_get_f(b: BufF, i: Int) -> Float [native] only` | Read b[i]. | | `buf_set_f` | `buf_set_f(b: BufF, i: Int, v: Float) -> () [native] only` | Write b[i] = v. | | `hashmap` | `hashmap() -> HMap [native] only` | Int → Int hash map (hm_get / hm_set / hm_inc / hm_len / hm_has). | | `hm_get` | `hm_get(m: HMap, k: Int) -> Int [native] only` | Value at k, 0 when absent. | | `hm_set` | `hm_set(m: HMap, k: Int, v: Int) -> () [native] only` | m[k] = v. | | `hm_inc` | `hm_inc(m: HMap, k: Int) -> () [native] only` | m[k] += 1 (inserting 1). | | `hm_len` | `hm_len(m: HMap) -> Int [native] only` | Number of keys. | | `hm_has` | `hm_has(m: HMap, k: Int) -> Bool [native] only` | Whether k is present. | | `strbuf` | `strbuf(capacity?: Int) -> SBuf [native] only` | Growable string builder (sb_push / sb_push_int / sb_push_char / sb_len / sb_finish). | | `sb_push` | `sb_push(b: SBuf, s: String) -> () [native] only` | Append a string. | | `sb_push_int` | `sb_push_int(b: SBuf, n: Int) -> () [native] only` | Append an Int's decimal digits. | | `sb_push_char` | `sb_push_char(b: SBuf, c: Int) -> () [native] only` | Append one character by code point. | | `sb_len` | `sb_len(b: SBuf) -> Int [native] only` | Bytes so far. | | `sb_finish` | `sb_finish(b: SBuf) -> String [native] only` | The built String. | ## internal | Builtin | Signature | Description | |---|---|---| | `_coalesce` | `_coalesce(a, b) -> Any` | Desugared form of `a ?? b`: returns b only when a is (). |