Datom.World

Universal AST vs Assembly: High-Level Semantics in Low-Level Form

Published Nov 15, 2025

The Observation

Look at how a simple function like (fn [count] (+ count 1)) appears when compiled by Yang into the Universal AST and projected as semantic tuples:

[-1 :yin/type :lambda]
[-1 :yin/params [count]]
[-1 :yin/body -2]
[-2 :yin/type :application]
[-2 :yin/operator -3]
[-2 :yin/operands [-4 -5]]
[-3 :yin/type :variable]
[-3 :yin/name +]
[-4 :yin/type :variable]
[-4 :yin/name count]
[-5 :yin/type :literal]
[-5 :yin/value 1]
[-1 :yin/root true]

Or when lowered into dao.jing as content-addressed index segments:

[-1 :lambda [count] -2]
[-2 :app -3 [-4 -5]]
[-3 :var +]
[-4 :var count]
[-5 :lit 1]

Now compare that to machine assembly:

MOV  R1, [R0]        ; load count from memory
ADD  R1, R1, #1      ; count = count + 1
RET                  ; return result

They look strikingly similar. Both are flat sequences of low-level instructions. Both reference entities by identifier (-1, -2, or R1). Both are explicit and verbose. Is the Universal AST just assembly code for semantics?

Yes and No

The Universal AST is like assembly in form, but fundamentally different in abstraction level.

Like Assembly:

  • Flat structure: no nested syntax sugar, every node connection is explicit
  • Low-level primitives: tuple rows and datoms instead of complex grammar trees
  • References by ID: entity IDs (-1, -2) identify sub-expressions just as register names identify hardware slots
  • Verbose: every semantic relationship is a separate, inspectable fact
  • Canonical: a single language-neutral representation, not separate dialects for Clojure, Python, or PHP

Unlike Assembly:

  • Semantic, not mechanical: types, functions, applications, and lexical scopes instead of registers, stack pointers, and cache lines
  • High-level abstractions preserved: closures, continuations, and stream effects remain first-class, not compiled down to raw load/store ops
  • Queryable: Datalog queries over dao.space can inspect call graphs and variable references directly without disassembling binaries
  • Content-addressed: code content in dao.jing is immutable and identified by hash; historical publication is tracked in a ledger without mutating code
  • Syntax-independent: Yang can project the same canonical AST back into Clojure, Python, or PHP syntax

Assembly for What Layer?

The central distinction comes down to one question: assembly for what abstraction layer?

Machine Assembly: Hardware Abstraction

Machine assembly is the lowest level before physical hardware:

ADD R1, R2, R3    ; hardware ALU adds two registers

It abstracts physical electronics:

  • Transistors -> registers
  • Voltage levels -> bits
  • Circuit paths -> instructions

Below assembly is hardware. Above it is everything else (runtimes, compilers, high-level languages).

Universal AST: Semantic Abstraction

The Universal AST is the lowest level of semantics before text syntax:

[-2 :yin/type :application]    ; semantic function application

It abstracts linguistic structure:

  • Characters -> AST nodes
  • Surface grammar -> semantic primitives
  • Language-specific idioms -> universal closures and applications

Below the Universal AST is text syntax. Above it is the pure semantic model of the program. The Universal AST is the canonical representation of the code. Text files are merely serialized projections.

The Inversion

This architectural stance inverts the traditional compilation pipeline:

Traditional Stack:

High Level:  Python / Java / Clojure  (source text)
             ↓ compile
Mid Level:   Bytecode / IR             (intermediate)
             ↓ compile
Low Level:   Assembly                  (machine code)
             ↓ assemble
Hardware:    CPU instructions

Traditional architectures treat source text as the source of truth, compiling downward and losing semantic structure at each phase.

Yin.vm Stack:

Semantic:    Universal AST (Map AST / Canonical Tuples)
             ↓ render                 ↓ lower (yin.vm.linearize)
Syntax:      Clojure / Python / PHP   Linear Executable Datoms (:yin.code/*)
             (projections / views)    ↓ load
                                      CESK Machine Execution (yin.vm.semantic)

In Yin, the Universal AST is canonical. Source text is a rendering for human readability. Executable datoms are an explicit linearization for fast execution.

The AST resembles assembly because it is the lowest level of the semantic domain, just as assembly is the lowest level of the execution domain.

Why This Matters: Different Optimization Targets

Machine assembly optimizes for hardware execution speed. The Universal AST optimizes for semantic preservation and structural inspection.

Assembly Optimizations:

  • Register allocation: minimizing memory bus round-trips
  • Instruction scheduling: avoiding pipeline stalls
  • Dead code elimination: stripping unused bytes
  • Branch prediction hints: keeping execution pipelines full

Universal AST Optimizations:

  • Semantic equivalence: preserving identical meaning across diverse host languages
  • Queryability: enabling indexers to expose structural Datalog relations
  • Content addressing: deduplicating code blocks by cryptographic hash in dao.jing
  • Continuations: preserving serializable CESK states across stream boundaries

Composability: The Key Difference

Assembly instructions do not compose into structured queries:

; Adding two numbers
MOV R1, #5
MOV R2, #3
ADD R3, R1, R2

; Adding two other numbers
MOV R4, #10
MOV R5, #7
ADD R6, R4, R5

You cannot easily ask an assembly file: Findallcallsitesofaddition. You would have to parse raw opcodes, reconstruct control-flow graphs, and guess register lifespans.

Because the Universal AST projects into structured semantic tuples, it is queryable with Datalog in DaoSpace:

;; Find all function applications calling +
[:find ?app
 :where
 [?app :yin/type :application]
 [?app :yin/operator ?op]
 [?op :yin/name +]]

;; Find all variable references to parameter count
[:find ?node
 :where
 [?node :yin/type :variable]
 [?node :yin/name count]]

The low-level tuple form enables high-level queries. Instead of obscuring intent behind register allocations, the explicit tuple representation makes intent directly inspectable.

The RISC Analogy

A useful mental model is RISC vs CISC, applied to semantics instead of microprocessors:

CISC: Complex Instruction Sets

  • Rich, idiosyncratic instructions for specific patterns
  • Specialized opcodes for string compares, object dispatch, or loop setup
  • Difficult to optimize across orthogonal paradigms

RISC: Reduced Semantic Sets

  • A compact core of orthogonal primitives: literals, variables, lambdas, applications, conditionals, and stream effects
  • Complex language constructs like classes or pattern matching lower into compositions of primitives
  • Uniform execution via CESK state transitions
  • Language-agnostic optimization and analysis

The Universal AST is RISC for semantics. It eliminates linguistic idiosyncrasies, giving every language a clean, minimal foundation.

Why It Looks Like Assembly: Explicit Is Better

High-level source code hides implicit semantics:

# Python
items.sort()

// Java
Collections.sort(items);

;; Clojure
(sort items)

Each hides critical operational choices:

  • Does this mutate in place or return a fresh collection?
  • What comparator is used?
  • Are side effects or stream interactions involved?

In the Universal AST, every reference and argument is explicit:

[-1 :yin/type :application]
[-1 :yin/operator -2]
[-1 :yin/operands [-3 -4]]
[-2 :yin/type :variable]
[-2 :yin/name sort]
[-3 :yin/type :variable]
[-3 :yin/name items]
[-4 :yin/type :variable]
[-4 :yin/name default-comparator]

This appears verbose, just as assembly is verbose. But while assembly is verbose about execution steps (moving bytes between registers and RAM), the Universal AST is verbose about semantic facts. Every argument, operator, and dependency is named and queryable.

Human Readability and Rendered Syntax

Assembly is difficult for humans to write because developers think in domain abstractions, not register swaps. Does the Universal AST suffer from the same problem?

No. Developers do not write raw AST tuples by hand. They write standard Clojure, Python, or PHP. Yang compiles source text into the Universal AST, and can render the Universal AST back into any supported syntax.

Different developers on the same team can view the same canonical program through different syntaxes. The canonical form in dao.jing is for machines to store, stream, and verify; the rendered syntax is for human eyes.

What About Performance?

Does an explicit, tuple-based representation create unacceptable runtime overhead?

No. In Yin, representation and execution are cleanly separated:

  1. Map AST for walking: The AST-walking evaluator steps directly through the nested Map AST in memory when rapid iteration and macro-like inspection are desired.
  2. Linear executable datoms: For fast execution, yin.vm.linearize lowers AST datoms into linear instruction datoms (:yin.code/*). The loader decodes these into a dense, PC-indexed vector of opcodes and operands, executed by the register-based CESK loop in yin.vm.semantic.
  3. Observer decoupling: Because code flows across a dao.stream, an execution engine and an indexer like dao.space.index can observe the exact same stream concurrently. One executes the image at full speed, while the other materializes Datalog indexes. Neither slows down the other.

You get assembly-grade execution performance through linear datom images, combined with full structural queryability in DaoSpace.

The Hierarchy

The resulting system architecture maps cleanly across abstraction layers:

Abstraction Level                     | Purpose
─────────────────────────────────────────────────────────────────────────────
Universal AST (:yin/* datoms / rows)   | Semantic preservation, Datalog queries
  (canonical data in dao.jing)         | Language-independent interchange
─────────────────────────────────────────────────────────────────────────────
Source Code Views                      | Human readability
  (Clojure / Python / PHP via Yang)    | Syntax preferences
─────────────────────────────────────────────────────────────────────────────
Linear Executable Datoms (:yin.code/*) | Execution optimization
  (yin.vm.semantic image vector)       | Fast CESK state machine transitions
─────────────────────────────────────────────────────────────────────────────
Host Assembly / Machine Code           | Physical hardware execution
  (JVM / V8 / Dart AOT / ARM / x86)    | Direct CPU execution

The Universal AST sits above source code in its semantic purity, but looks like assembly in form because it is the fundamental, irreducible ground of the semantic domain.

Conclusion: Low-Level Form, High-Level Semantics

The Universal AST looks like assembly because it is flat, explicit, and composed of primitive facts. But:

  • Assembly represents low-level execution (registers, memory addresses, CPU flags).
  • Universal AST represents low-level semantics (closures, bindings, expressions, effects).
  • Assembly loses information as high-level code compiles down.
  • Universal AST preserves complete information, allowing high-level syntax to be projected upward without loss.

By decoupling syntax from semantics and execution from representation, Yin inverts the traditional compiler stack. The AST is not an ephemeral intermediate step. It is the permanent, queryable, canonical reality of the program.

Learn more: