Datom.World
Compiler

Yang Compiler — The generative half of Yin

If Yin is the continuation engine, the Yang compiler is its expression engine. Together they form a dual system—Yin for execution, reception, and introspection; Yang for generation, expression, and emission.

Yang transforms high-level intents into continuations that Yin can execute. It shapes the Universal AST (U-AST), seeds migrations, and ensures every emitted agent carries context for the runtime to honor.

Multi-language compiler collection

Yang is not a single compiler—it's a collection of compilers that transform code from different programming languages into the same Universal AST format. Each language compiler understands its source syntax and semantics, then produces the common intermediate representation that Yin executes.

Currently implemented:

  • yang.clojure — Compiles Clojure s-expressions to Universal AST
  • yang.python — Compiles Python syntax to Universal AST

This architecture enables true language interoperability —Python functions can call Clojure functions and vice versa, because both compile to the same Universal AST that Yin understands.

The Clojure compiler

The yang.clojure compiler transforms Clojure s-expressions into Universal AST. Written in .cljc format, it runs identically on both JVM and Node.js.

Compilation pipeline

  • Input: Clojure forms as data (s-expressions)
  • Dispatch: Pattern matching on form structure (literal, symbol, list, special form)
  • Transform: Map Clojure semantics to Universal AST nodes
  • Output: Universal AST maps ready for Yin execution

Supported features

  • Literals (numbers, strings, booleans, collections)
  • Variables (symbol lookup)
  • Lambda expressions (fn [x] (* x 2)) → {:type :lambda ...}
  • Function application (+ 1 2) → {:type :application ...}
  • Conditionals (if test then else) → {:type :if ...}
  • Let bindings (desugared to nested lambda applications)
  • Do blocks (sequential execution via lambdas)

Smart desugaring

The Clojure compiler transforms high-level constructs into simpler primitives:

(let [x 1 y 2] (+ x y))
 ;; Desugars to:
 ((fn [x] ((fn [y] (+ x y)) 2)) 1)

This transformation eliminates the need for a separate let construct in the Universal AST—everything becomes lambdas and applications.

The Python compiler

The yang.python compiler parses Python source text and compiles it to Universal AST. Unlike the Clojure compiler, which operates on already-parsed s-expressions, the Python compiler includes a complete tokenizer and recursive-descent parser .

Three-stage compilation

  • Tokenize: Python source → token stream
  • Parse: Tokens → Python-specific AST (with operator precedence)
  • Compile: Python AST → Universal AST

Tokenizer

The tokenizer breaks Python source into meaningful tokens using regex patterns:

"lambda x: x * 2"
 ;; Tokenizes to:
 [[:keyword "lambda"]
  [:identifier "x"]
  [:colon ":"]
  [:identifier "x"]
  [:operator "*"]
  [:number "2"]]

Parser

The recursive-descent parser constructs a Python-specific AST with proper operator precedence:

Python Python AST
2 + 3 * 4 {:py-type :binop, :op +, :left 2, :right {:py-type :binop, :op *, ...}}

Operator precedence ensures 3 * 4 evaluates before + 2 .

Supported Python features

  • Literals: 42 , "hello" , True , None
  • Variables: x , my_var
  • Lambda expressions: lambda x, y: x + y
  • Function definitions: def double(x): return x * 2
  • Function calls: f(1, 2) , (lambda x: x * 2)(21)
  • Binary operators with precedence: + , - , * , / , == , < , >
  • If expressions: 10 if x > 5 else 20
  • Higher-order functions: (lambda f: f(5))(lambda x: x * 2)

Language interoperability

Because both compilers produce the same Universal AST format, Python and Clojure code can be composed together :

;; Define a function in Python
 (def py-triple
   (yang.python/compile "lambda x: x * 3"))
 
 ;; Define a function in Clojure
 (def clj-add10
   (yang.clojure/compile '(fn [x] (+ x 10))))
 
 ;; Compose them!
 (def composed
   {:type :application
    :operator clj-add10
    :operands [{:type :application
                :operator py-triple
                :operands [{:type :literal :value 5}]}]})
 
 ;; Execute: add10(triple(5)) = add10(15) = 25
 (vm/run initial-state composed)
 ;; => {:value 25}

The Python lambda x: x * 3 and Clojure (fn [x] (+ x 10)) compile to identical Universal AST structures (both are :lambda nodes), so Yin can execute them interchangeably.

Universal AST as lingua franca

The Universal AST serves as the common language between all source languages and the Yin VM:

Source Language Universal AST
lambda x: x * 2 Python {:type :lambda, :params [x], :body {...}}
(fn [x] (* x 2)) Clojure {:type :lambda, :params [x], :body {...}}
x => x * 2 JavaScript (future) {:type :lambda, :params [x], :body {...}}

All three languages compile to the same Universal AST, enabling seamless cross-language function calls and data sharing.

Learn more about how the Universal AST enables cross-language semantic preservation in Yin.vm: Chinese Characters for Programming Languages , how different language semantics map to a unified representation in The Semantic Impedance Mismatch , and how ASTs are stored as queryable datom streams in AST as Higher Dimensional Construction of Datom Streams .

Cross-platform architecture

Both compilers are written in .cljc format— single source code that runs on both JVM and Node.js :

  • JVM: Full Clojure environment with Java interop
  • Node.js: ClojureScript compilation to JavaScript
  • Browser: (Future) In-browser compilation

The compilers use only portable Clojure/ClojureScript features— no platform-specific dependencies in the core logic. Platform-specific I/O (file reading) is isolated in separate yang.io modules.

Documentation & Source

Explore the Yang compiler implementation in depth:

  • Clojure Compiler Source — 226 lines of portable code
  • Python Compiler Source — 450+ lines including tokenizer and parser
  • Clojure Compiler Documentation — Usage examples and implementation details
  • Python Compiler Documentation — Tokenizer, parser, and compiler architecture
  • Multi-Language Architecture — How Yang enables cross-language composition

Where to go next

  • Revisit Yin to see how continuations are scheduled and migrated.
  • Explore DaoFlow to watch emitted continuations shape living interfaces.
  • Study Shibi to combine economic signaling with generated agents.