Streaming Datoms with Transducers

One of the quiet superpowers behind Datom.World is that every fact is a datom in an immutable stream. Because we model changes as a log of tuples, we can rebuild state anywhere — on a phone, in a WASM runtime, or across a mesh of edge nodes — as long as we can fold over the stream.

Clojure's transducers let us process those streams with almost no ceremony. In this post we will build a focused pipeline that filters datoms, transforms them into indexed entities, and materializes a snapshot ready for UI consumption.

From tuples to pipelines

Start with a handful of datoms. Each tuple follows the `[entity attribute value tx-id context]` shape, so we can describe anything from identities to permissions. Transducers let us chain pure steps over that stream without creating intermediate collections.

(ns datomworld.blog.streams
  (:require [clojure.core.reducers :as r]
            [clojure.set :as set]))

(defn stream->entities
  "Extract entity/type pairs from the datom log."
  [datoms]
  (sequence
   (comp
    (filter #(= (:a %) :entity/type))
    (map (juxt :e :v)))
   datoms))

(defn materialize-snapshot
  "Build an entity map indexed by :db/id."
  [datoms]
  (into {}
        (map (fn [[entity type]] [entity {:db/id entity
                                          :entity/type type}]))
        (stream->entities datoms)))

(defn transduce-datoms
  "Apply any transducer to the datom stream in a single pass."
  [xf datoms]
  (transduce xf conj datoms))

Because transducers are just functions that describe composition, we can reuse the same building blocks on the edge, on the desktop, or anywhere else we ship the stream.

Entangling facts at the edge

With the helper in place we can materialize a snapshot for a UI or analytics job. Here is a practical sequence that derives entity metadata while counting assertions in a single pass.

(def datoms
  [{:e 1 :a :entity/type :v :person :tx 1001 :c :alpha}
   {:e 1 :a :person/name :v "Ada" :tx 1001 :c :alpha}
   {:e 2 :a :entity/type :v :device :tx 1002 :c :beta}
   {:e 2 :a :device/os :v :ios :tx 1002 :c :beta}])

(def entity-types
  (materialize-snapshot datoms))

(def assertion-count
  (transduce (map (constantly 1)) + datoms))

Using a pure transducer pipeline keeps our runtime deterministic. No matter where the stream lands, the tuple log can be folded into the same shape, so conflict resolution becomes another transformation instead of bespoke state management.

Where to take it next

1. Compose filters for permission-aware replication flows.
2. Emit on-change notifications by threading transducers into `core.async` channels.
3. Ship the same pipelines to a WASM runtime for offline-first experiences.

This is the style of Clojure you will find throughout Datom.World — declarative, stream-native, and easy to reason about in production.