The Trillion-Dollar Glitch: Why Software Architecture Fails

Published Dec 17, 2025

A recent report by IEEE Spectrum, echoing a seminal 2005 investigation, highlights a staggering reality: the global economy is losing trillions of dollars to failed IT projects. The report identifies the usual suspects: unrealistic expectations, willful ignorance by executives, and "bad requirements." It describes a "Groundhog Day" scenario where, despite twenty years of better tools, we keep making the same mistakes.

But these are symptoms, not causes. The IEEE report is descriptive. It catalogs what is failing, but it does not explain why these patterns persist across decades, organizations, and technologies.

If unrealistic expectations were the root cause, better project management tools would have solved the problem. If bad requirements were the issue, clearer documentation would have fixed it. Yet the trillion-dollar leak continues.

What if "bad management" is downstream of a deeper structural problem? What if the reason we cannot manage software projects is that the fundamental physics of how we build them is broken?

The issue isn't communication or discipline. The core problem is that modern software has no source of truth.

The missing wave function

In quantum mechanics, the wave function (Ψ) describes the entire state of a system. If you know the function, you know the probability of every measurement.

In software, we have no system wave function. Instead, we have structural decoherence: multiple independent representations of reality that drift apart over time.

Consider a standard "User" entity:

• In the database: It is a row in a normalized SQL table
• In the API: It is a JSON object
• In the backend: It is a Java class or Rust struct
• In the frontend: It is a TypeScript interface or Redux store

These are not four views of the same truth. They are four independent definitions of reality that we manually try to glue together with ORMs, serializers, and adapters.

The "trillion-dollar leak" described by IEEE comes from the translation tax. Every time we move data across these boundaries, we risk losing the truth. When the business rule changes (e.g., "Users must have two emails"), we have to update the logic in four different places. If we miss one, the system fractures.

This is a profound architectural insight that cuts deeper than "bad management." If the system doesn't know what is true (or worse, has multiple conflicting versions of the truth), failure isn't just a risk; it's a mathematical certainty.

The organizational fracture: where truth fragments

The User entity example shows the technical manifestation of structural decoherence: multiple runtime representations that must stay synchronized. But the problem extends beyond technical layers into organizational ones. The IEEE article's failures map precisely to three distinct domains where truth fragments:

1. The data layer (state integrity)

As systems move toward microservices and distributed architectures, we shatter the single source of truth.

The problem: When data replicates across caches, read-replicas, and client-side stores, where is the truth? Is it in the Postgres database? The Redis cache? The Redux store on the user's browser?

The failure: Bugs arise from state desynchronization. A user sees one balance on their phone and another on the web. The system makes decisions based on stale data. The complexity of keeping these "truths" aligned (cache invalidation, eventual consistency) consumes massive engineering effort and is a primary source of the "glitches" the IEEE report describes.

2. The logic layer (business rules)

Where do the rules of the system live?

The problem: Business logic is smeared across the stack. A validation rule (e.g., "User must be 18") might exist in the SQL constraint, the backend API code, and the frontend JavaScript form validation.

The failure: When the rule changes (e.g., "User must be 21"), it gets updated in two places but forgotten in the third. The "truth" of the business rule is now contradictory, leading to security holes or corrupted data.

3. The definition layer (requirements vs. reality)

This connects to the management failures the IEEE report identifies.

The problem: The requirements document is treated as the source of truth for what the software should do. But documents go stale the moment coding starts.

The failure: The code becomes the de facto truth of how the system works, but no one reads the code except developers. Stakeholders continue making decisions based on the "Paper Truth" (the outdated spec), while developers build against the "Code Truth." The software fails not because it crashed, but because it solved a problem that no longer existed or worked in a way no one expected.

The requirements didn't fail. The physics of state propagation failed.

A new physics: Datom.World

To solve this, we don't need better Jira tickets or smarter project managers. We need a unified source of truth that transcends runtime boundaries. We need to move from place-oriented programming (data lives in mutable variables) to value-oriented programming (data is a flow of immutable facts).

This is what Datom.World provides: a platform where structural decoherence becomes physically impossible.

How does Datom.World make structural decoherence physically impossible? By introducing a single source of truth that all systems read from: the datom stream.

The datom stream is the wave function

The wave function is the complete description of a system's state. It's a single source from which all measurements derive.

In Datom.World, the immutable datom stream is the system wave function. It is the complete, canonical description of all facts that have ever existed in the system.

As the system's wave function, the datom stream has unique properties:

• Complete history: Every datom ever appended is preserved. The stream contains not just the current state, but the entire causal history of how that state came to be.
• Deterministic measurement: Any "measurement" (query) of the system state is a pure function of the stream. Given the same stream, you always get the same result.
• No hidden state: There is no information about the system that exists outside the stream. No cached values, no replica inconsistencies, no "unknown unknowns."
• Queryable at any point: The datom stream can be queried for any fact, at any time, from any perspective.

The four-layer problem (database, API, backend, frontend) can now dissolve. Instead of four independent sources of truth that must be kept synchronized, there is one source of truth (the datom stream) and four interpreters that transform it into different views. We'll see exactly how this works below.

This is not a metaphor. It is a precise structural correspondence:

When an interpreter consumes the datom stream, it is performing a measurement. The interpreter transforms the complete history into a specific view: a database index, a UI component, an API response, an execution trace. Interpretation produces a particular observable structure, but the stream itself remains unchanged and complete.

DaoDB is just one such interpreter. It materializes queryable indexes from the stream, and then Datalog queries run against those indexes. But the frontend (DaoFlow), the backend (Yin.VM continuations), and analytics pipelines are all equally valid interpreters, each transforming the same stream into their own specialized data structures.

1. Everything is a stream

The foundational axiom: everything is a stream of datoms. A datom is an immutable fact represented as [e a v t m] (entity, attribute, value, time, metadata).

Instead of overwriting a database row (destroying history), we append a datom. The "database" is no longer a bucket; it is a log. This provides the "physics" of the universe: an immutable record of everything that has ever happened.

Code, data, state changes, execution traces: all are append-only logs of datoms. This gives us total auditability (nothing deleted, perfect history), time travel (query any past state), and a unified data model. Everything (messages, files, databases, APIs) collapses into a single primitive: the stream.

This solves structural decoherence: there are no longer four independent definitions of a User across database, API, backend, and frontend. There is only one definition: the stream of datoms about that User entity. Each layer is merely an interpreter of the same immutable facts.

The power of restriction

Fixing all information into a five-element tuple eliminates the translation tax entirely. When every system agrees on the same structural shape, there is no negotiation, no schema discovery, no version-specific adapters. The uniform structure makes composition effortless and semantic evolution natural.

Restriction becomes power: a smaller tuple opens a larger world.

2. Everything is a continuation

If streams are how data flows, continuations are how computation flows. Every running process is a continuation: a serializable snapshot that can pause, migrate, and resume anywhere.

The backend, the frontend, and the analytics engine are simply continuations running over the same datom stream. They consume the same datoms and "collapse" them into the state they need (a UI view, an invoice, a report). Because they all derive state from the same immutable stream using deterministic functions, they cannot be out of sync.

Built on Yin.VM, a CESK machine (Control, Environment, Store, Continuation), your code becomes portable and secure. Start on your phone, continue on an edge server, finish on a GPU cluster. In a standard runtime (like the JVM), the state is hidden in registers and heaps. In a CESK machine, the state is a visible, immutable data structure.

3. Interpretation creates meaning

Meaning only emerges through interpretation. Data is just syntax; semantics are created when an interpreter observes and acts on it.

This separation of structure from semantics is what enables the datom model to work. Because all interpreters consume the same uniform stream, they can evolve independently without coordination. This enables semantic evolution (new interpreters don't break old ones), multiple perspectives (different teams interpret the same stream differently), and AI collaboration (agents evolve their understanding without rigid schemas).

4. The universal AST is canonical code

The Universal Abstract Syntax Tree (AST) is the canonical representation of code. But unlike traditional ASTs stored as tree structures scattered across the heap, the AST itself is expressed as datoms. Code and data are literally the same thing: streams of datoms.

This means the entire system (from database schema to frontend rendering to program execution) is expressed in a single, uniform, queryable fabric. Syntax becomes a rendering preference, compilation becomes a query, and the translation tax disappears even at the code level.

Learn more: Universal AST vs Assembly, AST Datom Streams, and Datalog as Compiler Infrastructure.

These four concepts work together to eliminate the runtime fragmentation we diagnosed earlier. Let's see concretely how the User entity problem is solved.

Solving the runtime fragmentation problem

Recall the User entity scattered across multiple runtime representations (database, API, backend, frontend). In the Datom.World model:

1. The stream contains the facts: [user-123 :user/email "alice@example.com" tx-1 {}], [user-123 :user/name "Alice" tx-1 {}]
2. DaoDB materializes an index: The database is just a queryable view over the stream, built by DaoDB. It doesn't "store" the user. It indexes datoms for fast queries.
3. The API streams datoms: Instead of serializing to JSON, the API uses DaoStream to send the raw datom stream. No translation layer.
4. The backend is a continuation: Business logic runs as a continuation in Yin.VM, consuming datoms and producing new datoms. Its state is explicit and serializable.
5. The frontend is also a continuation: Using DaoFlow, the UI renders by interpreting the datom stream as a continuation that can pause and resume. You can use React with Reagent, or any rendering approach. The frontend consumes the stream and materializes views. No separate TypeScript interface, just interpreters over the stream.

When requirements evolve, you don't migrate the data. You evolve the interpreters. If "Users must have two emails" changes to "Users can have many emails," the schema shifts from a scalar attribute to a one-to-many relationship. Old interpreters ignore the new relationship datoms. New interpreters understand them. Interpreters themselves can be upgraded like data by asserting AST datoms. You insert code (as datoms) into the stream that teaches old interpreters the new schema. No migration scripts. No API versioning. No coordinated deploys. The stream accumulates both facts and the code to interpret them.

The translation tax disappears because there is nothing to translate. The runtime representations don't have separate definitions. They have separate interpretations of the same immutable facts.

Runtime fragmentation solved. But recall the organizational fracture: the three domains where truth fragments at the process level (data, logic, definition). The same interpreter model solves these as well.

Solving the organizational fracture

The organizational domains where truth fragments also collapse into interpreters:

1. Data layer (state integrity)

The problem solved: The data layer itself is a datom stream. All state (Postgres cache, Redis replica, Redux store) becomes interpreters materializing views of the same underlying stream of facts.

When DaoDB indexes the stream and the frontend queries it, they cannot be out of sync. They're both pure functions of the same append-only log. Cache invalidation becomes trivial: re-query the stream.

2. Logic layer (business rules)

The problem solved: Business logic runs as continuations, and continuations themselves are expressed as datom streams. A validation rule is a continuation that consumes datoms, produces new datoms, and whose execution state is itself datoms.

When the rule changes from "User must be 18" to "User must be 21," you don't update SQL constraints, backend code, and frontend validation separately. You assert new AST datoms that define the updated continuation. The rule exists in one place: as datoms in the stream.

3. Definition layer (requirements vs. reality)

The problem partially addressed: Because code is expressed as datoms (the Universal AST), the implementation itself becomes queryable. Stakeholders can query what the system actually does, not just what documentation says it should do.

Translating prose requirements into executable code remains a human activity. But the restriction principle suggests that translating requirements into AST datoms is fundamentally easier than translating into Java, Python, or JavaScript.

Why? Because the target is smaller:

• Fixed structure: Every AST node is expressed as [entity attribute value time metadata]. An LLM generating datoms searches a constrained space, not the vast syntactic surface of traditional languages.
• No syntactic variation: The same semantic intent always maps to the same datom structure. The LLM doesn't need to choose between for (int i = 0; i < n; i++) vs for i in range(n): vs (0..n).each. Restriction eliminates ambiguity.
• Compositional guarantees: Malformed AST compositions are structurally impossible. You can't have dangling references in a well-formed datom graph. The five-element constraint enforces well-formedness.
• Semantic focus: When generating Python, an LLM must simultaneously handle indentation, syntax rules, library APIs, idioms, and edge cases. When generating AST datoms, it focuses purely on semantic relationships expressed through entity-attribute-value triples.

The requirements vs. reality gap doesn't disappear, but it narrows because LLMs can more accurately translate natural language requirements into datoms than into Java or Python. The restricted target means fewer translation errors, not easier human review.

What Datom.World provides is a queryable, auditable record of what the code actually implements. The same data available to interpreters and compilers at runtime is available to humans through Datalog queries. This makes the gap visible rather than hidden, and the restricted target representation makes LLM-assisted translation from requirements more tractable.

Time travel debugging by default

The most expensive part of software failure is the "unknown unknown": the bug that cannot be reproduced.

In the IEEE report, failures are often catastrophic and mysterious. Engineers dig through logs, trying to reconstruct what happened from incomplete traces. It's detective work: piecing together clues from fragments.

In the Yin.VM/Datom model, failures are deterministic. Because the system is a pure function of the datom stream, every execution is perfectly reproducible.

If the system crashes at step 1,000,000:

1. We rewind the stream to any previous point
2. We replay the CESK transition function deterministically
3. We query the exact state difference between step 999,999 and step 1,000,000
4. We see precisely which invariant broke and why

We don't guess from logs. We don't reconstruct from memory dumps. We rewind and replay the exact sequence of immutable facts that led to the failure. The datom stream is a complete recording of everything that happened.

Entropy is not a bug, it is a law

The datom stream solves structural decoherence: the translation tax that causes the trillion-dollar leak. But solving the physics of state propagation doesn't eliminate all architectural challenges.

Even with the right physics, architectural constraints still face a fundamental challenge: pressure always finds the cheapest path.

As code evolves:

• requirements change
• teams grow and rotate
• timelines compress
• knowledge fragments

Entropy increases not because of bad intent, but because change has a cost.

The question is not whether entropy increases, but where it increases.

Just as biological systems maintain internal order while increasing entropy in their environment, good code distributes complexity outward (more modules, more nodes) while keeping each module internally coherent. System-level entropy increases (more parts, more distribution), but module-level entropy stays low (each piece remains organized).

Bad architectures concentrate entropy internally: tangled coupling, shared mutable state, unpredictable interactions. The result is rigidity. Changes cascade unpredictably because the disorder lives inside the components.

Good architectures push entropy to the boundaries: more distributed nodes, explicit message passing, immutable streams. The system topology becomes more complex (higher system entropy), but each node stays simple (lower local entropy). The result is malleability. Changes are local because the disorder lives in the structure of the network, not within the nodes.

No developer writes bad code on purpose. But the path to high-entropy code is easier than the path to low-entropy code. Without structural constraints, systems naturally drift toward disorder simply because there are more ways to be disorganized than organized.

Architectures that survive long-term are not those that ask developers to take the harder path, but those that make the correct path the easiest path.

This is the distinction that matters.

Case study: Why Polylith will decay

To illustrate the difference between physics-enforced and policy-enforced constraints, consider Polylith: a well-designed architectural pattern that nonetheless faces the same entropy problem as traditional layered architectures.

Polylith is a good idea. It exists for the same reason many architectural patterns exist: to slow the natural accumulation of entropy in growing codebases.

But slowing entropy is not the same as stopping it. And Polylith cannot stop it because it cannot be enforced.

What Polylith gets right

Polylith recognizes several truths that many architectures ignore:

• Unbounded coupling is the fastest path to entropy
• Reuse must not imply shared ownership
• Composition should be explicit
• Dependencies should be directional

By separating:

• components (reusable units)
• bases (applications)
• projects (shared contexts)

Polylith attempts to localize change and prevent global entanglement.

At the conceptual level, this is correct.

Where Polylith fails: three fundamental problems

Beyond the enforcement problem, Polylith faces three structural challenges that reveal why policy-based constraints cannot match physics-based ones. Each represents a domain where Polylith adds complexity to manage problems that stream-based systems eliminate entirely.

1. The problem of coupling (interfaces vs. interpreters)

Polylith's approach: Polylith fights coupling by enforcing strict interfaces. Brick A cannot talk to Brick B unless it goes through interface.clj. This is syntactic decoupling. You are still mentally binding "Caller" to "Callee."

Datom.World's simplification: It introduces semantic decoupling. You don't call a function; you emit a datom stream.

Instead of UserComponent calling EmailComponent.send(), the UserComponent emits a fact: [:user/registered "bob"].

The EmailInterpreter (which might live in a Polylith component) observes the stream, sees the fact, and interprets it as a command to send an email.

Result: The "Caller" doesn't know the "Interpreter" exists. The coupling is strictly to the data schema, not the code signature.

2. The problem of boundaries (microservices vs. continuations)

Polylith's approach: Polylith spends significant energy defining "Bases" to wrap code into artifacts (JARs, Lambdas) to create network boundaries. It assumes the network is a cliff you have to build bridges over.

Datom.World's simplification: It uses continuations. Since the computing model is "Interpreters processing streams," the code state is portable.

A stream of datoms can flow from a client (Interpreter A) to a server (Interpreter B) transparently. You don't need complex Polylith logic to "expose" a component over HTTP. The stream is the API.

Result: The boundary dissolves because the data flow is uniform regardless of location. Network topology becomes a deployment detail, not an architectural constraint.

3. The problem of logic (functions vs. rules)

Polylith's approach: Inside a Polylith brick, you typically write standard imperative or functional code (A calls B calls C). As logic grows, the call graph gets messy, requiring more Polylith bricks to split it up.

Datom.World's simplification: Computation is framed as interpretation.

You don't write a sprawling process-order function. You write an interpreter (perhaps using Datalog rules or transducers) that transforms a stream of [:order/placed] datoms into [:inventory/reserved] datoms.

Result: The logic is flattened. It's just "Data In → Interpreter → Data Out". This removes the cyclomatic complexity that necessitates Polylith's extreme modularity in the first place.

Coexistence: Polylith as an interpreter container

This analysis doesn't mean Polylith is incompatible with Datom.World. You can validly use Polylith to house your Datom.World interpreters:

• bases/: Defines the runtime nodes (e.g., a Browser Node, a Server Node)
• components/: Defines the interpreters
  • components/order-interpreter: Reacts to order streams
  • components/ui-interpreter: Reacts to state streams to render HTML (DaoFlow)

The simplification: If you use Datom.World, your Polylith structure becomes incredibly boring. You don't need complex interface hierarchies or dependency injection frameworks. You just have a collection of interpreters listening to the same "World" of streams.

The overhead of Polylith (the tooling, the structure, the interface definitions) vanishes because the complexity of coordination has been moved out of the file structure and into the stream. Polylith becomes a simple organizational convention rather than a load-bearing architectural framework.

Where Polylith fails: enforcement

But even when used to organize interpreters, Polylith's fundamental weakness remains. It enforces its constraints through:

• directory layout
• tooling checks
• conventions
• team discipline

These are social constraints.

Social constraints decay.

Not immediately. Not catastrophically. But inevitably.

Why?

Because under pressure:

• violating the rule is cheaper than maintaining it
• exceptions feel justified
• "temporary" shortcuts become permanent

If a constraint can be bypassed, it eventually will be.

The predictable decay pattern

Polylith systems tend to follow a familiar trajectory:

1. Early phase

Clean components, strict boundaries, architectural clarity.

2. Growth phase

Shared utilities emerge, small violations appear, tooling warnings get ignored.

3. Pressure phase

Deadlines dominate, cross-component access becomes "necessary", boundaries soften.

4. Late phase

The Polylith structure remains, but the entropy has re-centralized.

The architecture survives symbolically. The constraints do not.

Policy vs physics

This is the core mistake.

Polylith encodes physics as policy.

It says:

"Components should not know about each other arbitrarily"

But it does not make that physically impossible.

The system relies on developers to continually choose the harder path.

Under entropy, the harder path always loses.

Streams: the same constraint, enforced at the right layer

In stream-based systems (such as Datom.World), the same architectural rule exists:

• dependencies are directional
• components cannot arbitrarily know about each other

But the enforcement is different.

An agent does not depend on another agent. It subscribes to a stream.

• no identity coupling
• no backchannel
• no shared state
• no reverse flow

Directionality is not a rule. It is a law.

You cannot violate it without breaking causality.

Why streams do not decay the same way

Because there is no cheaper shortcut.

You cannot:

• "just this once" reach into another component
• share state for convenience
• bypass the model under pressure

The only way to interact is through the stream.

The cheapest path is the correct path.

This is the difference between discipline and physics.

A general law of architecture

This leads to a simple rule:

Any architectural constraint that is not enforced by the execution model will decay over time.

Polylith slows internal entropy through discipline. Streams redirect entropy to the topology through physics.

One requires vigilance. The other requires nothing.

What Polylith really is

Polylith is not wrong.

It is a human-scale approximation of a deeper principle:

• localize change
• make dependencies explicit
• prevent uncontrolled coupling

It works — temporarily.

But it cannot survive long-term pressure, because it lives above the layer where enforcement must happen.

Conclusion

The IEEE Spectrum report is a phenomenological description of a system in entropy. It blames the people trying to manage the chaos.

But you cannot manage chaos. You must structure it.

Until we adopt an architecture where the source of truth is mathematically defined (a system wave function) and structurally enforced (CESK/datoms), we will continue to burn trillions of dollars building towers of Babel that collapse under their own complexity.

Entropy is not fought with rules. It is fought with constraints that cannot be bypassed.

Polylith asks developers to behave well. Stream-based systems make bad behavior impossible.

One will decay. The other will evolve.

That difference is not about taste. It is about physics.

The industry is waiting for better management. It should be waiting for better physics.

Learn more:

Core concepts:

• Datom.World: Platform overview
• The Power of Restriction: Why a fixed tuple eliminates structural negotiation
• Streaming Datoms with Transducers: Everything is a stream
• Structure vs Interpretation: Interpretation creates meaning
• Yin.vm: Chinese Characters for Programming Languages: The universal AST
• AST Datom Streams: Bytecode Performance with Semantic Preservation: Even the AST is expressed as datoms

Architecture components:

• Yin.VM: The CESK continuation machine
• DaoStream: Universal streaming API
• DaoDB: Materialized views from streams
• DaoFlow: Reactive UI interpretation

Related:

• All Money Is Monopoly Money: On entropy and flow
• Datalog as Compiler Infrastructure: Queryable compilation