The Loop Was the Top of a Stack

Two essays ago I typed the thirty-two jobs that run underneath my days and found the word loop had been hiding seven kinds of thing. One essay ago I drew those jobs as a graph and found the pile I had feared was a short chain with one load-bearing root. I thought that was the bottom of it. I thought the loop was the top of the system, the thing everything else served.

The loop is the top. It is just the top of a stack, and I had never counted the floors below it. When I did there were eight, and the one bearing all the weight was the smallest, and the one I had paid the least attention to: the edge.

The floor is the edge

For a year I built my system the way most people build anything they are trying to understand. I catalogued the things. Every job got a name, a type, a criticality, a place in a registry, and the registry grew more complete every week and no more legible. What I did not see until I went looking for the floor is that a catalogue of things, however complete, is an inventory, and an inventory is the least intelligent structure you can build. It can tell you what you own. It cannot tell you what any of it is for.

The thing that tells you what a job is for was never in the job. It was in the line running out of it to another job. My registry had those lines all along, sitting in two plain fields on every entry: what this job reads, and what this job writes. I had been reading them as attributes of the node. They are not attributes of anything. Reads-what-that-one-writes is a relationship, and a relationship is its own object, with its own existence, independent of the two things it joins.

Call that object the edge, and the whole picture inverts. The node stops being the unit of the system and becomes a place where edges meet. A goal that requires a capability, a task that advances a goal, a decision that was based on a piece of evidence, an event that changed a state: in every one of those the node is a noun you could have guessed and the edge is the sentence that carries the meaning. A system built around its nodes accumulates into an inventory. A system built around its edges becomes something you can reason about, because reasoning is almost entirely a matter of relationships, and the edge is where a relationship finally gets to be a first-class thing instead of a footnote on a node. This is the same inversion I make with any territory I try to chart. The things were never the map. The lines between them were.

Three floors of meaning

An edge tells you two things are related. It does not tell you what the relation means, whether it is still true, or what it might cause. Those are three separate questions, and the stack answers them on three separate floors, which is the part I would have collapsed into one if I had built it carelessly.

The first floor is knowledge, and its whole job is to refuse to treat unlike claims as alike. A deployment failed is a fact. The deployment failed because of configuration drift is a claim. The configuration might be generated nondeterministically is a hypothesis. Freeze deployments until we know is a decision. The failure went away after the rollback is an observation. Those five sentences are about the same night, and a careless system files them as five equal truths and moves on. They are not equal. A system that files them as equal knows nothing; it only stores strings. A fact and a hypothesis have different rights: one you build on, the other you test, and a structure that cannot tell them apart will eventually act on a guess as though it were the ground.

The second floor is time. My first instinct with any piece of state was to keep it current. This project is healthy, and when it stops being healthy, overwrite healthy with at risk. That instinct quietly destroys the most useful thing the system could have known. The moment you overwrite the old value you can no longer ask when it changed, or what was decided while the old value was still believed, or how long this had been drifting before anyone noticed. So the temporal floor keeps the transitions instead of the latest value. Healthy from the first to the twelfth. Degraded from the twelfth to the fourteenth. At risk since. Now the state is not a fact, it is a history, and a history answers questions a snapshot cannot: what changed, when the slipping started, which assumptions have quietly expired. A system without this floor is a dashboard. A system with it has a memory.

The third floor is cause, and it has to be built to stay humble. It is tempting to write message overload causes lost focus as though it were a law. It is not a law, it is a guess with a number on it, and the causal floor makes the guess wear its status on its face: message overload possibly contributes to lost focus, confidence of sixty-eight percent, here is the evidence, here is the counterevidence, status still open. Once a cause is provisional in that way, an intervention stops being a fix and becomes an experiment. Put a twice-daily wall around the messages. Predict that the context switching drops. Measure the switches. Watch the number move or fail to move, and let the result raise or lower the confidence on the edge. That is what turns a diagram of your intuitions into a model that updates when the world disagrees with it.

The floor where it finally moves

Everything so far is understanding, and understanding does not do anything. At some point the system has to act, and action cannot happen in a graph, because a graph is allowed to contain cycles and action is not. Research informs a build, the build produces observations, the observations revise the research: a perfectly good cycle to understand, and an impossible one to execute, because you cannot do the first thing until you have done the last. So the execution floor does something specific with the graph above it. It projects a path through it. It takes the goals that are actually live and the evidence that is actually current, and it flattens that into a directed acyclic graph, an ordering with no cycles, a thing you can walk from front to back. The discipline that governs orderings like this is a solved one, which I found out the slightly embarrassing way.

This is the sharpest line in the whole stack, and I had it backward for a long time. The graph is supposed to hold the complexity. The DAG is not supposed to hold any of it. The DAG is a single actionable slice pulled out of the graph for right now, and when now changes you pull a new one. I used to keep an eternal task list and try to make it match reality, which is exactly the wrong shape, because a list that never regenerates rots the moment the world moves. When I drew my own jobs as a graph, fourteen of the thirty-two lay in a real dependency spine and eighteen depended on nothing at all, and the head of that spine was a single job that builds the store almost everything else reads. That spine is the DAG the graph was carrying inside it. The other eighteen are free and can run in any order, and the whole point of keeping the two floors separate is that I can see at a glance which is which. The graph is where the complexity lives. The DAG is the one clean path I pull out of it when it is time to move.

The two floors that keep it alive

Everything under here is inert. It is a very good map, and a map does not update itself. The seventh floor is the one that does, and it is where the system stops being a data model and starts being something closer to an attention. Give the map five verbs and it comes alive. It senses what changed. It orients that change against the goals and risks already in the graph. It proposes options and the next path to walk. It acts, through me and through the agents I trust. And then it learns, by comparing what it expected against what actually happened and revising the edges accordingly. That last verb is the one most systems skip, and skipping it is what turns a living thing back into a filing cabinet. A confidence that never moves after the evidence arrives is not a belief. It is a decoration.

In practice this floor has a shape I have come to rely on. It runs on its own, does the sensing and the orienting and the proposing without me, and surfaces only the things that need a human: the broken edge, the goal that nothing is moving toward, the task moving toward no goal, the assumption that has gone stale. I review the exceptions, not the roster. That inversion, from watching everything to being shown only what changed and matters, is the whole return on building the floor, and it is the most direct thing I have built toward the objective function I keep coming back to: improve my orientation while reducing what it costs me to stay oriented.

Then the eighth floor, the one I mistook for the whole building. The loop is not another structure. It stores nothing the floors below it do not already store. It is the thing that keeps running, the pass that goes sense, orient, frame, model, decide, compose, act, observe, revise, and return to the top with attention pointed at whatever now deserves it. People compare this to the OODA loop and it is close, but OODA was built for a fighter pilot who has no time to update a causal model mid-turn. This loop has the time, because most of what I am trying to stay inside does not move at the speed of a dogfight. It moves at the speed of a quarter, a codebase, a body, a business, and the slower speed is exactly what makes the revision step both affordable and necessary. The loop is real. It was just never the foundation. It is the roof, and it only stands because there are eight floors under it holding it up.

The rule the whole stack exists to protect

If I had to throw away everything in the stack but one rule I would keep this one, because every other floor is quietly built to enforce it. There are three structures in the building that must never collapse into each other. There is the graph of what appears to be related out in the world. There is the graph of what the system currently believes those relations mean. And there is the single path it has decided to act on next. World, belief, action.

Almost every system that fails at this fails the same way. It lets them touch. It treats what it retrieved as true, then treats what is true as what it intends, then treats what it intends as permission to move. Each of those little promotions feels like efficiency, and each one is where the accident comes from. The stack refuses all three. A relationship that shows up in the world does not get to become a belief on its own. A belief does not get to become an action on its own. Something has to carry it across each gap on purpose, with the evidence attached, and where it matters with my hand on it.

That gap is where governance lives, and it is why I keep building this instead of buying a tool that would move faster by skipping it. The whole point of standing inside something complex without coming apart is that you get to keep what is out there, what you think it means, and what you are about to do about it as three separate things. Every floor of the stack is in service of holding those three apart long enough to choose. This is also the clearest example I have of why I think complexity is a thing to make visible rather than a thing to reduce: I did not simplify the system, I gave it floors, and the floors are what let me see it.

I started by typing my loops and thinking the loop was the top of the system. It is the top. The system is taller than I knew. It goes all the way down to a single relationship between two things, and every floor in between exists so that a relationship never turns into an action without someone deciding it should.