Models: the math under the cybernetics

A matrix is a list of where each basic direction goes. Everything else is made of those directions, so the list is the whole story. Coordinates are a recipe: how much of each ingredient. The matrix swaps in new ingredients and keeps the recipe.

Coordinates come from Descartes and Fermat in the 1630s: geometry turned into algebra. Grids of coefficients are much older. The Chinese Nine Chapters on the Mathematical Art (compiled about two thousand years ago) solved systems of equations by arranging numbers in a grid and eliminating, the method we now name after Gauss.

Sylvester coined the word matrix in 1850. Cayley (1858), building on Gauss and Eisenstein, defined matrix multiplication as doing one substitution and then another, so the multiplication rule was derived from composition, exactly step 5 here. Nobody memorized it first; it was a consequence.

Every rotate, zoom and skew on a screen is a matrix acting on points. Robot arms chain one matrix per joint. A spreadsheet that multiplies quantities by prices is a matrix times a vector. Each layer of a neural network is one matrix.

Two coffees and a bagel, priced at two different shops. Your order is the vector (2, 1). Each shop's prices form a row. The bill at every shop at once is the matrix times your order.

Any time you say "everything follows from what happens to a few basic things", you are thinking in a basis.

A whole transformation is known from what it does to a few things. Ashby's transformation is the same idea without the straight-line condition: model 7 is his table. Linearity is what lets a matrix get away with storing only its columns.

î lands on (0, 1) and ĵ lands on (−1, 0). Where does (3, 2) go? Use the recipe, not a rule.

Adding a dimension adds an ingredient and nothing else. The rules you learned in the plane hold unchanged in three, or in seven hundred and eighty-four. Area becomes volume, and the idea stays the same.

The letters are Hamilton's. In 1843, looking for a way to multiply in three dimensions, he found the quaternions and carved their rule (i² = j² = k² = ijk = −1) into Brougham (now Broom) Bridge in Dublin. The carving is gone; a plaque marks the spot. He gave the astronomers' old word vector its modern meaning and coined scalar. Grassmann, in 1844, worked out spaces of any dimension. In the 1880s Gibbs and Heaviside pulled the useful part out of quaternions and gave physics the i, j, k unit vectors it still writes (the hats came later, from textbooks).

3D graphics and games, physics (forces, velocities), and colour: a pixel is a 3-vector of red, green and blue, and a camera's colour correction is a 3×3 matrix mixing them. A network layer reading a 28×28 image is a matrix with 784 columns: the same idea, just more columns.

A smoothie with three ingredients is a point in 3D. A recipe change that trades some banana for more berries is a transformation. When a photo looks too blue and you warm it, that's often a 3×3 matrix applied to every pixel.

Ashby works with systems of any number of variables: a state is a vector of readings, one per variable. The homeostat had four units, so its state lived in four dimensions. That's no stranger than your three, only harder to draw.

Make the third column equal to the first plus the second. What is the volume scale, and why?

One number for how much the transformation scales space, and whether it flips it. It's a gauge: big means stretched, small means squeezed, zero means a dimension was lost, negative means a mirror image.

Determinants are older than matrices. Seki Takakazu in Japan (1683) and Leibniz in Europe (1693) found them independently, while eliminating unknowns from equations. Cauchy (1812) gave them their modern meaning, and he and Binet proved that they multiply. The picture of area and volume makes all of it obvious, but it came later than the algebra. Here you get the picture first.

Does a system of equations have one answer? Check the determinant. In calculus, a curved map scales area differently at each spot, and the local determinant (the Jacobian) is how integrals change variables. Graphics uses the sign to tell which side of a line a point is on, or which way a triangle faces.

A world map: the Mercator projection's local area scale grows toward the poles, which is why Greenland looks the size of Africa. Two survey questions that really measure the same thing make a near-zero determinant (exactly zero only if they're perfectly redundant), so you can't separate their effects. Statisticians call that collinearity.

A zero determinant is a machine that forgets. Model 4 counts what is forgotten, and Ashby measures it as lost variety.

Without the formula: what is the determinant of [[3, 0], [0, 2]]? And of [[1, 2], [2, 4]]?

Variety is how many different things you can tell apart. A many-to-one step makes some of them indistinguishable, and nothing later can split them again. Information is the ability to make distinctions, and this is how it is lost.

Ashby defined variety in An Introduction to Cybernetics (1956) as the number of distinguishable elements, often counted in bits, following Shannon's information theory (1948). Linear algebra's version is the rank-nullity theorem: the dimension of what comes out (the rank) plus the dimension of what gets crushed to zero (the null space) equals the dimension you started with.

Lossy compression (JPEG and MP3 throw away what your senses miss), hashing, summary statistics, and privacy, where aggregation loses who is who on purpose.

An average, a grade, a dashboard number, a one-line summary: each collapses many situations into one reading. From a class average you can't get any one student's score back. That's fine, as long as you chose what to lose. A gauge is a deliberate collapse.

Ashby counts variety, and every many-to-one step reduces it. A regulator is a machine built to reduce variety on purpose, at the outcome, while keeping enough variety inside itself to do so. Models 7 and 8 show how.

A class average is 80. Can you tell who scored what? What would you need to keep to be able to?

The dot product asks how much one arrow points along another. A neuron asks the same question about its input and its weights, then decides: past the threshold it fires, otherwise it doesn't. The line where it changes its mind is a distinction drawn across the plane.

The dot product, in the notation we use, comes from Gibbs and Heaviside's vector analysis (1880s); Grassmann and Hamilton had it in other forms. McCulloch and Pitts (1943, the same year as the Rosenblueth, Wiener and Bigelow paper) showed that units like this, with weighted inputs and a threshold, can compute logic. Rosenblatt's perceptron (1958) learned the weights from examples. Minsky and Papert's Perceptrons (1969) proved hard limits on single-layer networks (XOR is the simplest case, parity the general one), and funding and interest moved elsewhere for a time; how much the book caused that is still argued. Backpropagation, invented several times and made widely known by Rumelhart, Hinton and Williams (1986), trained multi-layer networks past the limit.

Every neuron in every network. Search by meaning: two pieces of text become vectors, and the dot product of the two, scaled to length 1 (cosine similarity), says how alike they are. Recommendation systems and spam filters work the same way.

A weighted decision. Score a job offer: salary counts heavily, commute somewhat, title a little. That's a dot product of weights with features, and the threshold is where you'd say yes. A course grade with weighted categories is one too.

The 1943 neuron is where cybernetics and computing share a root. McCulloch went on to chair the Macy conferences. A single neuron draws one distinction, a straight line. Spencer-Brown's Laws of Form begins from the same act: draw a distinction.

Weights (1, 1), threshold 1.5. Which of the four corners fire? What logic is that?

The arrow is one thing, and its coordinates are a description that depends on who is describing it. Change the standpoint and the numbers change while the arrow doesn't. To act in someone else's language: translate in, act, translate back. That's P⁻¹AP.

The question of what stays true when the observer's frame changes runs through physics, from Galileo's ship to Einstein's relativity. In linear algebra it became similarity: two matrices that are the same transformation seen from two bases. Segner (1755) and Euler (1758 to 1765) found that every rigid body has principal axes, a basis in which its motion is simplest. That basis is made of eigenvectors.

Maps and GPS convert between coordinate systems constantly. Graphics moves between world, camera and screen space. Principal component analysis turns data onto the axes it varies along most. Diagonalizing (finding the basis where a matrix only stretches) makes repeated application easy, which is model 9's subject.

A recipe in cups when you think in grams: convert in, cook, convert back. Two colleagues describe the same project in different vocabularies, and both are accurate. Before arguing about the numbers, check whose basis they're in.

This is the second-order move in miniature: the description includes the describer. The vector doesn't need an observer, but its coordinates do.

Her basis is b₁ = (2, 1) and b₂ = (−1, 1). She says a point is at (1, 1) in her coordinates. Where is it in yours?

A machine is a table of "this state becomes that one". Run it and you see where things end up. Stability is where the arrows lead and stay, not stillness. Every path in a finite table ends in a loop.

Ashby, Design for a Brain (1952) and An Introduction to Cybernetics (1956), where the drawing is called a kinematic graph. The same object shows up as Markov's chains (1906) when the arrows carry probabilities, and as Turing's machines (1936) when a tape is added. Ashby's homeostat (1948) was a physical machine that searched its own table until it found a stable one.

Software state machines: an order goes placed, then paid, then shipped. A user interface's screens. Network protocols. Give the arrows probabilities and you get a Markov chain: Google's original PageRank (Brin and Page, 1998) treated the web as one, and ranked pages by where a random surfer ends up.

Draw your evenings as states: tired, phone, couch, walk, desk, bed. Where do the arrows lead? A loop you keep falling into is an attractor, a habit. Changing one arrow (the phone charges in the kitchen) moves the attractor more cheaply than willpower spent on every state.

Ashby builds equilibrium, stability and adaptation from this one drawing. When arrows merge, variety drops (model 4 again). A regulator chooses its own arrows so the system's paths end in the states it wants.

If every state points to d, what is the variety after one step? What kind of machine is that?

To hold outcomes steady against a varied world, you need about as many kinds of response as the world has kinds of disturbance. You can't beat this by being clever, because it's counting, as long as the world itself doesn't absorb some of the variety first (a wall, a buffer, a filter). As Stafford Beer put it, only variety can absorb variety.

Ashby, An Introduction to Cybernetics (1956), chapter 11. He tied it to Shannon's tenth theorem (1948), which says you can only correct noise with a correction channel of enough capacity. Stafford Beer applied it to management from the late 1950s; in Brain of the Firm (1972) the moves are attenuate the variety coming in, amplify the variety going out. Conant and Ashby (1970) argued that the simplest regulator that does best must be a model of what it regulates.

Security (many kinds of attack against a set of defences), operations (runbooks, on-call), support desks, immune systems, thermostats, traffic control, any governor.

When overwhelmed, there are only three moves:

1. More responses: templates, tools, agents, a second person.
2. Fewer disturbances: filters, routines, standards, saying no.
3. Accept more outcome variety where it doesn't matter: let some things be different each time.

An inbox is a disturbance stream. Someone with one response for everything will see everything go wrong.

Playing your best meant picking, for each row, the column that cancels it. That rule is a model of the disturbances, and the best simple regulator has to carry one: that's Conant and Ashby's theorem.

12 kinds of disturbance, 4 kinds of response. How few outcomes could you possibly hold it to?

Run a process on its own output and see what comes back unchanged. That's a fixed point. For a matrix, an eigenvector is a direction that stays on its own line, only scaled. Whether it settles depends on one thing: does each pass shrink the difference (slope under 1 in size) or grow it?

The averaging method for square roots is very old (try the √2 option): guess, average the guess with 2 ÷ guess, and repeat. Heron of Alexandria wrote it down in the first century. Babylonian tablets show √2 to six places far earlier, though how the scribes got it isn't recorded, so the old name "Babylonian method" is a tradition, not a fact. The principal axes of a spinning body (Segner 1755, then Euler) were eigenvectors before the word existed. Hilbert (1904) used the German eigen, meaning own. Banach (1922) proved that a map which shrinks every distance by at least a fixed factor below 1 (on a complete space, such as the number line) has exactly one fixed point, and that repeating it from any start finds that point. Von Mises and Pollaczek-Geiringer (1929) described the matrix version you see here, now called power iteration.

PageRank is the main eigenvector of the web's link table, adjusted so the surfer sometimes jumps at random. Vibration modes, such as a bridge's sway or a guitar string's harmonics, are eigenvectors. The long-run state of a well-mixed Markov chain (one that can reach every state and doesn't cycle) is its eigenvector with λ = 1. Principal component analysis, quantum mechanics, and population models all use them.

A microphone near its speaker squeals because the loop gain is over 1: each pass makes the sound louder. Turn it down below 1 and the loop calms. Reputations, habits and moods settle into whatever reproduces itself. Asking "what keeps regenerating this?" is often more useful than asking what caused it.

Von Foerster's essay title, "Objects: tokens for (eigen-)behaviors", names it: what we call a stable thing may be what a process settles on when it keeps acting on its own results. Kauffman's eigenform is that staircase, applied to how an observer sees. A person's steadiness can be read the same way, as a fixed point of their routines.

What is the fixed point of x → x/2 + 1? Does the staircase settle there?