The History of AI, Part 3: Neurons on Paper, McCulloch, Pitts, and the Birth of Cybernetics

The History of AI, Part 3 of 24. In Part 2, Alan Turing defined computation and asked whether machines could think. Now we meet the people who asked a different question: what if the brain itself is a kind of computer? This is where neural networks, feedback, and information theory all begin.

By the early 1940s, two big ideas were sitting side by side, almost touching.

On one side: logic and computation. Boole had shown that reasoning can be written as algebra, and Turing had defined exactly what a computing machine is.

On the other side: neuroscience. Biologists had established that the brain is made of billions of individual cells called neurons, connected in vast networks, passing tiny electrical signals along their branches. A neuron receives signals, and if enough arrive, it "fires" and sends its own signal onward. That's about as much as anyone knew for sure.

Someone was bound to put them together and say: a neuron is a kind of logic gate, and the brain is a computing machine built from them. That someone turned out to be an odd partnership of a middle-aged psychiatrist and a homeless teenage prodigy.

This chapter is about the birth of the neural network idea, about a sprawling science called cybernetics that briefly promised to explain minds, machines, and societies in one framework, and about the quiet groundwork that made later AI possible. It's also, unusually for a history of technology, a story with a genuinely sad ending.

The psychiatrist and the runaway

Warren McCulloch (1898-1969) was a restless, poetic, charismatic American neurophysiologist and psychiatrist. He had studied philosophy and psychology and medicine, and he carried around a question that he said had obsessed him since youth: "What is a number, that a man may know it, and a man, that he may know a number?" In other words, how can a physical, biological thing, a brain, come to know abstract truths?

Walter Pitts (1923-1969) was a very different figure. Born into a rough, poor family in Detroit, he was a self-taught prodigy who, as the often-told story goes, escaped bullies as a boy by hiding in a library, where he read Bertrand Russell and Alfred North Whitehead's Principia Mathematica. He is said to have written to Russell with corrections, and Russell invited him to study in England. He couldn't go. Instead, at around fifteen, he left home for good and wandered into the University of Chicago, where he sat in on lectures by the philosopher Rudolf Carnap and impressed everyone he met, despite having no official student status and often nowhere to live.

Through a fellow student, Jerome Lettvin, Pitts met McCulloch. McCulloch, who had a large house and a large heart, essentially took the young man in. The two began working together on a way to use logic to describe the brain.

"A Logical Calculus of the Ideas Immanent in Nervous Activity"

In 1943 they published a paper with that mouthful of a title in the Bulletin of Mathematical Biophysics. It is one of the most important and least-read papers in the history of AI.

Here is their central move. They took the real, messy biological neuron and simplified it brutally into an abstract unit with these properties:

• It has several inputs, each either on (1) or off (0).
• Some inputs are excitatory (they push the neuron toward firing), and some are inhibitory (they veto it).
• The neuron has a threshold. If enough excitatory inputs are on, and no inhibitory input is on, the neuron fires (outputs 1). Otherwise it stays silent (outputs 0).

That's all. And from this tiny building block, McCulloch and Pitts proved something remarkable: networks of these units can compute any logical function. With the right wiring you can build AND, OR, and NOT, and since you can build any logic circuit from those three, you can build any logical computation. They also argued that, given a tape, such networks could in principle compute anything a Turing machine can.

Let's see how little code this takes. Here is the McCulloch-Pitts neuron:

def mp_neuron(inputs, weights, threshold): """Fire (1) if the weighted sum of inputs reaches the threshold.""" total = sum(i * w for i, w in zip(inputs, weights)) return 1 if total >= threshold else 0 # AND: both inputs must be on AND = lambda a, b: mp_neuron([a, b], [1, 1], threshold=2) # OR: at least one input must be on OR = lambda a, b: mp_neuron([a, b], [1, 1], threshold=1) # NOT: an inhibitory (negative) weight flips the signal NOT = lambda a: mp_neuron([a], [-1], threshold=0) for a in (0, 1): for b in (0, 1): print(f"{a} {b} -> AND={AND(a,b)} OR={OR(a,b)}") print("NOT 0 =", NOT(0), "| NOT 1 =", NOT(1)) 

Run it and you'll see the truth tables Boole would recognize. Real neurons are enormously more complicated than this, and McCulloch and Pitts knew it. But the paper's power isn't biological accuracy. It's the bridge. It said: whatever the brain does, if it can be described logically, then a network of simple threshold units can do it. Thinking might not be mystical. It might be wiring.

Wires and weights

Notice the two ingredients that appear in the code: weights and a threshold. In the 1943 model, the weights were fixed by hand. Later researchers would let the weights change, and that single modification, learning, would become the central theme of modern AI. A modern deep learning model has billions of weights, and nearly all of its "knowledge" is stored in them. But the core operation, "multiply inputs by weights, add them up, compare to a threshold," is still the McCulloch-Pitts neuron, with the hard threshold usually replaced by a smooth function.

There's one more thing worth noticing: a single McCulloch-Pitts unit can't compute everything. It can do AND and OR, but a single unit cannot compute XOR (exclusive or: "one or the other, but not both"). You need at least two layers of units to do it. That small fact will come back with a vengeance in Part 6, when it becomes the center of a famous and devastating critique of early neural networks.

Why it mattered

The paper's impact was less about what researchers did with it immediately and more about the frame it created:

• It made the brain a legitimate object of mathematical study.
• It linked neuroscience to computation, suggesting the mind is something a machine might replicate.
• It inspired John von Neumann, who cited the paper in his 1945 report describing the stored-program computer architecture and used its neuron-like notation to describe how computer components might work.

It also contained an idea that would take years to mature: networks with loops, where signals feed back into earlier units, could have memory. That insight prefigured recurrent neural networks.

Meeting Norbert Wiener

In 1943, Pitts, only 20 years old, moved to Cambridge, Massachusetts, drawn to MIT by one of the most formidable intellects in America: Norbert Wiener.

Norbert Wiener (right) examines the chess-playing automaton El Ajedrecista in 1951. Public domain, via Wikimedia Commons.

Wiener (1894-1964) was a former child prodigy who earned his PhD from Harvard at 18. A mathematician with wide-ranging curiosity and a reputation for absentmindedness (there are many stories of him wandering the halls of MIT lost in thought), he had spent the early war years on a very practical problem: anti-aircraft fire.

The prediction problem

To shoot down a fast-moving airplane, you can't aim at where it is. By the time your shell arrives, the plane will have moved. You must aim at where it will be. That requires predicting the pilot's path from recent observations, which is hard because the pilot is a human being trying to evade you.

Wiener, working with the engineer Julian Bigelow, developed mathematical methods for predicting such trajectories, treating the pilot's motion as a noisy signal and finding the best statistical guess about the future. But they noticed something deeper: the whole system, a gun, a radar, a human operator, and an airplane, behaved as a loop. The gun makes an observation, adjusts its aim, observes the result, and corrects. This is feedback.

In a famous 1943 paper, "Behavior, Purpose and Teleology," Wiener, Bigelow, and the physiologist Arturo Rosenblueth argued that goal-directed behavior, in both animals and machines, could be explained through feedback. A thermostat has a "goal" (a target temperature). A guided missile has a "goal." A person reaching for a cup has a "goal." In each case, the system compares its current state with the desired state and acts to reduce the difference.

That sounds obvious today, but in 1943 it was heretical. Scientists had avoided talk of "purpose" as unscientific, a leftover from Aristotle. Wiener showed that purposeful behavior could be explained mechanistically.

Cybernetics

In 1948, Wiener published a book titled Cybernetics: Or Control and Communication in the Animal and the Machine. He had coined the word from the Greek kybernetes, "steersman", the person who steers a ship, constantly correcting course in response to wind and waves. (The same Greek root gives us "govern" and "governor.")

The book is a strange mixture of dense mathematics and sweeping speculation, and, to the astonishment of its publisher, it became a bestseller. Its central claim was that control and communication are the same kind of thing across biology, machines, and even society. Organisms, thermostats, computers, and economies are all systems that process information and use feedback to regulate themselves.

For a few years, cybernetics was the hot interdisciplinary idea. It influenced psychology, engineering, biology, anthropology, and management theory, and it supplied a vocabulary (feedback, input/output, signal, noise, control) that we still use constantly.

The Macy Conferences

The movement's social hub was a series of ten invitation-only meetings, the Macy Conferences, held between 1946 and 1953 in New York. They were the sort of gathering that seems almost impossible today: mathematicians, neurophysiologists, psychologists, anthropologists (including Margaret Mead and Gregory Bateson), engineers, and philosophers all in one room, arguing late into the night, with McCulloch chairing.

They debated how brains compute, how language works, how communities regulate themselves, and whether machines could think. Many of the questions that structured the next 70 years of cognitive science and AI were first sharpened in those rooms.

Claude Shannon: a bit of information

One regular in these circles quietly changed the world twice before he turned 33.

Claude Shannon in a 1958 MIT portrait. Public domain, via Wikimedia Commons.

Claude Shannon (1916-2001) was a playful, inventive engineer who loved unicycles, juggling, and building odd gadgets. In his 1937 master's thesis at MIT, "A Symbolic Analysis of Relay and Switching Circuits," he noticed that the on/off behavior of electrical switches could be described using Boolean algebra, the same algebra Boole had invented in 1854 to describe logic. Series circuits correspond to AND, parallel circuits to OR. This turned circuit design from an art into a mathematical discipline, and laid the foundation for digital computers. It is often called the most important master's thesis of the century.

Then, in 1948, the same year as Wiener's book, Shannon published "A Mathematical Theory of Communication," founding information theory. He defined the bit as the unit of information and showed how to measure the information content of a message, how much a noisy channel can carry, and how to encode messages so errors can be corrected. Every text message, streamed video, and compressed file depends on his ideas.

Shannon cared about machine intelligence too. In 1950 he published "Programming a Computer for Playing Chess," outlining the basic strategy of searching a tree of possible moves and scoring positions, which became the foundation of computer chess. He also built Theseus, a small mechanical mouse that could learn to find its way through a maze, one of the first demonstrations of a machine that visibly learned from experience.

His mathematics also hides a connection that will matter greatly in modern AI. The measure of information, entropy, is the same quantity that today's language models are trained to minimize (in the form of cross-entropy loss) when they predict the next word. Shannon was already experimenting in 1948 with predicting English text one letter at a time using statistical models. In a very real sense, today's chatbots are descendants of those experiments.

Hebb: how brains learn

McCulloch and Pitts left a gap. Their neurons had fixed connections. But real brains learn. How?

In 1949, the Canadian psychologist Donald Hebb proposed a simple and influential answer in his book The Organization of Behavior. In his words:

When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A's efficiency, as one of the cells firing B, is increased.

It is usually summarized, a little loosely, as: "neurons that fire together, wire together." (That catchy phrase actually came later, from the neuroscientist Carla Shatz.) The idea is that connections between neurons strengthen when the neurons are active at the same time.

This is Hebbian learning, and it was the first plausible learning rule for artificial neural networks. A tiny demonstration:

import numpy as np # Learn an association between two patterns using Hebb's rule: # weight change = learning_rate * input * output patterns = [ (np.array([1, 0, 1]), np.array([1])), # input pattern -> "fire"  (np.array([0, 1, 0]), np.array([0])), # input pattern -> "silent" ] w = np.zeros(3) lr = 1.0 for x, y in patterns: w += lr * x * y # strengthen weights where input and output are both active  print("weights after learning:", w) # [1. 0. 1.] print("response to [1,0,1]:", int(w @ np.array([1, 0, 1]) >= 1)) # fires print("response to [0,1,0]:", int(w @ np.array([0, 1, 0]) >= 1)) # silent 

Pure Hebbian learning has problems. Weights can grow without limit, and it can only learn correlations, not mistakes. Later rules, such as the perceptron learning rule and backpropagation, which we'll meet in Parts 5, 6, and 9, are much more powerful. But Hebb's idea, that learning means adjusting the strengths of connections, is exactly what every modern neural network does during training.

The first neural network machine

The theory was ready; someone had to build one. In 1951, two Harvard graduate students, Marvin Minsky and Dean Edmonds, built the SNARC (Stochastic Neural Analog Reinforcement Calculator). It used about 3,000 vacuum tubes and 40 artificial "neurons", each with a motor-driven potentiometer to represent an adjustable connection strength, and it simulated a rat learning to find its way out of a maze. When the "rat" made a choice that led to success, the connections that were involved were strengthened.

It was crude and temperamental, and Minsky later said it was not much of a success. But it was probably the first physical neural-network learning machine ever built. The irony is rich: Minsky would become one of the most prominent critics of neural networks in the 1960s, as we'll see. First, he built one.

Machines that behave like animals

Cybernetics wasn't only abstract. Two British researchers built machines you could actually watch behaving.

A replica of one of W. Grey Walter's robotic tortoises. Image: Anders Sandberg from Oxford, UK, CC BY 2.0, via Wikimedia Commons.

W. Ross Ashby was a psychiatrist who built the Homeostat in 1948, a device made of four interconnected units that, when disturbed, kept adjusting themselves until they returned to a stable state. It demonstrated "ultrastability": a machine that could find equilibrium on its own, without being told how. Ashby's 1952 book Design for a Brain argued that adaptive behavior could be explained by systems that seek stability through feedback.

W. Grey Walter, a neurophysiologist in Bristol, built a pair of little three-wheeled robots in 1948 and 1949, nicknamed Elmer and Elsie and collectively classified as Machina speculatrix, the "speculative machine." Often called "tortoises," they had a light sensor, a touch sensor, and just two vacuum-tube "neurons." Yet their behavior seemed lifelike: they wandered around exploring, moved toward moderate light, backed away from bright light, and when their batteries ran low, headed back to their charging hutch. When Grey Walter put a light on one tortoise and set it in front of a mirror, it danced in front of its own reflection.

The tortoises are important to AI history for a deeper reason. They showed that complex-seeming behavior can emerge from very simple mechanisms interacting with the environment, with no central model of the world, no stored reasoning, no deliberate planning. This idea would be reborn in the 1980s in Rodney Brooks's robotics, which argued for "intelligence without representation."

What cybernetics got right, and what happened to it

Looking back, cybernetics seems to have been right about a surprising amount.

• Feedback is everywhere. Control theory is now a core part of engineering. Thermostats, cruise control, autopilots, and robot arms all depend on feedback loops.
• Information is a physical, measurable thing. Shannon's theory is the foundation of the digital age.
• Brains can be modeled with networks. The McCulloch-Pitts neuron is the great-grandparent of modern deep learning.
• Learning is adjusting connections. Hebb's idea persists.
• Behavior emerges from interaction. Complex systems, from ant colonies to the internet, reward the cybernetic view.
• Reinforcement through feedback. The notion of learning from reward and punishment is the heart of reinforcement learning, which we'll see in AlphaGo.

Yet cybernetics as a movement faded by the late 1950s and 1960s. Why?

1. It was too broad. When a theory claims to explain everything, from thermostats to politics, it risks explaining nothing in particular. Researchers wanted sharper tools and clearer funding categories.
2. The new computers took over. With stored-program digital computers arriving, many researchers found it more promising to write programs that manipulated symbols than to build analog feedback systems or networks of simple units.
3. Personalities and politics. Wiener was outspoken, a pacifist who refused to take military funding after the war and criticized the arms race, and he did not play the Washington game of grant-seeking. In a field where influence increasingly depended on defense and government money, that mattered.
4. A new name. When John McCarthy organized a summer workshop in 1956 to explore machine intelligence, he deliberately invented the term "artificial intelligence" in part to avoid associating the project with cybernetics and, as he later said, to avoid having to accept Wiener as a guru. Naming has consequences: the new label gave the symbolic, computer-centered approach its own banner, and cybernetics became a side current.

A tragedy

The partnership that started all this ended badly.

Pitts had thrived at MIT, working with Wiener and McCulloch, and Wiener was reportedly planning to supervise his PhD and treated him as one of his most gifted students. Then, in the early 1950s, something went wrong. According to accounts by later historians, Wiener's wife Margaret, who strongly disliked McCulloch's bohemian household, told Wiener that McCulloch's circle had corrupted their daughter. (The claim, by most accounts, was false.) Wiener angrily cut off all contact with McCulloch, Pitts, and their colleagues, and refused to explain why.

Pitts was devastated. He had idolized Wiener. Reports say he withdrew, stopped working, and burned much of his own unpublished research, including a dissertation he had been working on. He retreated into isolation and alcohol, declining prizes and refusing to publish. Warren McCulloch and Walter Pitts both died in 1969, Pitts at just 46, largely forgotten outside a small circle.

The story is a reminder that history of science is human history. The idea that launched neural networks came from two people who were, in different ways, outsiders, and the field they started would later honor the idea far more than the people.

Why this chapter matters for modern AI

If you've ever trained a neural network, you've inherited this chapter. Consider the lineage:

At the same time, these early thinkers differed on a point that would split the field in two, and that divides researchers to this day:

• The symbolic view. Intelligence is the manipulation of symbols according to rules, like logic and Turing machines.
• The connectionist view. Intelligence emerges from many simple units connected in networks and learning from experience.

McCulloch and Pitts, interestingly, belong to both camps. Their paper described neurons, yet it proved things about logic. For the next two decades, though, the symbolic camp would win the funding, the prestige, and the headlines. The connectionist camp would be pushed to the margins, and nearly declared dead, before staging one of the great comebacks in the history of science.

The 1940s at a glance

It helps to see how crowded this decade was. The same few years contain the founding documents of computing, information theory, neural networks, and cybernetics:

• 1936 to 1937: Turing defines computation (Part 2); Shannon applies Boolean algebra to relay circuits.
• 1943: McCulloch and Pitts publish their neuron model; Wiener, Bigelow, and Rosenblueth publish "Behavior, Purpose and Teleology."
• 1945: John von Neumann circulates the EDVAC report, describing the stored-program computer.
• 1946: ENIAC is unveiled, and the first Macy Conference meets.
• 1948: Wiener's Cybernetics and Shannon's "A Mathematical Theory of Communication" appear in the same year; Ashby builds the Homeostat; Grey Walter builds his first tortoises.
• 1949: Hebb publishes The Organization of Behavior.
• 1950 to 1951: Shannon writes about computer chess and builds Theseus; Turing publishes "Computing Machinery and Intelligence"; Minsky and Edmonds build SNARC.

Fewer than fifteen years separate Boole's algebra becoming a circuit-design tool from a machine learning to run a maze. That compression is part of why the early pioneers were so optimistic.

Three common misconceptions

"The McCulloch-Pitts neuron is a model of a real neuron." Not really, and its authors knew it. A real neuron has branching dendrites that compute in their own right, fires in timed spikes, and is modulated by dozens of chemicals. The 1943 unit is a deliberate cartoon, chosen because it can be analyzed with logic. Modern artificial neurons are "inspired by" biology in the way an airplane wing is inspired by a bird's: the principle is borrowed, the details are not.

"Cybernetics failed." As a movement with its own conferences and gurus, it faded. But its ideas were absorbed so completely that we no longer notice them. Control theory, robotics, systems biology, ecology, and cognitive science all use feedback, information, and regulation as basic concepts. The prefix "cyber-" in cybersecurity and cyberspace descends from Wiener's coinage. What failed was the label, not the substance.

"Hebb's rule is how brains learn, and backpropagation is how AI learns, so the two are basically the same." They are cousins, not twins. Hebb's rule is local: a connection changes based only on the activity of the two neurons it joins. Backpropagation needs error signals that flow backward through the whole network, which no one has found a clear biological equivalent for. Whether the brain approximates something like backpropagation by other means is an open research question, and Geoffrey Hinton, among others, has proposed alternatives to it, partly for that reason.

Quick glossary

• Threshold: the level of input a unit must reach before it fires.
• Weight: a number scaling how strongly one unit's output influences another.
• Feedback: using the result of an action to adjust the next action.
• Homeostasis: a system's tendency to restore a stable state after disturbance (Ashby's Homeostat, your body temperature).
• Entropy: in Shannon's sense, a measure of the uncertainty (or average information) in a message source.
• Bit: the amount of information in a single yes/no choice.

Key takeaways

• McCulloch and Pitts (1943) modeled the neuron as a simple threshold logic unit and showed that networks of such units can compute any logical function. This is the starting point of neural networks.
• Norbert Wiener founded cybernetics (1948), explaining goal-directed behavior in machines and animals through feedback.
• Claude Shannon applied Boolean algebra to circuits (1937) and created information theory (1948), giving us the bit and the mathematics later used to train language models.
• Donald Hebb (1949) proposed that learning strengthens connections between co-active neurons, the first plausible learning rule for artificial networks.
• Minsky and Edmonds built the first neural network machine (SNARC, 1951), while Grey Walter and Ashby built lifelike feedback machines.
• Cybernetics faded, in part because the new field of AI adopted a different name and a different, symbol-focused approach.

Try it yourself

1. Build XOR the hard way. A single McCulloch-Pitts neuron can't compute XOR. Combine three of the neurons from the code above, using AND, OR, and NOT, to build it: XOR(a, b) = AND(OR(a, b), NOT(AND(a, b))). You've just built a two-layer network by hand.
2. Stretch Hebb. Run the Hebbian learning example with more patterns, and see what happens when two patterns overlap. Why does Hebb's rule start making mistakes? What kind of rule would fix them?
3. Simulate a thermostat. Write a loop that reads a temperature, compares it with a target, and turns a heater on or off. Then add noise and delay and see how the system oscillates. You've just rediscovered the core problem of control theory.

Next up

In Part 4, we go to a small New Hampshire college campus in the summer of 1956, where a group of ambitious young researchers met for a workshop whose proposal contained a bold prediction: that "every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it." It was the moment the field got its name, artificial intelligence, and the moment its first great wave of optimism began.

If this chapter changed how you see neural networks, follow the series and leave a comment: which of these ideas surprised you most?

See you in Part 4.

☕ If this series helped you, you can support my writing on Ko-fi.