Skip to content

McCulloch and Pitts

Abstract

In December 1943 the psychiatrist Warren McCulloch (1898–1969) and the self-taught logician Walter Pitts (1923–1969) published “A Logical Calculus of the Ideas Immanent in Nervous Activity,” the first mathematical model of a network of neurons. Their idealised nerve cells either fired or did not, and they showed that networks of them could realise the operations of propositional logic. Six years later the Canadian psychologist Donald Hebb proposed how such connections might strengthen with use. Together the two ideas, a neuron that computes and a synapse that learns, underlie every artificial neural network since, from Rosenblatt’s perceptron to the large language models.

The Psychiatrist and the Runaway

Warren Sturgis McCulloch was born in Orange, New Jersey, on 16 November 1898. He studied philosophy and psychology at Yale, took a medical degree in New York in 1927 and worked in a large psychiatric hospital before joining Yale’s neurophysiology laboratory in 1934. In 1941 he moved to the University of Illinois in Chicago to teach psychiatry.

Walter Pitts was born in Detroit on 23 April 1923, the son of a boilermaker. He taught himself mathematics, logic, Latin and Greek, left home at fifteen and got by in Chicago, spending most of his time at the university. He was in contact with Bertrand Russell and Rudolf Carnap and attended the lectures of the Ukrainian-born biophysicist Nicolas Rashevsky, who in 1939 had founded the Bulletin of Mathematical Biophysics. Between September 1942 and March 1943, aged nineteen, Pitts published three papers on networks of nerve cells in Rashevsky’s journal.

McCulloch and Pitts met in 1942 and found that they shared an interest in the brain and an admiration for the logical program of Leibniz. McCulloch, who lived with his wife and children in a Chicago suburb, took the young man into his home.

A Logical Calculus

Their paper appeared in the December 1943 issue of the Bulletin, eighteen pages long, with a bibliography of three titles, all from mathematical logic. They modelled the neuron as it was understood at the time: a cell body whose axon passes signals through synapses to other cells. Time advanced in discrete steps. A neuron that fired at step t delivered its impulse to the next cells at step t + 1; a cell fired if it received enough excitatory signals to reach its threshold, and an inhibitory signal stopped it from firing whatever else arrived.

With these rules small networks became logic gates. A cell that fires when either of two inputs suffices to reach its threshold computes OR; one that needs both inputs computes AND; a cell excited by one neuron and inhibited by another computes “A and not B.” Since every cell was either firing or silent, the model was digital, and since networks of such cells could realise any formula of propositional logic, the brain could in principle be described as a logical machine. The paper came out of the circle that soon called itself cybernetics (see Norbert Wiener and Cybernetics).

Hebb’s Rule

The 1943 model had fixed connections; it could compute but not learn. Donald Hebb, born in Chester, Nova Scotia, on 22 July 1904, the child of two doctors, had studied literature and taught school before turning to psychology at McGill University, where he took a master’s degree in 1932; he took his doctorate at Harvard in 1936, studied chimpanzees at a research station in Florida from 1942, and was professor of psychology at McGill from 1947 to 1972.

In the autumn of 1949 he published The Organization of Behavior: A Neuropsychological Theory. On page 62 it stated a postulate: when an axon of cell A is near enough to excite cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells so that A’s efficiency in firing B is increased. Hebb offered it as a hypothesis and suggested new synaptic knobs as the mechanism. It broke with the belief that the adult brain was fixed and replaced it with the idea of neural plasticity, which experiments from the 1960s onward, Eric Kandel’s work on sea slugs among them, confirmed under the name long-term potentiation. Computer scientists took it up as a learning rule for artificial networks: when two connected units are active together, increase the weight of their connection, summed up as “neurons that fire together wire together.” Hebb died in Chester on 20 August 1985.

The HNF points out an earlier literary version. In his 1930 novel Das Automatenzeitalter the German chemical engineer Ludwig Dexheimer described robot brains built from “electroneurons” whose connections strengthened with use in just this way.

After 1943

Pitts moved to MIT at the end of 1943 to study and work with Norbert Wiener, and McCulloch joined MIT in 1952. In 1959 both were co-authors, with Jerome Lettvin and Humberto Maturana, of “What the Frog’s Eye Tells the Frog’s Brain,” which showed that the frog’s retina already processed images in ways the logical model did not capture. The result undermined Pitts’s own picture of the brain as a logic machine. In 1952 Wiener had abruptly broken with McCulloch and everyone around him, a rift that Wikipedia traces to false allegations by Wiener’s wife, and Pitts, who had depended on both men, never recovered from it. He burned the draft of his doctoral thesis on probabilistic three-dimensional neural networks, drank heavily, and died on 14 May 1969 of bleeding oesophageal varices. McCulloch, who had become one of the stars of cybernetics and whose essays were collected as Embodiments of Mind, died four months later, on 22 September 1969.

Neural networks went through two decades of neglect after Minsky and Papert’s critique of the perceptron in 1969, and returned with backpropagation and then deep learning (see John Hopfield and Neural Networks and Geoffrey Hinton and Deep Learning).

📚 Sources