Skip to content

Edward Feigenbaum and the Expert System

Abstract

Edward Feigenbaum (born 1936) was in the room in January 1956 when Herbert Simon announced that he and Allen Newell had “invented a thinking machine,” and he spent the next sixty years arguing that the machine’s intelligence would come from what it knew rather than how it reasoned. DENDRAL, the program he began at Stanford with Joshua Lederberg in 1965 to read mass spectra, was the first expert system; his 1977 paper named the trade “knowledge engineering” and stated its creed, “in the knowledge is the power.” His 1983 book The Fifth Generation sold the Japanese AI programme to Washington as a threat. He shared the 1994 Turing Award with Raj Reddy and spent 1994 to 1997 as Chief Scientist of the US Air Force.

Edward Feigenbaum USAF
Edward Feigenbaum as Chief Scientist of the US Air Force, 1994–1997. Image: United States Air Force, public domain, via Wikimedia Commons.

Weehawken and Carnegie Tech

Edward Albert Feigenbaum was born on 20 January 1936 in Weehawken, New Jersey, across the Hudson from Manhattan. His stepfather, an accountant, took him to the Hayden Planetarium once a month and brought home a hand-cranked mechanical calculator, which the boy learned to run. At the Carnegie Institute of Technology he took a BS in 1956 and, in the first semester of his senior year, a course from Herbert Simon called Mathematical Models in the Social Sciences, with five graduate students and himself.

The course resumed after the Christmas break of 1955–56, and Simon walked in and told the class that over the holiday he and Allen Newell had invented a thinking machine. It was the Logic Theorist (Newell and Simon). Simon handed out an IBM manual, and Feigenbaum later described reading it as a “born-again” experience. He stayed at Carnegie for his PhD under Simon, finishing in 1960 with EPAM, the Elementary Perceiver and Memorizer, a program that modelled how people learn and forget lists of nonsense syllables and reproduced the errors human subjects made. With Julian Feldman he edited Computers and Thought (McGraw-Hill, 1963), the first anthology of AI papers and, for a decade, the field’s textbook.

Lederberg and DENDRAL

After five years teaching in Berkeley’s business school, Feigenbaum joined Stanford’s new computer science department in 1965 and ran its Computation Center until 1968. Before he arrived he had met Joshua Lederberg, the Nobel-winning geneticist, at a seminar on memory: Lederberg was relearning computers, writing ALGOL, and building an instrument package to fly to Mars and look for life by mass spectrometry of amino acids. Feigenbaum told him he wanted to model scientific induction. Lederberg said he had just the problem: given a mass spectrum, work out the structure of the molecule that produced it.

DENDRAL began as Lederberg’s algorithm for enumerating every acyclic molecule with a given formula, and became, with the chemist Carl Djerassi and Bruce Buchanan, a system that used chemists’ own rules of thumb to prune that enumeration to the handful of structures a real spectrum allowed. By 1970 it interpreted spectra at the level of a postdoctoral chemist, and its results appeared in chemistry journals rather than AI ones. A second program, Meta-DENDRAL, learned the mass-spectrometry rules from data. Its successor at Stanford, Edward Shortliffe’s MYCIN (1972–76), applied the same architecture to bacterial infections, and the two programs defined the genre (Expert Systems and the First AI Winter).

In the Knowledge Is the Power

The lesson Feigenbaum drew from DENDRAL was a reversal of the field’s founding assumption. Newell and Simon had bet on general problem-solving methods; DENDRAL worked because of what it knew about chemistry, and its inference engine was almost beside the point. He put it on record in 1977, in the paper that won the best-paper award at the International Joint Conference on Artificial Intelligence, “The Art of Artificial Intelligence: Themes and Case Studies of Knowledge Engineering”: “the problem solving power exhibited in an intelligent agent’s performance is primarily a consequence of the specialist’s knowledge employed by the agent, and only very secondarily related to the generality and power of the inference method employed. Our agents must be knowledge-rich, even if they are methods-poor.” And, three pages later, the sentence that was quoted for the next fifteen years: “in the knowledge is the power.”

The paper also named the job. When Feigenbaum described DENDRAL to Donald Michie in 1968, Michie called it “epistemological engineering,” which Feigenbaum simplified to knowledge engineering: a person who sits with an expert, extracts what the expert knows, and writes it down as rules a program can run. His Heuristic Programming Project, later the Knowledge Systems Laboratory, trained a generation of them, and the three-volume Handbook of Artificial Intelligence (1981–89) codified what they knew. In 1981 he co-founded two companies to sell the method, IntelliCorp and Teknowledge; the industry that followed, and its collapse at the end of the decade, is the AI winter of the article linked above.

The Fifth Generation

In 1983 Feigenbaum and Pamela McCorduck published The Fifth Generation: Artificial Intelligence and Japan’s Computer Challenge to the World (Addison-Wesley). Japan’s Ministry of International Trade and Industry had launched a ten-year programme in 1982 to build parallel inference machines running logic programs; the book presented it as a national challenge and argued that the United States would lose the “knowledge industry” unless it responded. The Microelectronics and Computer Technology Corporation, DARPA’s Strategic Computing Initiative and Britain’s Alvey programme all cited the Japanese threat, and the Japanese project itself spent just under ¥57 billion and produced no market (Dead End: Buying a Computer Industry). The book is remembered as the most effective piece of technological alarm of the decade, and as wrong about the technology, in both directions: the fifth generation never came, and neither did the American defeat.

Later Years

Feigenbaum was elected to the National Academy of Engineering in 1986. The ACM gave him and Raj Reddy of Carnegie Mellon the 1994 Turing Award “for pioneering the design and construction of large scale artificial intelligence systems, demonstrating the practical importance and potential commercial impact of artificial intelligence technology.” The same year he became Chief Scientist of the US Air Force, a post he held until 1997; he retired from Stanford as Kumagai Professor Emeritus in 2000, was made a Computer History Museum Fellow in 2012 and received the IEEE Computer Pioneer Award in 2013. He gave the museum three oral histories, one of them interviewed by Donald Knuth, and interviewed Knuth in return.

Dead End: The Knowledge That Would Not Scale

The knowledge principle was right and the knowledge-engineering method was not. DENDRAL and MYCIN worked because a single expert’s rules could be written down in a few hundred lines. Every attempt to go wider found that the rules multiplied, contradicted each other, and had to be rewritten by hand whenever the world changed; the field called this the knowledge acquisition bottleneck and never removed it. The bet that failed was not on knowledge but on people typing it in. When knowledge-rich systems finally arrived, in the large language models of the 2020s, they were knowledge-rich because they had read everything, and nobody had engineered any of it. Feigenbaum’s sentence survived the method it was written to promote.

📚 Sources