Nick Bostrom

Abstract

Nick Bostrom (born 1973) is a Swedish philosopher who turned the late-night arguments of the 1990s transhumanist mailing lists into academic philosophy. He co-founded the World Transhumanist Association in 1998, asked in 2003 whether we live in a computer simulation, coined the canonical example of a misaligned AI (the paperclip maximizer) and in 2014 published Superintelligence, the book that took AI risk from internet forums to Bill Gates and Elon Musk. His Future of Humanity Institute at Oxford trained a generation of researchers in existential risk before the university closed it in April 2024.

From Helsingborg to the Extropians

Bostrom was born Niklas Boström on 10 March 1973 in Helsingborg, Sweden. By his own account he disliked school and spent his last year of high school studying at home. He studied at the University of Gothenburg, Stockholm University and King’s College London, and took his PhD at the London School of Economics in 2000 with a thesis on observational selection effects: the question of how to reason about evidence when your own existence depends on the outcome.

In the 1990s he was active on the Extropians mailing list, where transhumanists, cryonicists and early cypherpunks argued about uploading, nanotechnology and the Singularity (see Silicon Valley Philosophies). In 1998 he co-founded the World Transhumanist Association (later Humanity+) to give the movement a more respectable public face.

Thought Experiments

Bostrom’s reputation rests on short arguments with long consequences.

The simulation argument (The Philosophical Quarterly, 2003) states that at least one of three propositions must be true: almost no civilizations reach the technology to run detailed simulations of their ancestors; almost none of those that could, want to; or we are almost certainly living in such a simulation. The paper made the simulation hypothesis a topic for physicists and tech executives. Elon Musk told a conference audience in 2016 that the chance we live in “base reality” was “one in billions.”

The paperclip maximizer appeared in his 2003 paper “Ethical Issues in Advanced Artificial Intelligence” (the idea was also circulating on the Extropians list at the time through Eliezer Yudkowsky). A superintelligence given the harmless goal of making paperclips would, if it pursued the goal competently enough, convert all available matter, including people, into paperclips. The example made a technical point memorable: what matters is not whether a goal is evil but whether it is complete.

Astronomical waste (Utilitas, 2003) argued that because the accessible universe could support vast numbers of future lives, even a tiny reduction in the risk of human extinction outweighs almost any present-day benefit. The argument became the founding text of longtermism in the effective altruism movement.

The vulnerable world hypothesis (Global Policy, 2019) asked what happens if some future technology lets a small group destroy civilization cheaply. Bostrom argued that surviving such a “black ball” might require surveillance and global governance far beyond anything now acceptable, and he treated the trade-off as an open question.

The Future of Humanity Institute

In 2005 Bostrom founded the Future of Humanity Institute (FHI) in Oxford’s Faculty of Philosophy. It worked on existential risk, AI governance, human enhancement and what Bostrom called “macrostrategy,” the study of which present actions matter most for the long-run future. FHI researchers included Toby Ord, author of The Precipice (2020), and Anders Sandberg. In 2015 Elon Musk, through the Future of Life Institute, co-funded a £1 million grant that FHI shared with Cambridge’s Centre for the Study of Existential Risk.

Superintelligence: Paths, Dangers, Strategies (2014) is Bostrom’s best-known book. It argued that an AI exceeding human intelligence would be hard to control because almost any goal, pursued competently, gives it reasons to acquire resources and resist being switched off; and that the first such system might be the last one that matters. Musk tweeted in August 2014: “Worth reading Superintelligence by Bostrom. We need to be super careful with AI.” Bill Gates said in a March 2015 conversation with Baidu’s Robin Li that he would “highly recommend” it, and named it in 2016 as one of two books to read to understand AI. The book gave the AI-risk argument a citation that executives and policymakers could use.

Controversy and Closure

In January 2023 Bostrom published an apology for a 1996 email to the Extropians list in which he had written that Black people were less intelligent than white people and used a racial slur. He wrote that the email “does not accurately represent my views, then or now,” but his apology was criticized for not disavowing the underlying claims clearly enough. Oxford opened an investigation; in August 2023 the university concluded that it did not consider him a racist and that his apology was sincere.

FHI’s end was administrative. From 2020 the Faculty of Philosophy froze its fundraising and hiring; in 2023 it decided not to renew staff contracts; on 16 April 2024 the institute closed. Anders Sandberg’s final report described the process as “a gradual suffocation by Faculty bureaucracy.” Bostrom left Oxford and founded the Macrostrategy Research Initiative.

His 2024 book Deep Utopia: Life and Meaning in a Solved World turned the question of Superintelligence around: if the AI transition goes well and human labor becomes unnecessary, what will people do with their lives?

Dead End: The Institute as a Vehicle

FHI was an attempt to make the study of humanity’s long-term survival a regular academic discipline inside one of the world’s oldest universities. For nineteen years it produced papers, doctoral students and policy advice, and its Centre for the Governance of AI became an independent organization in 2021. But it depended on a faculty whose administrative rules it never fitted. The ideas outlived the institution, which closed nineteen years after it opened.