Skip to content

Jaan Tallinn

Abstract

Jaan Tallinn (born 1972) was one of the three Estonian programmers who wrote Kazaa and then Skype, and when eBay bought Skype for $2.6 billion in 2005 he became rich enough to spend the rest of his life on whatever he thought mattered. In 2009 he read Eliezer Yudkowsky on the risks of artificial intelligence and decided that was it. He was an early investor in DeepMind and led the Series A of Anthropic, co-founded the Centre for the Study of Existential Risk at Cambridge in 2012 and the Future of Life Institute in 2014, and has given more money to research on keeping AI safe than almost anyone. His position is the uncomfortable one of a man who funds the laboratories he is warning about, on the argument that they will be built anyway and had better be built by people who take the warning seriously.

Jaan Tallinn
Jaan Tallinn in 2014. Image: Christopher Michel, CC BY 2.0, via Wikimedia Commons.

Bluemoon

Tallinn was born in Tallinn, Soviet Estonia, on February 14, 1972, and was writing games before the country was independent: with his schoolfriends Ahti Heinla and Priit Kasesalu he founded Bluemoon Interactive, whose Kosmonaut (1989) was the first Estonian game sold abroad, for $5,000, and whose SkyRoads (1993) was played as shareware around the world. He took a degree in theoretical physics at the University of Tartu in 1996, with a thesis on interstellar travel by warping spacetime, and went back to programming.

In 2000 the three were hired by Niklas Zennstrรถm and Janus Friis to build Kazaa, the peer-to-peer file-sharing network that succeeded Napster, and its FastTrack protocol, in which the best-connected users’ machines became supernodes routing traffic for the rest. When the music industry’s lawsuits made the founders want a business with the same architecture and fewer subpoenas, the same team turned the network to voice. Skype was written in Tallinn and launched on August 29, 2003; Jaan Tallinn’s part was the peer-to-peer core. eBay bought the company in September 2005 for about $2.6 billion, and the Estonian engineers, who held a stake, sold. The product’s rise and long decline are in Dead End: Skype; the country’s story is in Estonia and Eastern Europe’s Tech Industry.

The Argument

Tallinn used the money as an investor, through Ambient Sound Investments with his Skype colleagues, and could have gone on doing so. In 2009 he read the essays of Eliezer Yudkowsky arguing that a machine intelligence exceeding human intelligence was likely within decades, that its goals would not automatically be ours, and that almost nobody was working on the problem. He flew to California to meet Yudkowsky, concluded he was right, and reorganised his life around it. He has said that the reasoning was the same as an engineer’s: if there is a non-trivial chance of an irreversible catastrophe and the work to reduce it is cheap, you do the work.

He became one of the first investors in DeepMind in 2011, when Demis Hassabis and Shane Legg were pitching a company to build general intelligence with safety as a stated goal, and stayed involved until Google bought it in 2014 (see DeepMind and AlphaGo). In 2012 he co-founded the Centre for the Study of Existential Risk at Cambridge with the astronomer Martin Rees and the philosopher Huw Price, funding it with an initial gift of about $200,000, and in 2014 the Future of Life Institute in Cambridge, Massachusetts, with Max Tegmark and others, which organised the 2015 Puerto Rico conference at which AI researchers first signed a joint letter on safety, and which in March 2023 published the open letter calling for a six-month pause on training systems more powerful than GPT-4, which Tallinn signed with 30,000 others. He has given more than $1.6 million to the Machine Intelligence Research Institute, Yudkowsky’s organisation, and runs the Survival and Flourishing Fund, which has distributed tens of millions of dollars to safety research through a mechanism he designed to have grantees and funders bargain over allocations.

Funding Both Sides

The contradiction in his position is one he states himself. He led the Series A investment in Anthropic in 2021, a $124 million round in the company founded by researchers who left OpenAI over safety; he has invested in other frontier laboratories. His argument is that the race is happening regardless, that a laboratory run by people who believe the risk is real will behave differently from one that does not, and that a seat at the table is worth more than a placard outside. Critics in the safety community reply that the money accelerates the thing it claims to slow. In May 2023 he signed the one-sentence statement organised by the Center for AI Safety that mitigating the risk of extinction from AI should be a global priority alongside pandemics and nuclear war; in interviews since he has put the odds of a bad outcome higher than most of the researchers he funds. The larger race, and the companies in it, are in The LLM Race and Anthropic and Claude.

๐Ÿ“š Sources