The Voice Assistant Revolution: Speaking to the Machine

Before Siri: The Long Road to Conversational Computing

The dream of speaking to computers is nearly as old as computers themselves.

In 1952, Bell Laboratories demonstrated Audrey — a system that could recognize spoken digits, useful for voice-dialing telephone numbers. In 1962, IBM showed Shoebox at the World’s Fair, a device that recognized sixteen spoken words, including the digits zero through nine. These were demonstrations of possibility, not products. The gap between recognizing a small vocabulary under laboratory conditions and understanding natural speech in real environments was enormous, and it would remain enormous for decades.

The Defense Advanced Research Projects Agency (DARPA) funded the first serious large-scale attempt to close that gap. Beginning in 1971, the Speech Understanding Research (SUR) program funded five years of research at Carnegie Mellon, MIT, BBN, and other institutions. The goal was a system that could understand a thousand-word vocabulary with reasonable accuracy. Carnegie Mellon’s Harpy system, completed in 1976, achieved the goal — it could recognize 1,011 words with 95 percent accuracy — but it required carefully controlled speech, quiet environments, and speakers who had trained the system to their voice. It was not a product.

The 1980s brought Hidden Markov Models (HMMs), a statistical approach to speech recognition that modeled speech as a sequence of probabilistic states. HMMs allowed systems to handle the variability of natural speech — the same word pronounced differently by different speakers, at different speeds, with different accents — without requiring explicit hand-coding of every variant. IBM’s Tangora system, built in the mid-1980s using HMMs, could recognize 20,000 words when the speaker paused between words.

Continuous speech recognition — understanding words without pauses — was the next major hurdle. Dragon Systems introduced DragonDictate in 1990, which required pauses between words, and Dragon NaturallySpeaking in 1997, which could transcribe continuous natural speech. NaturallySpeaking required training — users spent several hours reading text aloud so the system could learn their voice — and it ran on hardware that strained the PCs of 1997. But it worked. It was the first mass-market product that demonstrated continuous speech recognition was solvable, even if not yet convenient.

The missing piece was understanding. Speech recognition converted sound to text. Understanding text was a different problem — one that decades of natural language processing research had not solved. When a user dictated “schedule a meeting with John at three on Thursday” to Dragon NaturallySpeaking, they received a text transcription. Whether the computer would then actually schedule that meeting depended on whether a separate, integrated application could parse the intent from the text and take the appropriate action. In most cases, it could not.

SRI International and the Birth of Siri

The breakthrough that became Siri emerged not from Apple but from DARPA — and specifically from a DARPA research program called CALO, which stood for Cognitive Assistant that Learns and Organizes. Launched in 2003 with funding of $150 million over five years, CALO was the largest AI research project in American history at the time. It involved twenty-five universities and more than three hundred researchers, with Stanford Research Institute (SRI International) as the lead contractor.

CALO’s goal was an intelligent assistant for military personnel — a system that could help officers manage information, schedule activities, prioritize tasks, and coordinate with colleagues. The project produced significant research in machine learning, natural language processing, and multi-agent systems. It also produced a spin-off.

In 2007, a team led by Dag Kittlaus, Adam Cheyer, and Tom Gruber incorporated Siri Inc. as a commercial venture based on technology developed within CALO. The name came from a Norwegian word — Kittlaus was Norwegian-American — meaning “beautiful woman who leads you to victory.” The original Siri app, launched on the App Store in February 2010, was not a voice assistant in the way the term is now understood. It was a natural language interface for a range of services: restaurants, movies, taxis, events. A user could type or speak “find me a good Italian restaurant near Union Square that takes reservations for four tonight” and Siri would parse the request, query OpenTable, Yelp, and other services, and return integrated results.

Siri was on the App Store for two months.

Apple acquired Siri in April 2010 for a reported $200 million. Kittlaus, Cheyer, and Gruber joined Apple, and the app was removed from the App Store. Apple spent eighteen months integrating Siri into iOS, expanding its capabilities, and preparing it for the hardware launch that would define the product.

On October 4, 2011, Apple CEO Tim Cook — making his first major product announcement as Apple’s CEO, a day before Steve Jobs’s death on October 5 — introduced the iPhone 4S. The headline feature was Siri. Apple described it as a “humble personal assistant” and demonstrated it handling questions, setting reminders, sending messages, and answering queries about weather and restaurants. The demonstration was polished. The reaction was a mix of impressed and skeptical.

The skepticism was not unfounded. Siri in 2011 was inconsistent. Under controlled demonstration conditions, it was remarkable. In real-world use, with background noise, ambiguous phrasing, and requests that fell slightly outside its training, it often failed. Apple’s servers struggled under demand — Siri’s responses required server-side processing, and the volume of iPhone 4S users querying those servers simultaneously exceeded what Apple’s infrastructure had prepared for. Response times were slow. Requests failed.

More fundamentally, Siri revealed the gap between speech recognition and natural language understanding. Transcribing speech accurately was largely solved. Understanding what the transcribed speech meant — the intent, the entities, the relationships, the implied context — remained genuinely hard. Siri could answer “What’s the weather in San Francisco?” reliably because that query type had been explicitly handled. It could not reliably answer “What’s the weather like in the city where the 49ers play?” because doing so required chaining reasoning across multiple facts.

Amazon’s Bet: The Intelligent Speaker

Amazon’s approach to voice interfaces began from a different premise.

In 2010, Jeff Bezos convened a working group at Amazon to think about the company’s long-term product strategy. The group, sometimes called the Lab126 team, was exploring what the successor to the smartphone might be — what computing paradigm might eventually displace the touchscreen device that Apple had made dominant. One answer that emerged was voice: a device that understood natural speech and could operate without any screen at all.

The project that resulted — internally codenamed Doppler — took four years to develop. The engineering challenges were substantial. Unlike Siri, which operated in the relatively controlled acoustic environment of a phone held near the user’s face, the Doppler device would sit on a kitchen counter or bookshelf and need to understand speech from across a room, over background music, over conversations, over cooking noises. It needed to detect its wake word — “Alexa” — reliably without requiring the user to push a button.

Amazon solved the far-field audio problem with a seven-microphone array and sophisticated beamforming software that could isolate a voice and suppress background noise. The Amazon Echo launched in November 2014, initially by invitation only, then broadly in June 2015.

The Echo’s design was deliberately domestic. It was a cylindrical speaker, attractive enough to sit on a countertop, with no screen and no buttons required for normal operation. The experience was simple: the user spoke the wake word (“Alexa”), asked a question or gave a command, and the device responded. Alexa could play music, answer questions, set timers, read news briefings, control compatible smart home devices, add items to shopping lists, and order products from Amazon.

That last capability was not incidental. Amazon’s thesis — stated publicly by Bezos and confirmed by the company’s subsequent product investments — was that voice interfaces, properly executed, would increase purchasing. A user who could reorder paper towels by saying “Alexa, order more paper towels” was more likely to buy from Amazon than one who had to find a phone, open the Amazon app, search, and complete checkout. The Echo was partly a convenience product and partly a loyalty mechanism.

The Echo’s early sales were modest; Amazon has never disclosed exact figures. But by 2016, Amazon reported that Echo sales had exceeded one million units, and by 2017, the installed base was estimated at more than fifteen million. The product line expanded: the Echo Dot (a smaller, cheaper version), the Echo Show (adding a screen), the Echo Auto (for cars), and dozens of variants. Alexa became a platform: Amazon opened the Alexa Skills Kit, allowing third-party developers to build capabilities that Alexa could invoke. By 2020, Alexa had more than 100,000 skills.

The skills ecosystem revealed a limit. Having 100,000 skills was impressive as a statistic. As a user experience, it was overwhelming. Users could not remember skill names, could not easily discover new capabilities, and often did not know that a skill existed for a task they wanted to accomplish. The most-used Alexa functions remained the same across years: music playback, timers, weather, and smart home control. The long tail of skills was essentially invisible.

Google’s Advantage: Knowledge

Google’s entry into the voice assistant space came from a different direction.

The company had been building voice capabilities into its products for years before Siri. Google Voice Search on Android preceded Siri by several years. But Google’s strategic advantage was not speech recognition — it was knowledge.

Google had spent fifteen years indexing the web and building structured knowledge about the world. Its Knowledge Graph, announced in 2012, was a database of facts about entities — people, places, things, events — and the relationships between them. When a user searched for “Barack Obama,” Google did not just return pages about Obama; it displayed structured information: birth date, spouse, political party, books written, offices held. The Knowledge Graph was the result of processing billions of web pages and extracting structured facts from unstructured text.

This was precisely what a conversational assistant needed. Answering the question “Who is the president of France?” was not a search problem — it was a knowledge lookup problem. Google’s Knowledge Graph could answer it directly, without retrieving a web page and parsing it for the answer. Siri, in its early versions, relied heavily on web search for factual questions, which was slower and less reliable.

Google Now, launched in 2012 for Android, was Google’s first major voice-and-AI assistant product. It was distinct from Siri in its emphasis on proactive information: Google Now showed cards with information it predicted you would want — flight status based on your email, traffic conditions on your route to work, sports scores for teams you followed — before you asked for it. The underlying capability was the same Knowledge Graph combined with the user’s personal data (calendar, email, location history) to anticipate needs.

Google Assistant, launched in 2016 alongside the Google Home smart speaker and the Pixel smartphone, was the synthesis of Google Now’s proactivity, the Knowledge Graph’s factual depth, and improved conversational capabilities. Google Assistant was demonstrably better than competitors at answering factual questions, particularly questions requiring multi-step reasoning across facts. A query like “What is the capital of the country with the highest population?” — a query requiring knowing which country had the highest population (China), then knowing its capital (Beijing) — was a challenge for Siri and Alexa that Google Assistant handled routinely.

Google Home, the smart speaker launched in October 2016, was a direct response to Amazon Echo. It offered similar hardware and similar primary use cases — music, questions, smart home control — with the addition of Google’s knowledge advantage and tighter integration with Google services: Calendar, Gmail, Maps, YouTube.

The Plateau

By 2018, voice assistants were everywhere: in hundreds of millions of smartphones, in tens of millions of smart speakers, in cars, in televisions, in laptops. Siri, Alexa, and Google Assistant had become familiar features. Usage data told a consistent story about what users actually did with them.

Timers. Music. Weather. Simple searches. Smart home control.

The sophisticated conversational capabilities — multi-turn dialogue, complex task completion, integration with third-party services through Siri Shortcuts or Alexa Skills — were used by a small fraction of users, infrequently. The “assistant” framing — the implication that these products could manage your life, answer complex questions, and handle sophisticated tasks — consistently exceeded the reality.

Several structural factors drove the plateau.

Accuracy on anything non-trivial was too low. Voice interfaces had approximately zero tolerance for error in task completion. If a user asked Alexa to play a specific album and it played a different album, the user’s experience was poor. If a visual search returned somewhat wrong results, the user could correct by scanning and selecting. Voice failures required reformulation, repetition, or abandonment. The threshold for “good enough” was higher for voice than for visual interfaces.

Context was shallow. Most voice assistants could not maintain coherent context across more than a few conversational turns. A user who asked “What movies are playing near me?” and then “Which one has the best reviews?” and then “Book me two tickets for the eight o’clock showing” was attempting a three-turn conversation. Most assistants handled the first question, sometimes the second, and reliably failed the third unless a specific skill for the specific theater chain had been explicitly configured.

Privacy concerns were real. Amazon, Google, and Apple all employed human reviewers who listened to voice recordings to improve their models — a practice that became public knowledge in 2019 through reporting by Bloomberg and others. The revelation that smart speakers were recording ambient audio and that human employees listened to clips raised concerns that many users found difficult to dismiss. Trust in always-listening devices declined in surveys conducted after the revelations.

The skill/shortcut model didn’t scale. Both Amazon’s Skills and Apple’s Siri Shortcuts required users to set up integrations explicitly — discovering that a skill existed, enabling it, learning its invocation phrase. This friction proved fatal to adoption. Users who managed to configure five or ten skills rarely configured more. The vision of a comprehensive voice interface to all digital services ran into the practical limit of human patience for configuration.

Dead End: Microsoft Cortana

No voice assistant’s trajectory illustrated the difficulty of the market more clearly than Microsoft’s Cortana.

Announced in April 2014 at Microsoft’s Build developer conference, Cortana was Microsoft’s most aggressive attempt to establish a major consumer AI product since the company had missed the smartphone revolution. Named after the AI character from the Halo video game series, Cortana was initially positioned as a comprehensive personal assistant deeply integrated into Windows 10 — the operating system that Microsoft was planning to ship to one billion devices by 2018.

The launch was genuinely impressive. Cortana integrated with the Windows 10 search bar, offering voice and text queries that could search the web, control the PC, manage calendar entries, and answer factual questions. Microsoft’s investment in Bing’s knowledge graph — built over years to compete with Google — gave Cortana reasonable factual accuracy. The integration with Microsoft services (Outlook, Office 365, OneNote) offered a coherent assistant for enterprise users already in the Microsoft ecosystem.

Microsoft’s mobile strategy was the first strategic failure. Cortana was available on Windows Phone, iOS, and Android — but Windows Phone’s market share was collapsing. On iOS and Android, Cortana was a third-party application competing with the first-party assistants built into the operating system. Siri and Google Assistant had privileged access to device hardware and system services that Cortana could not match. A user on an iPhone who asked Cortana a question received a slower, less integrated experience than the same query to Siri. There was no compelling reason to use Cortana on a non-Windows device.

Microsoft attempted to differentiate Cortana with cross-device continuity — the ability to start a task on a Windows PC and continue it on a phone, or to receive PC notifications on a mobile device. The capability was genuine but required significant setup and worked primarily for users who invested in the Microsoft ecosystem across devices. In practice, few users did.

The second strategic failure was the enterprise pivot. Faced with declining consumer adoption, Microsoft reoriented Cortana toward enterprise productivity use cases — scheduling meetings, summarizing emails, providing briefings before calls. This was a defensible niche, but it required abandoning the broad consumer vision and accepting a more limited positioning.

The reorientation accelerated after 2019, when Microsoft reorganized Cortana away from consumer features. Cortana was removed from several international markets where it had been available. The standalone Cortana mobile apps for iOS and Android were discontinued in January 2021. Integration with Windows 10’s search — Cortana’s most prominent feature — was progressively diminished. By 2023, Cortana had been effectively retired as a consumer product.

Microsoft’s response to the AI moment of 2023 was not to revive Cortana but to abandon the brand entirely and launch Microsoft Copilot — an assistant built on OpenAI’s GPT-4 — into the same Windows integration points that Cortana had occupied. Copilot was what Cortana had aspired to be, enabled by a generation of AI progress that made 2014’s ambitions achievable. The lesson Microsoft drew from Cortana’s failure was not that the category was wrong but that the technology had not yet caught up to the vision.

The LLM Turn

The arrival of large language models — and specifically ChatGPT in November 2022 — changed the voice assistant category fundamentally, even if the change arrived through a different interface than voice.

The limitation that had constrained voice assistants for a decade — the shallow context, the inability to handle novel queries, the failure to maintain coherent dialogue across multiple turns — was precisely what LLMs addressed. GPT-4 could maintain context across thousands of words of conversation. It could handle novel queries, reason across facts, generate text, write code, and produce analyses. The conversational capability that Siri’s original demos had promised and never delivered was now technically achievable.

Apple’s response was Apple Intelligence, announced in 2024, which promised a substantially rebuilt Siri based on LLM technology and integration with ChatGPT for queries requiring broader knowledge. Amazon announced a rebuilt Alexa based on its own large language model. Google’s Assistant was progressively replaced or supplemented by Gemini, Google’s LLM family.

The integration of LLMs into voice assistants raised a question the field had not previously confronted: if the underlying model could handle any question, was the voice interface actually necessary? ChatGPT’s explosive adoption — 100 million users in two months, faster than any consumer application in history — occurred primarily through a text interface. Users preferred to type. The voice interface added friction without adding capability for many tasks.

The answer the industry arrived at was that voice and LLMs served different moments: voice for hands-free, screen-free contexts; LLMs (via text or voice) for contexts where the user wanted extended, thoughtful, iterative conversation. These were not the same moment, and the same product need not serve both.

The Niche That Mattered

The voice assistant did not replace the screen. It did not become the universal computing interface that early predictions imagined. But it did succeed in the niches where it had genuine advantages.

Driving was the clearest success. Voice interfaces for navigation, music control, calls, and messages in cars reduced the distraction of interacting with a screen while driving. Automotive voice integration — initially through Apple CarPlay and Android Auto, later through native automotive assistants — became standard equipment. Users who would never issue a voice command at their desk did so routinely in their cars.

Smart home control was a genuine success. Controlling lights, thermostats, locks, plugs, and appliances by voice when hands were full, in the dark, or from across the room represented a real improvement over opening an app, navigating to a control, and adjusting a slider. Amazon Echo and Google Home sold hundreds of millions of units because they made smart home control — a category that had been cumbersome and technically complex — accessible to mainstream users.

Timers and reminders remained the most-used voice assistant function for a reason. These tasks were simple, deterministic, and genuinely faster by voice than by any alternative. “Alexa, set a fifteen-minute timer” required no screen interaction and no hands. It worked.

The voice assistant revolution, in the end, produced not a revolution but an expansion: a new input modality that was genuinely superior in specific contexts and genuinely inferior in most others. The aspiration was HAL 9000 — an AI that understood everything and could do anything by voice. The product was a very good kitchen timer with a knowledge graph attached.

For the AI systems that eventually made conversational intelligence genuinely possible, see The Natural Language Processing Revolution and The Rise of Artificial Intelligence. For the smart home ecosystem into which voice assistants became embedded, see The Open Hardware Movement.

---

📚 Sources

• Levy, Steven: “The Perfect Thing” — Simon & Schuster (2006) — on Apple and the music device era preceding Siri
• Siri — Wikipedia
• Stone, Brad: “Amazon Unbound” — Simon & Schuster (2021) — Amazon Echo development backstory
• Vogt, PJ: “Alexa, are you listening?” — Reply All podcast, Gimlet Media (November 2018)
• Vincent, James: “Amazon’s Alexa has 100,000 skills — most are useless” — The Verge (September 2019)
• Day, Matt et al.: “Amazon Workers Are Listening to What You Tell Alexa” — Bloomberg (April 2019)
• Google Nest (smart speakers) — Wikipedia
• Cortana — Wikipedia
• Virtual assistant — Wikipedia
• Gruber, John: “Siri, twelve years in” — Daring Fireball (2023)
• Speech recognition — Wikipedia
• Warren, Tom: “Microsoft’s Cortana is being shut down in more regions” — The Verge (July 2021)