Dead End: The Semantic Web

Zusammenfassung

In May 2001, Tim Berners-Lee published a vision in Scientific American for the next phase of the World Wide Web: a “Semantic Web” in which information would carry machine-readable meaning, enabling software agents to navigate it intelligently on behalf of humans. The article described a future where a medical agent could locate, schedule, and coordinate appointments autonomously, reasoning across health records, calendars, and insurance databases that exposed their data in standardized ontological formats. Two decades and billions of dollars in academic and industrial research later, the Semantic Web’s core technologies — RDF, OWL, SPARQL — are largely confined to academic research groups, pharmaceutical knowledge management systems, and the infrastructure underlying Wikidata. The vision of a machine-readable web of data as the peer of the human-readable web of documents never arrived. What succeeded instead was a combination of search engine heuristics, structured markup schemas designed for commercial benefit, and, ultimately, large language models that achieved machine understanding through statistical pattern recognition rather than explicit semantic representation. The Semantic Web failed not because its technology was wrong, but because its economic and social assumptions were.

The 1994 Web and Its Limits

The World Wide Web that Tim Berners-Lee built at CERN between 1989 and 1991 was a system of linked documents. HTML encoded presentation and structure for human readers. Hyperlinks connected documents to other documents. The system’s power was its simplicity: anyone could publish a page, anyone could link to it, and a browser could render any page on any computer. The web’s meaning lived in the text that humans read.

By the mid-1990s, the web was growing faster than anyone had anticipated, and its limitations were becoming visible. Search engines — AltaVista (1995), HotBot (1996), Google (1998) — were building indexes by crawling HTML and extracting keywords from text. This worked well enough to be useful. It worked badly when precision mattered: a search for “bass” returned pages about guitars, fish, and speakers without distinguishing them; a search for “jaguar” conflated the car and the animal. The web knew that the word “bass” appeared on a page. It did not know what “bass” meant.

Berners-Lee had been thinking about this problem since the early days of the web. His proposed solution was not to build smarter search engines that extracted meaning from text — the approach that would eventually produce Google’s Knowledge Graph and, ultimately, the large language models — but to extend the web’s infrastructure so that meaning was expressed explicitly, in machine-readable formats, by the people who published information.

The key insight was borrowed from knowledge representation in artificial intelligence: if you could express the relationships between concepts formally — that a person has a name, that a medication treats a condition, that a museum is located in a city — then software could reason across those relationships. The web would become, in Berners-Lee’s phrase, not just a web of documents but a web of data.

The Semantic Web Stack

The W3C, under Berners-Lee’s direction, began standardizing the Semantic Web’s component technologies in the late 1990s.

RDF (Resource Description Framework), first published as a W3C Recommendation in 1999, was the foundation. RDF expressed information as triples: a subject, a predicate, and an object. “Paris — is the capital of — France” becomes an RDF triple. Resources are identified by URIs, making them globally unambiguous: the “Paris” in this triple is not Paris, Texas. A collection of RDF triples forms a graph — a network of interconnected facts — rather than a document or a table.

RDFS (RDF Schema) added a vocabulary for expressing basic relationships between classes: that a Dog is a subclass of Animal, that hasOwner is a property that expects a Person as its value. RDFS enabled rudimentary inference: if you know that Fido is a Dog, and you know that all Dogs are Animals, you can infer that Fido is an Animal.

OWL (Web Ontology Language), finalized as a W3C Recommendation in 2004, extended RDFS with the formal machinery of description logics — a well-studied area of mathematical logic. OWL allowed ontology designers to express cardinality constraints, disjointness (a Student is either Undergraduate or Graduate, not both), and property chains with mathematical precision. OWL ontologies could be fed to automated reasoners — software that could derive conclusions not explicitly stated, detect logical contradictions, and answer queries about inferred knowledge.

SPARQL (SPARQL Protocol and RDF Query Language, W3C Recommendation 2008) was SQL for RDF graphs: a query language that could retrieve structured data from RDF stores, joining across multiple data sources and filtering by complex conditions.

The complete vision — the Semantic Web Stack — placed these technologies in layers, with RDF as the universal data model, OWL as the ontology language, SPARQL as the query interface, and an envisioned Proof and Trust layer on top that would allow agents to evaluate the provenance and reliability of information. The higher layers never reached the standardization stage.

Info

The 2001 Scientific American paper’s scenario: Berners-Lee, Hendler, and Lassila illustrated the Semantic Web vision with a story about “Pete and Lucy,” siblings trying to schedule physical therapy for their mother. They imagined agents that could navigate health records, insurance databases, and provider scheduling systems, automatically finding an appointment slot that satisfied all constraints. The agents communicated using shared ontologies that allowed a health record system and a scheduling application, built by different companies, to exchange meaning without prior coordination. The scenario was technically coherent. It assumed that every stakeholder in the story had deployed Semantic Web infrastructure and made their data available through it.

What Got Built

Academic and research investment in the Semantic Web was substantial throughout the 2000s. Several genuine achievements accumulated.

DBpedia (2007) extracted structured information from Wikipedia’s infoboxes into RDF format, making Wikipedia’s factual content queryable via SPARQL. For the first time, one could ask “list all cities with a population over one million in countries that border France” against a real dataset and receive a structured answer. DBpedia became a cornerstone of the Linked Open Data project, a Berners-Lee initiative (2006) that encouraged data publishers to expose their datasets in RDF with interlinks to other datasets.

Wikidata (2012), a Wikimedia Foundation project, replaced DBpedia’s extraction approach with a community-maintained structured database of facts about the world, stored in its own RDF-compatible data model. Wikidata has approximately 100 million items as of 2026 and is used by Wikipedia infoboxes, Google’s Knowledge Graph, and numerous research projects. It is arguably the most successful large-scale implementation of Linked Data principles.

Enterprise ontologies found genuine niches. In pharmaceutical research, the Gene Ontology (GO) — predating the Semantic Web era but fully compatible with OWL — became standard infrastructure for biological knowledge management. The NCI Thesaurus (National Cancer Institute) standardized terminology across cancer research databases. Financial services developed the FIBO (Financial Industry Business Ontology). These domain-specific applications succeeded because they had both sufficient funding and a clear organizational mandate to maintain ontologies, and because the data they managed was too structured and professionally maintained to be handled by keyword search.

Schema.org (2011) represented a pragmatic compromise. Google, Bing, Yahoo!, and Yandex jointly created a vocabulary for marking up structured data in HTML — events, products, reviews, recipes, organizations, people — using a simplified ontology embedded in web pages via Microdata or JSON-LD. In exchange, search engines prominently displayed structured search results (rich snippets, knowledge panels) for pages that used Schema.org markup. The incentive was commercial: markup your restaurant’s hours in Schema.org format and Google displays them directly in search results. By 2020, over 40% of web pages used Schema.org markup in some form.

Schema.org was not the Semantic Web. It was a closed vocabulary controlled by the search engine consortium, not an open interoperable standard. It enabled search engines to extract structured data for their own use, not to facilitate agent reasoning across open datasets. But it was the closest the mainstream web came to adopting machine-readable semantics at scale.

Why It Did Not Scale

The Semantic Web’s failure to achieve its original vision had several distinct causes that compounded each other.

The economic alignment problem. The Semantic Web assumed that data publishers would annotate their data for the benefit of third-party agents. This assumption required publishers to do work — writing ontologies, maintaining RDF exports, keeping data current — whose benefit accrued primarily to others. A retailer who exposed its product catalog as RDF was providing competitive intelligence to rival retailers whose agents could scrape it. A health insurer who exposed its coverage database in machine-readable form was enabling policyholders to compare it unfavorably to competitors. The business logic of the commercial web ran directly against the open-data principle the Semantic Web required.

Schema.org succeeded precisely because it solved the economic alignment problem: markup your data and Google rewards you with better search result presentation. The benefit accrued to the publisher. The broader Semantic Web vision had no such incentive structure.

Ontology development was genuinely hard. Creating an OWL ontology that correctly captured a domain’s semantics required expertise in both the domain and formal logic, and the resulting ontologies were difficult to maintain as knowledge evolved. The ontology alignment problem — making two independently developed ontologies for the same domain agree on the meaning of shared concepts — was a research problem that consumed significant academic effort and produced no general solution. Two pharmaceutical companies that each built OWL ontologies of drug-disease relationships would find those ontologies incompatible in subtle ways that required expert human intervention to reconcile.

The open world assumption was wrong for most web data. OWL’s reasoning is based on the open world assumption: if a fact is not stated, it is not necessarily false — it is merely unknown. This is logically correct for an open, incomplete dataset like the web. It is operationally difficult: a query that asks “what medications does this patient not take?” cannot be answered from an OWL dataset that does not list them, because absence of assertion is not assertion of absence. Database reasoning, by contrast, uses the closed world assumption: if it is not in the database, it is false. Most real business and medical applications needed closed-world reasoning, which put them at odds with OWL’s theoretical foundations.

The middle tier never materialized. The Semantic Web architecture required not just data publishers but also agents — software that would navigate RDF data on users’ behalf. These agents needed to exist and be trusted by users before the semantic data had value; the semantic data needed to exist before agents could be usefully built. Neither developers nor publishers moved first. The chicken-and-egg problem that affects every new platform standard was particularly acute here because the Semantic Web’s benefits only emerged at scale.

Warnung

The standards complexity cost: By the mid-2000s, the Semantic Web stack comprised RDF, RDFS, OWL (in three dialects: OWL Lite, OWL DL, and OWL Full, with different computational properties), SPARQL, RIF (Rule Interchange Format), SKOS (Simple Knowledge Organization System), and a growing collection of domain ontologies. A developer who wanted to publish Semantic Web data needed to understand which ontologies to use, which OWL dialect was appropriate, how to serialize RDF (Turtle? N-Triples? RDF/XML? JSON-LD?), and how to run a SPARQL endpoint. The HTML of 1994 had no prerequisites. The Semantic Web of 2008 had a substantial learning curve that deterred the generalist web developers who had built the first web.

What Succeeded Instead

The semantic understanding that the Semantic Web promised arrived through different mechanisms.

Google’s Knowledge Graph (2012) embedded structured knowledge about entities — people, places, organizations, events — directly in Google’s search infrastructure. It drew from Wikipedia, Freebase, Wikidata, and proprietary sources, but the data was not exposed publicly as RDF. It was Google’s private knowledge base, queried through Google’s own systems. The “machine-readable web” became Google’s internal semantic layer, not a shared infrastructure.

Microformats and JSON-LD represented pragmatic implementations of semantic markup that achieved partial adoption. JSON-LD in particular, as the recommended serialization format for Schema.org, brought RDF-compatible structured data to mainstream web development in a format that JavaScript developers could handle without learning the RDF data model. Millions of web pages carry JSON-LD markup for Schema.org purposes, processed by search engines that use it for their own knowledge bases.

Large language models (2018 onward, with transformer architectures) achieved the practical goal of machine text comprehension without explicit semantic annotation. A system like GPT-4 or Claude could answer the query “list the capital cities of countries that border France” by reasoning from text it had seen during training, without requiring RDF triples. The Semantic Web’s approach was deductive: make the structure explicit, then reason over it formally. The LLM approach was inductive: learn the patterns of meaning from massive text corpora, then apply them. The latter approach required no cooperation from data publishers and scaled automatically with available text — which meant it scaled with the web itself.

The irony is complete: the machine understanding that the Semantic Web was designed to provide arrived in a form that bypassed the Semantic Web entirely.

The Survivors

The Semantic Web’s ideas did not disappear uniformly. Several threads survived and prospered.

Wikidata is the Semantic Web’s most successful implementation, though it works by having a community maintain its structured data rather than by having web publishers annotate theirs. It contains facts about over 100 million entities, is used by Wikipedia for infobox data, and is freely queryable via SPARQL. It demonstrates that Linked Data works when there is organizational will and funding to maintain it.

Biomedical and life sciences ontologies remain central infrastructure. The Gene Ontology, SNOMED CT (clinical terminology), ChEBI (chemical entities), and dozens of domain-specific ontologies are used in pharmaceutical research, clinical data management, and scientific data integration. These domains have the professional curation resources, the regulatory requirements, and the clear interoperability benefits that justify the investment.

Knowledge graphs in enterprise — proprietary, often built on RDF and OWL tooling but not publicly exposed — are standard infrastructure in financial services, pharmaceutical research, and large-scale e-commerce. Amazon’s product knowledge graph, Google’s Knowledge Graph, and LinkedIn’s economic graph are all descendants of Semantic Web technology applied in contexts where a single organization controls both the data and the reasoning system.

The Semantic Web failed as an open web standard. It succeeded as an internal enterprise architecture.

Why It Failed

The Semantic Web was a technically coherent solution to a real problem. Its failure was not technical; it was the failure of a set of social and economic assumptions that the technology required but could not enforce.

The web’s first phase succeeded because it was permissive. HTML was simple enough to publish without expertise, and the benefit to a publisher — a globally accessible document — accrued to that publisher immediately. The Semantic Web’s benefit to any individual publisher depended on network effects: value came from the density of the semantic data network, which only accumulated when many publishers adopted the standards, which only happened when the benefits were already visible. The social technology for bootstrapping this — the standard body mandates and regulatory requirements that can sometimes achieve critical mass — never appeared for the public web.

The web had also developed, by the time Berners-Lee’s vision was articulated, a powerful set of intermediaries — search engines — who had strong commercial reasons to maintain the opacity of their indexes. A truly machine-readable web would have made Google’s index replicable by any SPARQL-speaking agent, eliminating Google’s moat. The search engines had no interest in the Semantic Web’s success, and they had the market power to define what the practical “semantic web” would mean: Schema.org, on their terms, in exchange for search result benefits.

Finally, the field moved. The statistical and neural approaches to natural language understanding that matured in the 2010s produced systems that could extract meaning from text without explicit annotation. Once BERT (2018) and GPT-2 (2019) demonstrated that machines could comprehend unstructured text at near-human levels, the argument for requiring publishers to annotate their data explicitly grew weaker every year.

Berners-Lee’s vision was not wrong. It was early, and then it was overtaken. The machine-readable web arrived; it just arrived through large language models reading HTML rather than through agents querying RDF.