The Networking Hardware Story: The Physical Internet

Packet Switching: The Idea That Changed Everything

In the early 1960s, telecommunications meant circuit switching: a dedicated physical path was reserved for the duration of a call. The system was reliable but wasteful — the entire circuit sat idle whenever neither party spoke.

Paul Baran at RAND Corporation (1962) and Donald Davies at the National Physical Laboratory in the UK (1965) independently proposed an alternative: break messages into discrete packets, each carrying its own destination address, and let each node forward packets toward the destination independently. A packet could take a different route than the one before it. Nodes could fail without severing communication.

Baran’s motivation was explicitly military: a network that could survive a nuclear strike by routing around damage. Davies coined the word “packet.” The concept was implemented in the ARPANET (1969), connecting UCLA, Stanford Research Institute, UC Santa Barbara, and the University of Utah — four nodes, first message on October 29, 1969.

The Router: The Internet’s Traffic Cop

Early ARPANET nodes used Interface Message Processors (IMPs) — specialized minicomputers built by Bolt Beranek and Newman (BBN) — to handle packet forwarding. Each IMP knew its immediate neighbors and could forward packets one hop at a time.

As networks multiplied and the concept of internetworking (connecting separate networks) emerged, a more sophisticated device was needed: one that could understand the topology of many interconnected networks, not just its local neighborhood. This was the router.

Cisco Systems, founded in 1984 by Stanford computer scientists Leonard Bosack and Sandy Lerner, shipped the first multi-protocol router — a device that could forward packets between networks using different protocols. The timing was perfect: the ARPANET’s transition to TCP/IP (January 1, 1983) and the subsequent explosion of university and corporate networking created a market Cisco was uniquely positioned to serve.

By the mid-1990s, Cisco controlled over 80% of the enterprise router market. Its IOS (Internetwork Operating System) became the de facto operating system of the internet’s backbone, and the mental model of “routers speak Cisco CLI” persisted for decades.

How Routing Actually Works

A router maintains a routing table — a map of which outgoing interface leads toward which destination network. Tables are populated by routing protocols:

• RIP (Routing Information Protocol, 1988): simple distance-vector protocol; routers share their entire tables with neighbors. Scales poorly; 15-hop maximum.
• OSPF (Open Shortest Path First, 1989): link-state protocol; routers share the state of their direct links, and each builds a complete map of the network. Scales within an organization.
• BGP (Border Gateway Protocol, 1989): the protocol that connects autonomous systems — the backbone of the global internet.

BGP: The Protocol That Holds the Internet Together

The internet is not one network — it is approximately 80,000 Autonomous Systems (ASes), each controlled by a different organization (ISP, university, company, government). BGP is the protocol these ASes use to announce which IP address blocks they can reach and to learn routes to the rest of the internet.

BGP’s design reflects the political reality of the internet: it is built on trust, not enforcement. When an AS announces “I can reach 192.0.2.0/24,” every other AS that hears this announcement has no cryptographic proof that the claim is true. This has led to two recurring failure modes:

• Route leaks: An AS accidentally announces routes it should not, drawing traffic through itself unintentionally. In 2010, China Telecom briefly absorbed a significant fraction of global internet traffic due to a BGP leak.
• BGP hijacking: A malicious AS falsely announces ownership of another AS’s IP space, intercepting or blackholing traffic. The 2008 Pakistan Telecom/YouTube incident briefly took YouTube offline globally.

RPKI (Resource Public Key Infrastructure), standardized by the IETF in 2012, adds cryptographic route origin validation to BGP — but as of 2025, adoption remains incomplete.

The Physical Layer: Fiber, Cables, and Colocation

Routing protocols are software — but the internet is also physical infrastructure, and that infrastructure has a geography.

Submarine Cables

Roughly 95% of intercontinental internet traffic travels through undersea fiber-optic cables, not satellites. The current network consists of approximately 400 active submarine cables totaling over 1.3 million kilometers. Key facts:

• A modern submarine cable carries 100–400 Tbps.
• Cables land at specific cable landing stations — choke points that are the subject of intense geopolitical attention.
• Most transoceanic cables are owned by consortia of telecommunications companies; increasingly, hyperscalers (Google, Meta, Amazon, Microsoft) are funding private cables.
• Cable cuts — by anchors, fishing trawlers, or deliberate sabotage — are a recurring cause of regional internet degradation. The 2022 Tonga eruption severed the island’s only submarine cable, leaving it largely isolated for weeks.

The Internet Exchange Point

Internet Exchange Points (IXPs) are physical locations where multiple networks interconnect directly, bypassing third-party transit providers. The largest — DE-CIX Frankfurt, AMS-IX Amsterdam, LINX London — handle traffic peaks exceeding 20 Tbps. Connecting at an IXP reduces latency, improves resilience, and lowers cost compared to transiting through a Tier 1 carrier.

The economics of IXPs shaped the internet’s geography: cities with major IXPs became hubs; regions without them face higher latency and cost. The absence of major IXPs in sub-Saharan Africa was a structural barrier to affordable internet access for decades.

Dead End: Token Ring and ATM

IBM’s Token Ring (1984) was a technically elegant alternative to Ethernet for LAN networking. Rather than the collision-based CSMA/CD of Ethernet, Token Ring used a circulating “token” — only the node holding the token could transmit, eliminating collisions entirely. Under high load, Token Ring was theoretically more efficient than early Ethernet.

It lost anyway. Ethernet’s open specification, lower hardware costs, and the sheer momentum of market adoption made it impossible for Token Ring to survive once Fast Ethernet (100 Mbps, 1995) arrived. IBM officially ended Token Ring development in 2003.

ATM (Asynchronous Transfer Mode) was the 1990s telecommunications industry’s answer to data networking: a unified standard for voice, video, and data using fixed 53-byte cells. It was technically sophisticated and backed by every major telco. It lost to IP, which was technically messier but ran on cheap commodity hardware and was already deployed everywhere. ATM survives as an internal technology in some DSL infrastructure, invisible to users.

Legacy: The Infrastructure Nobody Sees

The internet’s physical layer is largely invisible to its users — until a submarine cable is cut, a BGP route leaks, or a colocation datacenter loses power. The gap between the internet’s perceived reliability and its actual fragility is maintained by thousands of network engineers, automated monitoring systems, and the pragmatic trust relationships of the BGP ecosystem. The physical internet is not infrastructure — it is a living negotiation, conducted in milliseconds, between tens of thousands of independent networks that agreed to talk to each other.

---

📚 Sources

• Paul Baran, On Distributed Communications - RAND Corporation, 1964
• The History of Cisco Systems - Computer History Museum
• BGP Security - RIPE NCC
• Submarine Cable Map - TeleGeography
• Internet Exchange Points - Euro-IX
• The 2010 China Telecom BGP Incident - Renesys