The Embedded Systems Story: From ROM Firmware to Safety-Critical Software

Zusammenfassung

The computer in your laptop runs software you can update, install, and replace. The computer in your car’s brake system does not. Embedded systems — processors built into devices to control them — are the most numerous computers in the world: a modern automobile contains over 100 microcontrollers. They run software that is often permanent, sometimes life-critical, and almost always invisible. The history of embedded systems is the history of computing stripped to its essentials: minimal memory, minimal power, hard real-time deadlines, and consequences for failure that range from an annoying product recall to a crashed aircraft. The engineering discipline that developed to build such systems — real-time operating systems, formal verification, DO-178C certification — is one of the most demanding in computing, and one of the least celebrated.

The First Embedded Computer: Apollo Guidance Computer

The first general-purpose computer designed to be embedded in a vehicle was the Apollo Guidance Computer (AGC), designed at MIT’s Instrumentation Laboratory (later the Draper Laboratory) by Charles Stark Draper’s team between 1961 and 1966.

The constraints were extreme: the AGC had to fit in 0.03 m³, weigh under 32 kg, consume under 55 watts, and survive the radiation environment of space. It had 2,048 words of RAM (each word was 16 bits — 15 data bits plus parity) and approximately 36,000 words of read-only rope core memory — a form of ROM where wires were literally threaded through magnetic cores to encode bit patterns. The entire lunar mission software — guidance algorithms, navigation calculations, thrust control, display management — had to fit in this memory.

Margaret Hamilton, who led the AGC software development team at MIT, coined the term “software engineering” to describe the discipline required to build the AGC’s code. The phrase was Hamilton’s deliberate argument that software development was as rigorous and consequential as hardware engineering — a claim that was not universally accepted at the time and that the Apollo program’s software quality helped establish.

The AGC ran a real-time operating system — the first practical RTOS for an embedded system — that scheduled five priority levels of tasks, handled asynchronous interrupts from guidance sensors, and managed the limited memory through overlays (programs that loaded into memory only when needed). When the 1202 program alarm appeared during the Apollo 11 lunar descent — indicating that the AGC was overloaded with sensor interrupts from an incorrectly configured radar — the RTOS’s priority scheduler automatically dropped the low-priority tasks and kept the critical guidance calculations running. The astronauts landed safely.

RTOS: Real-Time Operating Systems

Desktop and server operating systems optimize for average throughput. Real-time operating systems optimize for deterministic timing: the guarantee that a specific operation will complete within a specific deadline, regardless of other system activity.

Hard real-time systems must meet every deadline absolutely. Missing a deadline in a hard real-time system is a failure — the system has not performed its function. Anti-lock braking systems, flight control computers, and pacemakers are hard real-time: if the brake control computer misses its 10ms response deadline, the brake may not engage.

Soft real-time systems tolerate occasional deadline misses. A missed deadline degrades quality but does not constitute failure. Video streaming, audio playback, and interactive applications are typically soft real-time.

VxWorks (Wind River Systems, 1987) became the dominant commercial RTOS for aerospace, defense, and industrial applications. It provided deterministic task scheduling, interrupt latency measured in microseconds, and a development environment (the Tornado IDE) that separated the development machine from the target hardware. VxWorks ran the Mars Pathfinder rover (1997), which experienced a famous real-time priority inversion bug: a low-priority task held a mutex needed by a high-priority task, which was blocked; a medium-priority task then preempted the low-priority task, effectively starving the high-priority task and causing the system to reset. The Mars Pathfinder team diagnosed and fixed the bug remotely by enabling priority inheritance in the VxWorks configuration — one of the most dramatic remote debugging episodes in RTOS history.

QNX (1980) targeted telecommunications and industrial control with a microkernel architecture — the RTOS core handled scheduling and IPC; all other services ran as user-space processes. QNX’s architecture meant that a failing device driver could be restarted without rebooting the system — critical for systems requiring continuous uptime. Research In Motion (later BlackBerry) used QNX as the OS for its BlackBerry 10 smartphones.

FreeRTOS (Richard Barry, 2003) was released under a modified GPL license, later relicensed under MIT, and became the dominant open-source RTOS for microcontrollers. Amazon acquired FreeRTOS in 2017 and released it as Amazon FreeRTOS with cloud connectivity libraries, positioning it as the RTOS for IoT devices.

The C Language and the Firmware Tradition

Embedded systems programming converged on C as the primary language, for reasons grounded in the language’s design: C mapped closely to machine instructions (a C integer addition compiled to a machine integer addition with no hidden overhead), provided direct memory access through pointers (essential for memory-mapped hardware registers), and generated compact code with predictable timing.

The embedded C style was distinct from application C:

No dynamic memory allocation: malloc() was typically forbidden. Heap fragmentation in a long-running embedded system caused unpredictable latency; a failed allocation in a safety-critical context caused catastrophic failure. All data structures were statically allocated.

No floating-point (on MCUs without FPUs): microcontrollers without floating-point hardware units emulated floating-point in software — slowly and with non-deterministic timing. Embedded code used fixed-point arithmetic (integers scaled by known factors) to achieve speed and determinism.

Volatile-qualified memory-mapped registers: hardware registers accessible at specific memory addresses required the volatile qualifier to prevent the compiler from optimizing away accesses that the programmer intended to generate hardware effects.

Interrupt service routines (ISRs): short functions called by hardware interrupts, required to execute in minimal time, forbidden from calling most standard library functions.

MISRA C: Coding Rules for Safety

The MISRA C guidelines (Motor Industry Software Reliability Association, 1998, updated 2004 and 2012) codified a subset of C safe for use in safety-critical embedded systems. MISRA C prohibited dynamic memory, required explicit casts for integer promotions, forbade certain control flow patterns (goto, recursion), and mandated documentation conventions. Automotive, aerospace, and medical device vendors required MISRA C compliance from their software suppliers as a contractual condition. Static analysis tools (PC-lint, LDRA, Polyspace) automated MISRA compliance checking. The guidelines were controversial among programmers who found them restrictive; they were accepted by the safety engineering community as a pragmatic response to C’s undefined behavior and implementation-defined behavior, which were intolerable in safety-critical code.

DO-178C and the Certification Burden

Avionics software faces the most demanding certification process in embedded systems: DO-178C (Software Considerations in Airborne Systems and Equipment Certification), a standard issued by RTCA in 2011 that specified the development process, documentation requirements, and verification activities required for software used in aircraft systems.

DO-178C defined five criticality levels, from Level E (failure has no safety effect) to Level A (failure causes or contributes to catastrophic failure of the aircraft). Level A software required:

Formal requirements for every software function, traceable to system requirements.
100% MC/DC coverage (Modified Condition/Decision Coverage) — every decision branch and every condition within decisions must be independently exercised in testing.
Independence between developers and testers — the people who wrote the code could not be the primary testers.
Tool qualification for any development tool whose output was not independently verified — a compiler that introduced code not present in the source had to be qualified to DO-330 standards.

The cost of DO-178C Level A certification was estimated at $1,000 per source line of code for complete documentation and testing. A flight control computer with 100,000 lines of code cost $100 million to certify — before writing a line of production code. This economics shaped which companies could participate in avionics software development and why aircraft systems ran on software from specialized vendors (Rockwell Collins, Honeywell, Thales) rather than Silicon Valley startups.

Dead End: The Formal Verification Gap

Formal verification — mathematical proof that software meets its specification — was the theoretically correct solution to safety-critical software’s quality problem. If you could prove that a program was correct, you would not need to test every path through it.

Partial successes existed: the seL4 microkernel (2009) was formally verified to be free of implementation bugs — the first production operating system kernel with a machine-checked proof of correctness. The CompCert C compiler (INRIA, 2009) was formally verified to produce binaries that faithfully implemented the C semantics of their source code.

Full formal verification of large embedded systems remained impractical for most applications: the specification effort required to express what correct behavior meant was comparable to the implementation effort, and the tools required significant expertise that most embedded development teams lacked. The gap between “formally verified microkernel” and “formally verified flight management system with 10 million lines of code” was measured in decades of unsolved research problems.