Reverse Engineering and the Law

Abstract

To build a program that works with someone else’s program, a programmer usually has to look inside it, and looking inside a binary means copying it into memory, disassembling it and writing the result down. Whether that copying is legal was settled in American courtrooms between 1989 and 2000, in a run of cases about microprocessors, game consoles and emulators: NEC v. Intel, Atari v. Nintendo, Sega v. Accolade and Sony v. Connectix. The courts held that disassembly for interoperability is fair use, and that a clean room is good evidence of independent work. Europe wrote a narrower version of the same rule into its 1991 Software Directive. Then the 1998 DMCA and enforceable licence terms took back much of what the courts had granted, and the question moved from “may I copy the code to study it?” to “may I break the lock, and did I sign away the right?”

The Problem

A compiled program is a long sequence of machine instructions. Its interfaces, the calls it expects, the bytes a console checks before it runs a cartridge, the messages a server sends, are not written down anywhere the outsider can read. The only way to learn them is to run the program and watch it, or to translate the machine code back into something like assembly language. Both involve copies: the program is loaded into RAM, printed out, annotated, stepped through in a debugger. Copyright law gives the owner the exclusive right to make copies. Read literally, it makes every act of study an infringement.

The industry’s answer in the early 1980s was procedural rather than legal. When Compaq cloned the IBM PC’s BIOS in 1982, one team read IBM’s code and wrote a specification of what it did, and a second team, which had never seen IBM’s code, wrote new code from the specification. Phoenix Technologies sold BIOS firmware made the same way from May 1984 (see Compaq and the BIOS Clean Room). Nobody had yet tested in court whether this clean-room design protected anyone.

The Clean Room in Court

The first test came from a chip. NEC had been a licensed second source for Intel’s 8086 family and in March 1984 launched its own compatible processors, the V20 and V30, with microcode NEC had written itself. Intel argued that microcode, the internal program that tells a processor how to execute each instruction, was a computer program protected by copyright, and that NEC’s was a copy of Intel’s. NEC sued for a declaration that it had not infringed.

The case took four and a half years. The first judge, William Ingram, had to withdraw after it emerged that he held $80 of Intel stock through an investment club. For the trial NEC prepared a clean-room version of its microcode: an engineer, Gary Davidian, was given the hardware specification without access to Intel’s or NEC’s microcode and asked to write his own. In February 1989 Senior Judge William Gray ruled. Microcode was copyrightable, he found, but Intel had forfeited its copyright by letting chips ship without a copyright notice, and in any case NEC’s microcode did not infringe. He accepted the clean-room work as evidence that similar code could arise independently, because the instruction set constrained what any implementation could look like. NEC kept selling the V20. Davidian’s microcode was put into a few chips and never needed. Intel did not license a second source for the 80386.

Nintendo’s Key

Game consoles turned the question from an academic one into a commercial war. After the North American video game crash of 1983, Nintendo controlled who could publish for its Entertainment System through a lockout chip, the 10NES: a console would only run a cartridge containing a matching chip, and Nintendo sold those chips only to licensees who accepted its terms. Atari Games, which formed the subsidiary Tengen in 1986 to publish for the NES, signed up as a licensee in 1987 and agreed to make its games exclusive to Nintendo for two years.

Tengen wanted out. Its engineers tried to reverse engineer the 10NES by watching the chip’s signals and by chemically peeling the chip and photographing its layers under a microscope. That failed. In 1988 Atari’s lawyers then obtained a copy of the 10NES program from the U.S. Copyright Office, where Nintendo had deposited it, by stating that they needed it for litigation with Nintendo. No such litigation existed. With the code Atari built the Rabbit chip, which produced signals the console could not tell apart from a 10NES, and sold unlicensed cartridges. Atari sued Nintendo for antitrust violations in December 1988; Nintendo countersued in November 1989.

On 10 September 1992 the Federal Circuit decided against Atari. Judge Randall Rader’s opinion stated the principle first: “reverse engineering, untainted by the purloined copy of the 10NES program and necessary to understand 10NES, is a fair use.” Atari could not claim it, because its copy had been obtained under false pretences. The court found Atari had “unclean hands.” The companies settled in 1994, with Atari paying damages and licence fees.

Sega’s Trademark Trap

Six weeks later the Ninth Circuit decided the case that defined the rule. Accolade, a San Jose publisher founded in 1984 by Alan Miller and Bob Whitehead, two former Atari programmers who had co-founded Activision, made games for computers and wanted to publish for the Sega Genesis without paying Sega’s licence fees. Its engineers bought a console, disassembled the code of three Genesis games, worked out what the console needed to accept a cartridge, and wrote their own games around it, starting with Ishido: The Way of Stones.

Sega’s countermeasure was legal as much as technical. At the Winter Consumer Electronics Show in January 1991 Accolade learned that new Genesis models would contain the Trademark Security System (TMSS): the console checked for the four letters “SEGA” at a fixed place in the cartridge, and when it found them, it displayed the message “Produced by or under license from Sega Enterprises Ltd.” before the game started. Any unlicensed cartridge that wanted to run on the new consoles had to contain Sega’s name, and would then display a false claim that Sega had licensed it. Sega could sue for trademark infringement as well as copyright.

Sega sued on 31 October 1991. On 3 April 1992 Judge Barbara Caulfield of the Northern District of California issued an injunction barring Accolade from selling Genesis games and, a week later, ordered a recall. Accolade’s Genesis sales stopped for several months; Alan Miller later put the damage at $15 to $25 million.

The Ninth Circuit lifted the injunction on 28 August 1992 and issued its opinion, by Judge Stephen Reinhardt, on 20 October. Disassembly, the court held, “is a fair use of the copyrighted work, as a matter of law” where it is the only way to reach the functional elements of a program and there is a legitimate reason to seek them. The functional requirements for compatibility were not protected by copyright, and Sega could not use copyright to control who made games for its console. On the trademark claim the court put the blame on Sega: by making its name the key, “Sega knowingly risked” that competitors’ products would be falsely labelled. Sega and Accolade settled on 30 April 1993, and Accolade became an official licensee.

The Emulator

The rule was tested once more at the end of the decade, on emulators (see Emulation and Retrocomputing). Connectix, a Mac utilities company, began work in July 1998 on the Virtual Game Station, a Macintosh program that ran Sony PlayStation games. Its engineers first ran Sony’s own BIOS inside their emulator, then wrote a replacement BIOS, using Sony’s as a debugging aid. They asked Sony for technical help; Sony declined in September 1998. The VGS went on sale in January 1999, and Sony sued on 27 January. The district court enjoined sales and impounded Connectix’s copies of the BIOS.

On 10 February 2000 the Ninth Circuit reversed, three judges to none. Copying the BIOS during development was fair use: the final product contained none of Sony’s code, the emulator was “modestly transformative” because it created a new platform for PlayStation games, and the loss of console sales to a competing platform was not the kind of harm copyright protects against. “The copyright law, however, does not confer such a monopoly,” the court wrote. The Supreme Court declined Sony’s appeal. Sony then won by other means: it bought the VGS from Connectix on 15 March 2001 and discontinued it on 30 June.

Europe Writes It Down

Europe settled the question by legislation. The Council Directive 91/250/EEC of 14 May 1991 on the legal protection of computer programs gave every member state the same rule. Its Article 6 allows decompilation without the owner’s permission only when three conditions hold: it is done by a lawful user, the information needed for interoperability is not otherwise readily available, and the work is confined to the parts of the program needed for interoperability. The results may not be used for anything else, passed on except as interoperability requires, or used to build a substantially similar program. A licence term that forbids what Article 6 allows is void. The 2009 codification, Directive 2009/24/EC of 23 April 2009, kept the wording; in Germany the rule is §69e of the copyright act.

The European version is narrower than Sega in one way and stronger in another. It allows decompilation only for interoperability, not for study in general, but a contract cannot take that right away. The Court of Justice drew its outer lines in two cases. In SAS Institute v. World Programming (2 May 2012) it held that the functionality of a program, its programming language and its data file formats are not protected by copyright, so a competitor who reimplemented SAS’s language by observing and testing the program, without access to its source code, did not infringe. In Top System (6 October 2021) it allowed a lawful user to decompile a program to correct its errors.

Interoperability also became a competition-law remedy. On 24 March 2004 the European Commission ruled that Microsoft had abused its dominant position and ordered it to disclose the protocols that workgroup servers need to work with Windows PCs and Windows servers. After Microsoft lost its appeal on 17 September 2007, the Samba project, which had spent years reverse engineering those protocols by watching network traffic, obtained the specifications through the Protocol Freedom Information Foundation for a one-time fee of €10,000, announced on 20 December 2007.

The DMCA’s Narrow Door

In the United States the court-made rule was overtaken by statute. The Digital Millennium Copyright Act, signed on 28 October 1998, made it illegal under Section 1201 to circumvent a technical measure that controls access to a copyrighted work, whether or not any copying followed (see Copyright and IP in the Digital Age). A lockout chip like the 10NES or the TMSS now had a second line of protection that fair use did not reach. Congress added one exception, Section 1201(f): a lawful user may circumvent “for the sole purpose of identifying and analyzing those elements of the program that are necessary to achieve interoperability of an independently created computer program with other programs,” provided the information was not readily available and the act is not otherwise infringing.

The courts read the exception in both directions. In December 2002 the printer maker Lexmark sued Static Control Components under Section 1201 and in March 2003 obtained an injunction that stopped it from selling replacement chips that let refilled toner cartridges work in Lexmark printers. On 26 October 2004 the Sixth Circuit vacated the injunction, holding that the authentication handshake did not control access to Lexmark’s printer software in the sense the statute meant, since anyone who bought the printer could read that code. A year later the exception failed the hobbyists who had written bnetd, a free server that let owners of Blizzard games play online without Blizzard’s Battle.net service. They had studied the protocol by logging the traffic between the games and Blizzard’s servers. On 1 September 2005 the Eighth Circuit, in Davidson & Associates v. Jung, held that their circumvention was not protected because the server let unauthenticated copies of the games play online.

Contract Beats Copyright

The second retreat came through licence terms. Most commercial software is sold under a licence that forbids reverse engineering, accepted by opening the box or clicking a button. If such a term is enforceable, Sega matters only to people who never agreed to it. In Bowers v. Baystate Technologies (2003) the Federal Circuit held that it was: Baystate had reverse engineered a CAD add-on whose shrink-wrap licence prohibited it, and owed damages for breach of contract, whatever copyright law allowed. Judge Timothy Dyk dissented, warning that “state law could extensively undermine the protections of the Copyright Act.” The bnetd court reached the same result two years later, holding that Blizzard’s licence terms were enforceable: by accepting them, the defendants had “expressly relinquished their rights to reverse engineer.”

The result was two systems with opposite defaults. In the European Union, a licence cannot forbid decompilation for interoperability. In the United States, a licence usually can, and a technical lock usually brings Section 1201 with it. American reverse engineers have since relied on the triennial exemptions the Copyright Office grants under Section 1201, which have covered phone jailbreaking, vehicle repair, security research and game preservation, each argued afresh every three years.

The largest interface case, Google v. Oracle, ended in the Supreme Court on 5 April 2021 without deciding whether interfaces can be copyrighted. Google had reimplemented the declarations of the Java API for Android, and the Court held 6 to 2 that this was fair use (see Android: The Dominant Mobile OS).

Dead End: The Trademark as a Key

Sega’s TMSS was an attempt to use trademark law where copyright law had failed, by making the act of compatibility itself a false statement. The Ninth Circuit refused: a company that designs its console so that compatibility requires printing its name cannot then sue competitors for printing it. The same court rejected the lock-as-copyright argument in Sony v. Connectix, and Lexmark’s attempt to use the DMCA against compatible toner chips failed in 2004.

Locks that rest on cryptography rather than trademark have fared better, because Section 1201 protects them without any need to show copying. Consoles moved to cryptographic signatures, and the legal fights over them moved from the question of whether a competitor may study the system to whether anyone may break the encryption at all, the question at the centre of Sony’s 2011 suit against George Hotz over the PlayStation 3 root key (see George Hotz).