Dead End: Intel Optane and 3D XPoint

Abstract

On 28 July 2015 Intel and Micron announced 3D XPoint, a non-volatile memory they presented as the first new memory category since NAND flash: up to 1,000 times lower latency than flash, 1,000 times its write endurance, and ten times the density of DRAM. Intel sold it from 2017 as Optane, first as fast SSDs and from 2019 as memory modules that sat in DRAM slots and kept their contents without power. It was the fastest storage money could buy and the slowest, cheapest main memory, and the market for something in between turned out to be small. Micron, which owned the only factory, left in March 2021 and sold the plant to Texas Instruments. On 28 July 2022, seven years to the day after the announcement, Intel told investors it was winding the business down and booked a $559 million inventory write-off.

An Intel Optane SSD DC P4800X, the first Optane product (2017), as a PCIe add-in card. Image: Larios-m2, CC BY 4.0, via Wikimedia Commons.

The Gap in the Hierarchy

Every computer since the 1950s has kept its data on two kinds of storage with nothing useful between them. Working memory (core, then DRAM) is fast, byte-addressable, and forgets everything when the power goes. Storage (drum, disk, then flash) keeps data but is reached in blocks through an I/O stack, thousands of times more slowly. Operating systems, databases, and file systems are built around this split: data is copied into memory to be used, and written back to storage to be kept (see The Storage Revolution).

By the 2010s flash had narrowed the gap from the storage side. A good NAND SSD answered in tens to hundreds of microseconds instead of the milliseconds of a disk, but DRAM answered in about a hundred nanoseconds, still roughly a thousand times faster. The industry had a name for the missing tier, storage-class memory, and a list of candidates: phase-change memory, resistive RAM, spin-transfer-torque MRAM, and HP’s memristor (see The Memristor). Each was either too small, too expensive or not yet out of the laboratory.

Intel and Micron had made NAND flash together since 2006 through a joint venture, IM Flash Technologies, with a fab in Lehi, Utah. Intel had also worked on phase-change memory for years; in 2009 it and Numonyx had presented stackable phase-change test chips. 3D XPoint came out of that line of work.

The Announcement

The two companies unveiled the technology on 28 July 2015. Their release called it the first new memory category since the introduction of NAND flash in 1989. Intel’s Rob Crooke, head of its Non-Volatile Memory Solutions Group, told Fortune that it was “pretty tough stuff” to make, and that moving production to it was about as easy as moving to the next step of chip shrinking. The figures were striking: up to 1,000 times lower latency than NAND, 1,000 times the write endurance, and about ten times the density of DRAM. Micron’s shares rose 9 percent that day.

The first chips held 128 gigabits on two stacked layers of 64 gigabits each, made in Lehi. The “cross point” in the name described the layout: memory cells sat at the intersections of perpendicular word and bit lines, with no transistor per cell, and the layers could be stacked. Each cell combined a storage element, whose bulk resistance changed between a high and a low state, with a selector called an ovonic threshold switch that let current through only above a set voltage. Intel insisted that the technology was neither phase-change memory nor a memristor and published little about the material. Teardown firms that took the chips apart reported a germanium-antimony-tellurium alloy of the kind used in phase-change memory, with a selenium-based selector.

The announcement promised samples by the end of 2015 and products in 2016. Micron said in August 2016 that its version, branded QuantX, would sell for about half the price of DRAM and four to five times the price of flash. That ratio defined the product for the rest of its life.

The Products

Intel shipped first. On 19 March 2017 it announced the Optane SSD DC P4800X, a data-centre drive of 375 GB on a PCIe card, at an estimated $1,520, about $4 per gigabyte when ordinary NAND SSDs cost a fraction of that. Its sequential throughput was no better than a fast flash drive. What it offered was latency: reviewers measured responses of around 10 microseconds and sustained 500,000 random operations per second, and unlike flash it did not slow down under mixed reads and writes or wear out under heavy writing. For databases with small random writes, logging, and caching, it was the fastest drive on the market.

A 32 GB Intel Optane Memory module (M.2), sold from 2017 as a cache in front of a hard disk. Image: Jacek Halicki, CC BY-SA 4.0, via Wikimedia Commons.

For consumers Intel sold Optane Memory from April 2017: small M.2 modules that worked as a cache in front of a hard disk, and only with recent Intel processors and chipsets. Their market was PCs that still shipped with a hard disk, a market that cheap flash SSDs were already removing. Enthusiast drives such as the Optane 900P (launched in October 2017) followed, fast and expensive.

The product that was meant to change computing came in April 2019. Optane DC Persistent Memory modules plugged into DRAM slots next to ordinary memory and came in 128, 256, and 512 GB, far more per module than DRAM offered. They worked only with Intel’s second-generation Xeon Scalable server processors (Cascade Lake) and could be used in two ways:

In Memory Mode the Optane modules served as large, cheap main memory, with the DRAM acting as a cache in front of them. Software needed no changes, but the data was treated as volatile, so the non-volatility went unused.
In App Direct mode the modules appeared as persistent memory: byte-addressable storage that the processor could read and write with ordinary load and store instructions, and that survived a power cut. This needed an operating system with a persistent-memory-aware file system and, to get the full benefit, software rewritten to manage its own data structures in persistent memory.

App Direct was the idea that excited researchers: a database that never had to copy its tables between storage and memory, or recover them after a crash. It also meant rewriting databases, file systems, and libraries for a memory that only one vendor’s processors could use. A few did. Oracle used Optane in its Exadata database machines, SAP’s in-memory database HANA supported it from 2019, and VAST Data built its storage systems around Optane as a write buffer for cheap QLC flash.

Micron’s own product, the X100 NVMe SSD, was unveiled in October 2019 and sold in small numbers. QuantX never shipped.

The Money

Intel did not report Optane separately. Jim Handy of Objective Analysis, who followed the memory market, estimated the losses from the published results of Intel’s non-volatile memory group: about $921 million in 2016, $1.7 billion in 2017, $2.1 billion in 2018, $1.6 billion in 2019 and $576 million in 2020, nearly $7 billion in five years. “A billion here, a billion there, and pretty soon you’re talking real money,” he said, quoting Senator Everett Dirksen. Tom Coughlin of Coughlin Associates put the problem in one sentence: Optane “needed to get higher volume to get the cost down,” and Intel had to subsidize it until it did.

Intel could afford the subsidy because Optane was also a reason to buy Xeons: the persistent-memory modules worked only with Intel processors, and AMD’s server chips, then winning market share, could not use them. The same lock-in kept cautious customers away, because software rewritten for App Direct tied them to one supplier.

The manufacturing arrangement unravelled first. Intel and Micron ended joint development in stages from 2018, Micron took over the Lehi fab, and Intel bought 3D XPoint wafers from it under contract. In January 2021 Intel stopped selling its consumer Optane drives. In 2020 it had also agreed to sell its NAND flash business to SK hynix, leaving Optane as the only memory product it kept.

Micron Leaves

In March 2021 Micron announced that it would stop developing 3D XPoint and sell the Lehi fab, citing “insufficient market validation to justify the ongoing high levels of investments required to successfully commercialize 3D XPoint at scale.” It said it would put the resources into memory for Compute Express Link (CXL), a new interconnect that let servers attach memory over the PCIe bus. On 30 June 2021 Texas Instruments agreed to buy the fab for $900 million in cash, to make analog and embedded chips on 65 and 45 nm processes. Because the price was below book value, Micron expected an impairment of about $435 million. The plant that had been the world’s only source of 3D XPoint would make microcontrollers.

Intel said the sale did not change its ability to supply Optane. The analyst Jim Handy was surprised that Intel had not bought the fab and expected it to make 3D XPoint at its own plant in Rio Rancho, New Mexico. Intel never did.

The Wind-Down

Intel reported its second-quarter results on 28 July 2022. The press release mentioned the “wind-down of our Intel Optane memory business” and an inventory impairment of $559 million. “After careful consideration, Intel plans to cease future product development within its Optane business,” the company said, as part of a review of businesses that were “not sufficiently profitable or not core to our strategic objectives.” Reports from the trade press, which Intel did not confirm, said it had been sitting on about two years’ worth of 3D XPoint chips. Intel promised to keep supplying existing customers.

The next server processor generation, Sapphire Rapids, had been shown with Optane support in 2021; by mid-2022 Intel’s slides for it showed only DDR5. VAST Data’s co-founder Jeff Denworth said the decision would not disrupt his company’s ability to supply its storage systems, and MemVerge, which sold software for large Optane memory pools, said Optane was “an optional component.” The industry’s plans for a memory tier between DRAM and flash moved to CXL, which could attach any kind of memory, including ordinary DRAM, without tying it to one processor vendor.

Dead End

Optane was a good technology aimed at a gap that the rest of the industry kept closing. As an SSD it competed with NAND flash, whose price per gigabyte fell every year as manufacturers stacked more layers, and whose speed was good enough for nearly every workload. As memory it competed with DRAM, which was faster, and its price advantage (about half of DRAM’s) was too small to justify rewriting software or accepting a single supplier. The applications that benefited most, such as database logs, write buffers, and very large in-memory databases, were real but few, and they did not produce the volume that the cost curve needed. Intel was the only seller and Micron the only maker, and neither could get the other to share the risk.

The pattern resembles magnetic bubble memory forty years earlier: a technology between two incumbents, each of which improved faster than the newcomer could scale. The ideas Optane brought into the mainstream survived it. Linux and Windows kept their persistent-memory support, the programming model for byte-addressable non-volatile memory is documented in libraries and textbooks, and CXL memory expansion, which Micron, Samsung, and SK hynix built products for, took over the role of the large, cheaper memory tier.