Analysis Since the virtual dawn of computing, storage – where data puts its feet up when it's at home – has been massively slower than memory, where data puts on its trainers and goes for a quick run.
That massive access speed gap has been getting narrower and narrower with each storage technology advance: paper tape, magnetic tape, disk drives, and flash memory. Now it is set to shrink again with even faster non-volatile solid-state memory, led by 3D XPoint.
This will have far-reaching effects on servers and on storage as system hardware and software change to take advantage of this much faster – near-memory speed in fact – storage.
Memory contents are volatile or ephemeral, in that they are lost when power is turned off. Storage contents, be they held in NAND, on disk or tape, are persistent or non-volatile because they stay unharmed by power being switched off to the device.
Paradoxically, storage media that doesn’t move is faster than storage media that does, with solid state, non-volatile memory or NAND being much faster to access than rotating disk and streaming tape.
A memory access can take 200 nanoseconds*. This is slow by CPU cache-access speed standards, with a level 1 processor cache access taking half a nanosecond, 400 times faster, and a level 2 cache access needing seven nanoseconds.
A disk-drive seek, the time for the read-write head to move across the platter and reach the right track area, can take 10 million nanoseconds – that’s 10 milliseconds or 10,000 microseconds. Let’s say it’s slightly more than two orders of magnitude slower than a memory access.
A solid state drive (SSD) is vastly faster than this. Micron’s latest 9100, implemented as an NVMe protocol PCIe bus card, has a 120 microsecond read-access latency and a 30 microsecond write-access latency. That’s at least an order of magnitude faster than disk but 150 times slower than a DRAM access for write access and 600 times slower for a read access.
Why the asymmetry between read and write access? We understand it is because writes are cached in an SSD’s DRAM buffer whereas reads have to access, of course, the slower NAND.
Although NAND is much faster than disk it is still a great deal slower than memory and the essential coding rule of thumb that says IO is slower than a memory access still stands. It means coders, keen to shorten application run times, will try to restrict the amount of IO that’s carried out, and server/storage system engineers will try to use NAND caching to get over disk data delays, and DRAM caching to get over NAND access delays.
Servers with multi-socket, multi-core CPUs are inherently liable to be in IO wait states, as the storage subsystems, hundreds of times slower than memory at best, struggle to read and write data fast enough, meaning the processors wait, doing nothing, while turgidly slow storage does its thing.
If we treat an L1 cache reference as taking one second, then, on that measurement scale, a disk access can take six years, four months, 19 hours, 33 minutes, and 20 seconds. Reading from Micron’s 9100 SSD is much faster though, taking two days, 18 hours and 40 minutes. That’s good by disk standards, but it’s still tortuously slow by DRAM access speed standards – which take six mins and 40 secs.
This great gaping gulf in access speed between volatile memory and persistent storage promises to be narrowed by Intel and Micron’s 3D XPoint memory, a new form of non-volatile, solid state storage - if it lives up to expectations.
XPoint is made from Phase Change Memory material, a chalcogenide glass, and it has electricity applied to it, causing a bulk change in some aspect of its state that causes a resistance-level change. This can be reversed, leading the resistance levels signalling a binary 1 or zero. The XPoint die has two layers and a crossbar design, hence its full 3D CrossPoint name.