Harnessing COTS modular potential for RF recording, playback, and analysis
Learn how M.2 NVMe-based PXIe SSD RAID modules exemplify the power of modular COTS products by enabling RF RPA developers, integrators, and users to eliminate external RAID subsystems to reduce costs and SWaP while improving performance.
RADXWideband RF RPA systems are widely used in commercial, aerospace, and defense markets. They digitize and store RF data in I/Q format (typically 16-bit/16-bit), support real-time and non-real-time analysis of the data, and, in some cases, convert it back to RF (playback) to stimulate devices under test (DUTs).
Wideband RF RPA systems are very data-transfer- and storage-intensive. A rule of thumb for data rate requirements for RF RPA systems is 5 bytes/second of I/Q data (16/16) per Hz of bandwidth. So, a single channel (1-CH), 500 MHz bandwidth wideband RF RPA system must sustain 2.5 GB/sec write and read speeds for the duration of the recording or playback session. At a 2.5 GB/sec data rate—which is about 10x the performance of a single SATA III 6gbps HDD—the data storage capacity requirements get very big, very quickly. The 500 MHz bandwidth 1-CH RF RPA system described above would need 2.5 GB/sec x 60 sec/min x 60 min/hour = 9 terabytes (TB) of storage to support just one hour of recording duration. The required data rates and capacities increase linearly with the number of channels and the bandwidth per channel in the system.
RAID is essential for wideband RF RPA
RADXTo sustain the multiple GB/sec data rates and multiple TB capacities required, RF RPA systems employ a redundant array of inexpensive disks (RAID) storage subsystems that combine multiple HDDs or SSDs operating in parallel to improve storage performance and capacity. With RAID level 0, data is striped across the drives, which produces a nearly linear increase in read/write performance and capacity. However, RAID level 0 can also be somewhat risky since the failure of a single drive can result in the complete loss of data. Other RAID levels (2, 5, 10, etc.) provide redundancy and error correction, albeit at the expense of throughput, capacity, SWaP, and additional cost.
Historically, the most popular interface for “inexpensive” HDDs and SSDs has been SATA III, which supports a 6 Gbps data rate. SATA III HDDs can typically RADXsupport a read/write data rate of about 200 MB/sec, while SATA III SSDs can typically support a rate of about 400 to 500 MB/sec. To achieve the 2.5 GB/sec RF RPA system data rate described earlier would notionally require a RAID 0 subsystem with 12 or more SATA III HDDs, or six or more SATA III SSDs. And, since most HDDs or SSDs are in a 2.5 or 3.5-inch form factor, a wideband RF RPA RAID subsystem would physically require a separate 1U to 4U rackmount subsystem that cables to the RF RPA system via a PCIe cable or other high-speed interface. For reference purposes, 2U RAID subsystems in the RF test & measurement market typically cost between $20k to $40k, consume about 200 W, and weigh between 35 and 40 pounds.
M.2 and NVMe
In late 2013, the PCI-SIG (the organization responsible for PCI Express) announced the PCIe M.2 Specification Revision 1.0. The M.2 Standard established the popular “stick of gum” format for low cost, high-performance NAND flash-based SSDs. M.2 SSDs support both SATA III 6 Gbps and direct PCIe Gen 3 x 4 interfaces, which supports up to about 4 GB/sec of bandwidth. The Non-volatile memory express (NVMe) standard establishes a set of protocols that run on top of PCIe that enable M.2 and other PCIe interface SSDs (e.g., 2.5-inch U.2 SSDs) to efficiently communicate with a host CPU. NVMe supports the essential services that the operating system, applications, and users need to effectively use and manage this new type of storage device. M.2 SSDs today support read/write speeds up to 3.5 GB/sec, capacities up to 2 TB for the most popular 22 mm W x 80 mm L (2280) size, low power consumption (typically under 10W per SSD,) and consumer-level pricing. Consequently, M.2 has rapidly become the most popular format SSD for new PCs, which bodes well for ongoing improvements in density and performance.
Growth of NAND Flash-based SSDs
Market Realist, via StatistaIn part due to the popularity of M.2 and NVMe, NAND flash-based solid-state drive (SSD) shipments have experienced extremely rapid growth over the past few years. SSD shipment growth is largely coupled to the rapid growth in mobile and cloud computing segments, both of which place a premium on performance (low latency and high throughput), size, weight, and power (SWaP). SSD adoption has driven SSD volumes which tends to reduce prices, thereby reinforcing the adoption cycle. Consequently, SSD shipments are poised to overtake hard disk drive (HDD) shipments by 2021, when SSD annual shipments are expected to exceed 350 million, according to 2017 report by Statista1.
PXIe-based RF RPA and SSD RAID modules
With its intrinsic scalability (to support multiple, wideband RF channels) and compact form factor, PXIe has become a very popular format for wideband RF RPA systems. With the advent of small, high data rate, low-power, low-cost M.2 SSDs that directly supports PCIe interface, it’s now possible to combine a 2, 4 or even 8 x M.2 SSDs with a PCIe switch to produce an extremely high-performance, high-capacity, cost-effective SSD PXIe RAID module that occupies a single PXIe slot. These “in-chassis” SSD RAID modules deliver storage capacities up to 16 TB per module with sequential read and write data rates of 5 GB/sec and up. SST RAID modules are priced between $7K to $14K, consume less than 50 W, and weigh less than a pound. One can install multiple SSD RAID modules to increase capacity and performance using software RAID. Accordingly, SSD RAID modules can effectively eliminate the need for external RAID subsystems for most applications—thereby reducing cost and SWaP—while delivering equal or greater performance.
There are three companies making COTS SSD RAID modules today, all of which employ multiple M.2 SSDs and support a PCIe Gen 3 x 8 interface. The National Instruments (NI) PXIe-8267 and the Conduant, Inc. DM-4M.2-3U are COTS, single-slot, PCIe Gen 3 x 8, SSD RAID modules with four (4) M.2 SSDs per module and published sequential write/read benchmark performance in the 5 to 7 GB/sec range. The RADX Trifecta-SSD supports 4 or 8 x 2 TB M.2 SSDs to provide 8 or 16 TB of storage per slot and 6.9 to 7.2 GB/sec sequential write/read (benchmark) performance.
|Manufacturer||National Instruments||Conduant Inc.||RADX|
|SSD RAID module||PXIe-8267||DM-4M.2-3U||PXIe-4M.2F-xTB|
|M.2 SSDs / module||4||4||4/8|
|Capacity||4 TB||4/8 TB||8 TB/16 TB|
|Write & read performance||5/5 GB/sec||5.8/7 GB/sec||6.9/7.2 GB/sec|
|For more info||https://bit.ly/2T94SDm||https://bit.ly/2xthGLv||https://bit.ly/308BWi2|
No free lunch
Sadly, there is no such thing as a free lunch in engineering. While SSDs often “feel” like fast HDDs, because of their underlying NAND flash architecture, they are fundamentally different and there are aspects of SSDs that must be actively considered when using them, especially in write-intensive wideband RF RPA applications:
- Endurance: NAND flash SSDs have a finite number of write cycles. It’s usually a high number, but one should be mindful of this feature when selecting an SSD-based storage solution. SSD manufacturers specify the endurance of an SSD in terms of terabytes written (TBW) or drive writes per day (DWPD). For example, a Samsung 970 EVO Plus 2 TB M.2 SSD supports a TBW of 1,200 TB. This means the SSD can be completely overwritten 1,200/2 = 600 times before suffering performance degradation for writes. The TBW scales with the number of SSDs in a given PXIe SSD RAID module, so a 16 TB SSD RAID module would support a TBW of 8 x 1,200 or 9,600 TB. DWPD is related to TBW but considers the warranty period of the SSD. Specifically, TBW = DWPD x warranty period.
- Clean Up (garbage collection) using SSD TRIM: Because NAND flash SSDs have limited write cycles, it’s important to treat them differently when deleting data and files. Operating systems like Microsoft Windows 10 that support NVMe “know” to use SSD “TRIM” as opposed to HDD “defragmentation” to minimize unnecessary SSD writes. Note that support for TRIM under MS Windows is limited to NTFS file systems. Also note that conventional RAID controllers (as opposed to PCIe switches) can interfere with TRIM command execution.
- Encryption: Besides being small and fast, M.2 and other NVMe SSDs are usually smart, too. Most recent M.2 and U.2 (2.5-inch NVMe format) SSDs support built-in AES-256 encryption using the Opal storage specification for self-encrypting drives (SED) developed by the Trusted Computing Group (see https://bit.ly/307IltF). If one enables Opal encryption, the data stored on the SSD or SSD module is encrypted with AES-256, which can significantly decrease the chance of unintended disclosure of sensitive data.
- Sanitizing SSDs: With frequent use, or for use in applications where the data is sensitive, one will eventually need to sanitize the SSD prior to disposal. NVMe SSDs support a “user data erase” command. Also, if one has enabled Opal encryption, one can also use “cryptographic erase,” which will make the data unusable since it destroys the keys. For government users, NIST recommends that one always verify after erasing. For more info on the NIST recommendations, visit https://bit.ly/2Nrol3I.
Because of the additional complexities associated with NAND flash-based devices, NVMe SSD device manufacturers publish free software tools that simplify SSD management. To check on the remaining life of the SSD, enable Opal encryption, or sanitize the SSD, one can either use operating system commands or utilities or use these tools from the manufacturer. Below are links to the tools from three of the leaders in M.2 SSDs: Intel, Samsung, and Micron:
- Intel: https://downloadcenter.intel.com/download/28111/Intel-Solid-State-Drive-Toolbox?v=t
- Micron: https://www.micron.com/products/solid-state-storage/storage-executive-software
- Samsung: https://www.samsung.com/semiconductor/minisite/ssd/download/tools/
NAND flash-based M.2 SSDs—when packaged as COTS, high-performance, single-slot, PXIe SSD RAID modules—enable wideband RF RPA developers, system integrators, and users to eliminate external RAID subsystems (and data recorders) to dramatically reduce costs and SWaP while improving performance and reducing complexity. And because COTS PXIe SSD RAID modules are based on commercial M.2 SSDs, one can fully expect their capacities and performance to improve over time, while prices should also continue to fall. Lastly, with standards like NVMe, new PXIe SSD RAID modules based on NVMe M.2 SSDs should be backward compatible with current modules, eliminating the need for expensive lifetime-buys. In short, M.2 NVMe-based PXIe SSD RAID modules exemplify the power of modular, COTS products.
Ross Q. Smith is co-founder and CEO of RADX Technologies, which manufactures LibertyGT modular, COTS real-time spectrum analyzers, wideband RF RPA systems, and Trifecta PXIe-SSD modules. Smith has a BSEE from UT Austin and has held co-founder and/or management roles at Ford Aerospace, MIPS, SGI, 3dfx, Quantum3D, Bruker, and GarrettCom.