- Solidigm 122.88TB SSD provided the storage for a test involving Nvidia’s Nano Super
- The system was used to run DeepSeek and although it worked, it wasn’t fast
- The Gen 4 PCIe SSD’s speed was restricted by the Nano Super’s Gen 3 connection
At the end of 2024, Solidigm added a 122.88TB QLC SSD to its product line. The D5-P5336 will be available in U.2 15mm to start and then in E1.L later in 2025, meaning it won’t fit in a typical consumer PC. Its price is expected to exceed $10,000 anyway, so you’d need deep pockets if you want to buy one.
If you’re wondering how such a giant-capacity SSD might perform, we have the answer – sort of – but it doesn’t come in the form of a traditional review.
StorageReview tested the Jetson Orin Nano Super – Nvidia’s compact AI single-board computer for edge computing – to see how it performed on AI development tasks, specifically LLM inference. The Nano Super comes with a 6-core Arm CPU, a 1024-core Ampere GPU, and 8GB of LPDDR5 memory. At $249, it is an affordable choice for AI developers, but its limited VRAM presents a challenge for running LLMs.
Not smooth sailing
“We recognized that onboard memory limitations challenge running models with billions of parameters, so we implemented an innovative approach to bypass these constraints,” the site explained. “Typically, the Nano Super’s 8GB of graphics memory restricts its capability to smaller models, but we aimed to run a model 45 times larger than what would traditionally fit.”
Doing this involved upgrading the Nano Super’s storage with Solidigm’s new U.2 drive, which has a Gen 4 PCIe x4 interface and promises sequential read/write speeds of up to 7.1 GB/s (read) and 3.3 GB/s (write), along with random performance of up to 1,269,000 IOPS.
The Nano Super has two M.2 NVMe bays, both of which offer a PCIe Gen3 connection. The team connected the SSD to an 80mm slot supporting a full four PCIe lanes using a breakout cable to get the most bandwidth and used an ATX power supply to deliver 12V and 3.3V to the SSD.
While the full potential of the drive was limited by the Jetson’s interface, it still managed up to 2.5GB/s of read speeds. Using AirLLM, which loads model layers dynamically rather than all at once, the site managed to run DeepSeek R1 70B Distilled, an AI model 45 times larger than what would traditionally fit on such a device.
Processing speed turned out to be a major bottleneck for the experiment. Running smaller models worked well, but generating a single token from the 70B model took 4.5 minutes. While not practical for real-time AI tasks, the test demonstrated how massive storage solutions, like the D5-P5336, can enable larger models in constrained environments.
You can see how the test was achieved, and the problems that were encountered and overcome along the way, in this YouTube video.
You might also like
https://cdn.mos.cms.futurecdn.net/FowG3FAppNQh4Y3PuRZrGD-1200-80.jpg
Source link
waynewilliams@onmail.com (Wayne Williams)
