设计工具
存储

英伟达 GTC 2024的三个要点

阿尔瓦罗·托莱多| 2024年4月

英伟达’s GPU Technology Conference (GTC) is one of the most anticipated events in the AI industry, 展示GPU计算的最新创新和趋势, 深度学习, 计算机视觉, 和更多的. 今年, GTC 2024于3月18日至21日举行, 有超过900个会议, 主题演讲, 演示, 以及该领域专家和领导者的研讨会. 作为英伟达的骄傲合作伙伴, 美光公司也在现场, highlighting how our memory and storage solutions are enabling the next generation of AI platforms and applications. Here are three key observations from GTC 2024 that demonstrate the importance of memory and storage in the AI ecosystem.
 

1. 内存和存储是人工智能平台的关键推动者
 

因为数据对人工智能至关重要, a key theme of GTC 2024 was the increasing demand for memory and storage performance and capacity in AI workloads. 随着人工智能模型的规模和复杂性的增长, they require more data to be processed and stored at 快 speeds and lower latencies. 这对传统的内存和存储架构提出了挑战, 哪些会成为人工智能性能和效率的瓶颈. 为了应对这一挑战, 微米 showcased its portfolio of memory and storage solutions that are designed to optimize the data flow and availability for AI platforms. 这些包括:

  • The industry-leading performance and power efficiency of 微米’s HBM3E 8-high 24GB and 12-high 36GB cubes. 微米 HBM3E 8-high 24GB is now in volume production and will be part of 英伟达’s H200 GPU shipping in Q2 of calendar year 2024
  • 微米 CZ120 CXL™内存模块，提供所需的容量, 带宽, 以及加速人工智能和内存工作负载的灵活性
  • 微米 9000 and 7000 series ssd that support up to 17 gpu in the 3D Medical Imaging benchmark Unet3D
  • 微米 6000 series ssd that boost AI data lake ingest by up to 48% when compared to our competitor’s capacity-focused ssd1
  • Real-world lab results demonstrating how 微米’s technologies improve AI training and inference including LLMs, 计算机成像, GNN, 和更多的

 


By leveraging these memory and storage solutions throughout the entire data stack (near memory, 主内存, 扩展内存, 固态硬盘数据缓存和网络数据湖), 美光正在帮助加速人工智能革命, 使人工智能平台能够处理更多数据, 快, 更有效率.
 

2. 人工智能是PCIe的杀手级用例® Gen5
 

Another highlight of GTC 2024 was the introduction of the new 英伟达 B100 accelerators, 基于PCIe Gen5接口. The PCI Express standard is the most widely used interface for connecting high-performance CPUs, gpu, ssd, 还有网卡. PCIe Gen5 offers double the 带宽 of Gen4, enabling up to 32GT/s of data transfer per lane. This is a game-changer for AI workloads, which can benefit from the increased data throughput.

然而, 充分挖掘PCIe Gen5在数据中心的潜力, the devices that are connected to the interface must also be able to support the higher speed and lower latency. 这就是美光的PCIe Gen5固态硬盘的用武之地.

英伟达, Dell and 微米 recently collaborated to showcase the benefits of 微米’s PCIe Gen5 固态硬盘, 大型加速器存储器2 (BaM)、英伟达 H100加速器和PowerEdge服务器. The demonstration showed a 50% reduction in graph neural network training time when offloading the GNN training model to a PCIe Gen5 high-performance 固态硬盘 vs. Gen4.
 


This test also showed a 5x performance improvement of 英伟达 H100 (Gen5) over A100 (Gen4). GPU性能提升了5倍, 存储设备需要快速发展才能跟上. A typical 深度学习 recommendation (DLRM) workload will result in queue depths of 10 to 100, 读取128K到512K的块. On a Gen5 固态硬盘 this will typically hit the maximum drive throughput of ~14GB/s. 随着AI模型的卸载，小块性能变得至关重要. 上面详细介绍的GNN演示在队列深度超过1的情况下读取4K块,000, easily reaching the maximum random read throughput of the fastest PCIe Gen5 固态硬盘.
 


在美光的PCIe Gen5技术演示中, 我们不仅展示了14GB/s的顺序吞吐量, 也可以随机读取3,300,000 IOPS. 在4K工作负载中，转换成吞吐量为13.2GB/s, between 22% and 32% 快 than competitive offerings in the market today.
 


通过提供如此高的性能和效率, 微米's PCIe Gen5 ssd can enable AI platforms to leverage the full power of the new 英伟达 accelerators, resulting in 快 outcomes and a better return on investment on your AI hardware purchases.
 

3. 网络数据湖越来越多地部署在大容量ssd上
 

A third observation from GTC 2024 was the growing trend of deploying networked data lakes on ssd, 而不是硬盘, to store and access the massive amounts of data that are generated and consumed by AI applications. Networked data lakes are large and distributed repositories of data that are connected to the AI platform via a network, 如InfiniBand或以太网. While networked data lakes provide scalable and flexible storage capacity for AI data, 支持跨不同平台和用户的数据共享和协作, 它们还在数据传输速度和密度方面带来了挑战, 这对总拥有成本(TCO)的计算有很大的影响.

为了克服这些挑战, many AI users and developers are opting for high capacity ssd such as the 微米 6500 ION, 而不是硬盘, 建立和运营他们的网络数据湖. This class of ssd offers several advantages over HDDs for networked data lakes, such as:

  • 更快的数据摄取和处理速度, 如何减少人工智能模型训练和推理的时间和成本. PCIe Gen4固态硬盘最多可提供6个容量.8GB/s的顺序读性能和超过5.7GB/s的顺序写性能1, 比hdd快得多, which can only deliver less than 300 MB/s of sequential read and write performance3. This means that these high capacity ssd can process data over 22 times 快 than HDDs, 哪些可以显著加快人工智能的工作流程和结果.
  • 更高的数据密度和更低的功耗, 如何降低TCO，提高网络化数据湖的效率. ssd可以存储30个.每个驱动器72TB的数据密度为4.8TB per cubic inch, which is nearly five times more than the densest 24TB nearline HDDs today4. 这意味着ssd可以在更少的空间中存储更多的数据, which can reduce the hardware and infrastructure costs of networked data lakes.

 


总之, GTC 2024 was a remarkable event that showcased the latest innovations and trends in the AI industry, 以及美光的内存和存储解决方案如何推动人工智能革命. 我们很自豪能成为英伟达的合作伙伴, and we look forward to continuing our collaboration and contribution to the AI ecosystem.

 

1 美光®6500 ION NVMe 固态硬盘沙巴体育结算平台简介

2 http://arxiv.org/pdf/2203.04910.pdf

3 exos-x24-dsm |希捷美国

4 2024 Predictions in storage, technology and the world part 2: bigger and 快

VP & 总经理，数据中心和存储

阿尔瓦罗·托莱多

Alvaro is Vice President and General Manager of Data Center 存储 at 微米. 他负责战略, 沙巴体育结算平台和技术路线图, 技术客户参与, 损益(P&L)用于数据中心存储.

Alvaro earned a bachelor’s degree in computer science from National University and an MBA from the Haas School of Business at the University of California, 加州大学伯克利分校.