Depending on the application scenario, latency may or may not be an issue, accuracy is always an issue, and power consumption, particularly for cloud and edge applications, is critical. While training is regarded as a more HPC-related activity, inference is regarded as the enterprise workhorse. In this, the sixth set of MLPerf inference results, a new division (Inference over the network) and a new object detection model (RetinaNet) were added. Moreover, MLCommons (parent org.) keeps evolving its test suites to stay current and relevant. But that’s sort of the goal in the sense that MLPerf’s extensive benchmark suites and divisions (closed, open, etc.) permit more granular comparisons for system evaluators but it takes some effort. In some ways, making sense of MLPerf results has grown more complicated as the diversity of submitters and system sizes and system configurations have grown. ![]() ![]() AI start-ups Biren, Moffet AI, Neural Magic, Sapeon were among the entrants. Intel ran a Sapphire Rapids CPU-based system, albeit in the preview category. While Nvidia again dominated, showcasing its new H100 GPU, Qualcomm impressed on both performance and power consumption metrics. ![]() Twenty-one organizations submitted 5,300 performance results and 2,400 power measurement. The steady maturation of MLCommons/MLPerf as an AI benchmarking tool was apparent in today’s release of MLPerf v2.1 Inference results. Since 1987 - Covering the Fastest Computers in the World and the People Who Run Them
0 Comments
Leave a Reply. |