Semiconductor Engineering

ResNet-50 Does Not Predict Inference Throughput For MegaPixel Neural Network Models

Customers are considering applications for AI inference and want to evaluate multiple inference accelerators. As we discussed last month, TOPS do NOT correlate with inference throughput and you should ...
insideHPC

Intel Xeon Scalable Processors Set Deep Learning Performance Record on ResNet-50

Today Intel announced a deep learning performance record on image classification workloads. Intel was able to achieve 7878 images per second on ResNet-50 with its latest generation of Intel Xeon ...
insideHPC

NVIDIA DGX-2 AI Supercomputer Sets Records for Training and Inference

NVIDIA has one-upped performance breakthroughs announced earlier this year, thanks to two new machine learning technologies. Related to data augmentation and tensor parameter interleaving, these ...
Semiconductor Engineering

Do Large Batches Always Improve Neural Network Throughput?

Common benchmarks like ResNet-50 generally have much higher throughput with large batch sizes than with batch size =1. For example, the Nvidia Tesla T4 has 4x the throughput at batch=32 than when it ...