Fig. 1
From: How Amdahl’s Law limits the performance of large artificial neural networks
The \(R_{\mathrm{Max}}\) payload performance in function of the nominal performance \(R_{\mathrm{Peak}}\), at different \((1-\alpha _{\mathrm{eff}})\) values. The figures display the measured values derived using HPL (empty marks) and HPCG (filled marks) benchmarks, for the TOP15 supercomputers. The computing performance of AI applications may be similar to the diagram line marked by HPCG. The diagram lines marked as HPL and HPCG correspond to the behavior of supercomputer \({\it Taihulight}\) at \((1-\alpha _{\mathrm{eff}})\) values \(3.3*10^{-8}\) and \(2.4*10^{-5}\), respectively. The uncorrected values of the new supercomputers \({{ Summit}}\) and \({ Sierra}\) are shown as diamonds, and the same values corrected for single-processor performance are shown as rectangles. The black dots mark the performance data of supercomputers \({ JUQUEEN}\) and K as of 2014 June, for HPL and HPCG benchmarks, respectively. The red dot denotes the performance value of the system used by [4]. The saturation effect can be observed for both HPL and HPCG benchmarks. The shaded area only highlights the nonlinearity