Repository navigation
HF/6-31G* full Fock build takes ~90 s for a ~2000-function system on a V100. Is this expected? #510
Replies: 1 comment
|
Thanks for the detailed report and for sharing the inputs, outputs, build information, and comparison data. This is very helpful. A few comments from our side. CUDA versionThe first thing that stands out is that you are running on a V100 with CUDA 12.9. We have observed significant performance regressions with newer CUDA releases on older GPU architectures. In particular, the QUICK 26.03 release notes recommend:
Unfortunately, the performance degradation can be quite substantial and has been difficult for us to fully understand. Before investigating the benchmark itself in detail, please repeat the test with CUDA 12.4.x (or earlier) and compare the timings. You can use a new driver, you just need an older CUDA toolkit to compile and run the code. We should probably make this recommendation more prominent in the compilation guide and known-issues documentation. Expected performanceThe timings you report (~90 s per full SCF iteration) are much slower than benchmark data we have collected on V100-class hardware. For reference, benchmark inputs, outputs, and timing data are available in the QUICK benchmark repository: https://github.com/merzlab/QUICK-tests Some representative single-GPU results are:
Early SCF iteration is before difference density kicks in to reduce computational effort. These calculations include DFT quadrature costs, although the majority of the SCF time is still spent in ERI evaluation. Obviously these are not identical systems, but based on our benchmark experience, ~90 s per iteration for a ~2000-function system on a V100 seems unexpectedly slow. Benchmark systemI have not yet looked at the attached gly30 files in detail. If this is the linear polyglycine benchmark commonly used in the literature, one might actually expect Schwarz screening to be relatively effective because the system is spatially extended rather than a compact 3D structure. I would expect timings to be lower than the 3D structures we used to benchmark. We have not historically used glycine chains as one of our standard benchmark systems, so I do not currently have a direct reference timing for gly30. Published QUICK timing in the GPU4PySCF paperRegarding the 44.1 s A100 timing cited from the GPU4PySCF paper, we have not independently reproduced that benchmark. Since gly30 is not currently part of our benchmark suite, it would be worthwhile for us to revisit that comparison and verify the reported QUICK timing. Additional commentsOne thing worth keeping in mind is that we have updated various cutoffs and thresholds in recent QUICK releases to improve numerical accuracy and SCF robustness. Depending on which version is being compared, some differences in timings may therefore be expected. It is also worth noting that QUICK is generally very efficient for low-angular-momentum ERIs, while higher-angular-momentum integrals (d, f, and beyond) tend to be increasingly memory-bandwidth limited. You will notice that if you move to benchmarks with cc-pVTZ. This requires some work on our end. Next stepsMy suggestion would be:
Thanks again for reporting this and for providing enough detail to make the comparison meaningful. |
Uh oh!
There was an error while loading. Please reload this page.
Title: HF/6-31G* full Fock build takes ~90 s for a ~2000-function system on a V100. Is this expected?
Hi all,
I'm benchmarking several GPU Hartree–Fock programs on the same hardware for a manuscript, and QUICK is one of them. I want to make sure we represent it fairly. Our timings look much slower than we expected, so I'd appreciate a check on whether our build or input is missing something.
What we measure
Restricted HF energies of linear polyglycine chains and water clusters (STO-3G to cc-pVTZ). Each program starts from its own guess and converges to about 1e-6. Every program does a full (non-incremental) Fock build each iteration, so build costs are comparable across codes. That's why we set
NCYC=1000; everything else is at QUICK's defaults.Setup
f33543c(version 26.3), run as serialquick.cudaon one GPU-gencode arch=compute_70,code=sm_70 -O2 --use_fast_math.Input
gly30, 6-31G*, 1999 Cartesian basis functions:
What we see
QUICK finds the V100 and converges correctly in 12 cycles, to within 3.5e-6 Eh of a tight reference. However:
NCYC): the total only drops to 822 s, with the later cycles still taking 52–87 s.For comparison, on the same GPU GPU4PySCF 1.8.1 converges the same system in 10 iterations and 16.2 s total. Its settings: Cartesian functions,
direct_scf_tol=1e-10,conv_tol=1e-6, MINAO guess, and a full build every iteration. The GPU4PySCF paper (arXiv:2407.09700) also reports QUICK at 44.1 s for 10 SCF iterations of gly30/6-31G(d) on an A100. That is far from the ~90 s per build we measure, even allowing for the different GPU.What we ruled out
Three builds on the same input give essentially the same time per full build:
f33543c-DUSE_LEGACY_ATOMICS(thevolta-only build leaves it out)The GPU runs at normal clocks, and other GPU programs perform as expected on the same card.
Questions
Attached (gly30_data.zip)
gly30.xyzgly30_631gs_fullbuild.in/.out: our benchmark settinggly30_631gs_default.in/.out: QUICK defaultsgly30_631gs_fullbuild_quick-23.08b_4cycles.out: the 23.08b buildThanks a lot for your time!
Miguel Alonso Mediavilla, Department of Chemistry, University of Copenhagen
All reactions