Performance
This page presents benchmark results comparing Cocoa with ADCIRC on a representative large-scale simulation, including strong-scaling behavior in CPU-MPI mode and single-GPU performance across three GPU generations.
Benchmark Configuration
The benchmark is a Hurricane Katrina (2005) hindcast with the following characteristics:
Mesh size: 1,568,749 nodes
Forcing: Tidal boundary, tidal potential, and hurricane wind and pressure fields
Time step: 2 s for both the implicit and explicit (lumped-mass) solvers, for both models
Simulated duration: 25 days
ADCIRC runs use double precision throughout. Cocoa computes in double precision with single-precision storage for bandwidth-bound fields.
Hardware:
CPU (PSC Bridges-2): dual-socket AMD EPYC 7742 64-core nodes (128 cores per node), 128 to 1024 MPI ranks (1 to 8 nodes)
GPU (Lambda Labs): NVIDIA H100 (SXM5, host Intel Platinum 8480+) and NVIDIA B200 (SXM6, host Intel Platinum 8592+), single device
GPU (workstation): NVIDIA V100 (host AMD Ryzen 9 9950X), single device
Both solvers use the same time step (2 s) and run for the same simulated duration, so the implicit and explicit results are a direct, step-for-step comparison. Results are still presented separately per solver because they exercise different work per step.
Note
The single-node, scaling and single-GPU results on this page were measured
in June 2026, before the first tagged release (0.4.2). They predate
element sorting for GPU memory locality (Element Ordering), the
0.6.0 multi-GPU work described under Multi-GPU Runs, and the 0.6.0
change of the default wetting test to ADCIRC’s linear form. Read them
as the relative standing of the two models, not as the timings of the
current release.
Implicit Solver Results
Fig. 12 Wall-clock time for the implicit solver (dt=2s) on the 1.57M-node Hurricane Katrina hindcast.
Model |
Hardware |
Wall Time |
|---|---|---|
ADCIRC |
128 cores (1 node) |
607m (10.1h) |
Cocoa |
128 cores (1 node) |
492m (8.2h) |
ADCIRC |
1024 cores (8 nodes) |
124m (2.1h) |
Cocoa |
1024 cores (8 nodes) |
142m (2.4h) |
Cocoa |
1x V100 GPU |
207m (3.5h) |
Cocoa |
1x H100 GPU |
82m (1.4h) |
Cocoa |
1x B200 GPU |
62m (1.0h) |
Explicit Solver Results
Fig. 13 Wall-clock time for the explicit lumped-mass solver (dt=2s) on the 1.57M-node Hurricane Katrina hindcast.
Model |
Hardware |
Wall Time |
|---|---|---|
ADCIRC |
128 cores (1 node) |
450m (7.5h) |
Cocoa |
128 cores (1 node) |
317m (5.3h) |
ADCIRC |
1024 cores (8 nodes) |
99m (1.6h) |
Cocoa |
1024 cores (8 nodes) |
78m (1.3h) |
Cocoa |
1x V100 GPU |
72m (1.2h) |
Cocoa |
1x H100 GPU |
35m (0.6h) |
Cocoa |
1x B200 GPU |
34m (0.6h) |
CPU-MPI Scaling
Fig. 14 Top: log-log scaling of wall-clock time vs MPI ranks on PSC Bridges-2, one panel per solver, with horizontal reference lines marking Cocoa’s single-GPU wall times for the same solver. Bottom: parallel efficiency relative to each configuration’s own 128-core run.
Wall-clock time in minutes by MPI rank count:
Ranks |
Cocoa Implicit |
ADCIRC Implicit |
Cocoa Explicit |
ADCIRC Explicit |
|---|---|---|---|---|
128 |
492 |
607 |
317 |
450 |
256 |
266 |
319 |
148 |
239 |
512 |
171 |
160 |
94 |
142 |
1024 |
142 |
124 |
78 |
99 |
Multi-GPU Runs
Each step of a multi-GPU run waits for the slowest GPU, so two inputs matter as much as the hardware: a mesh whose elements are sorted for GPU memory locality (see Element Ordering), and a partition that balances the wet nodes as well as the elements (see Balancing Wet Work). Together they keep two GPUs within about 2% of each other on the global STOFS mesh.
With both, a 6 h global STOFS run (12.8M nodes) with the implicit solver on two H100 GPUs takes 87.1 s in Cocoa 0.6.0, against 111.1 s before. Besides the balance, 0.6.0 builds the implicit solver’s matrix structure once at startup and reuses it, where each solve used to allocate and free device memory, and packs ghost exchanges straight from the field arrays into persistent buffers.
Quadrilateral and Hybrid Meshes: Zero Cost on Triangle-Only Runs
A mesh with zero quadrilaterals runs through the same device kernels, launch
counts, and allocations as it did before quadrilateral support existed.
Every quadrilateral code path is guarded by if (num_quads > 0) at the
launch site, never inside a shared kernel body: the quadrilateral tables
(ElementTables<Quadrilateral>) default-construct to zero-extent Views, no
quadrilateral compactor is constructed, and no quadrilateral kernel is
launched. This was checked by comparing the Kokkos kernel-timer launch list
and the Kokkos::View allocation labels on a triangle-only production mesh
(WNAT) before and after each change: identical names, identical counts.
Machine code and timing. An identical launch list does not prove the
triangle kernels themselves are unaffected, since a shared header touched to
add the element-type tag could still change triangle code generation, so
every change to shared code was checked two ways: the SASS of every kernel
was compared byte for byte against main, and two binaries were run
through the same production configuration under nsys profile with
interleaved, order-alternated repeats and every kernel’s median per-step
time compared at a 1% gate. On the gstofs production mesh (12.8M nodes)
every triangle kernel compiles to SASS byte-identical to main except the
GWCE right-hand-side and momentum assembly kernels, which carry the
per-element advection predicate; the per-step time is 0.2% below main
in both run orders and the two changed kernels sit within slot-order drift.