Testing

Cocoa uses Catch2 for unit and integration testing, with Kokkos/Trilinos initialization handled in a custom main entry point.

Test Framework

Cocoa uses Catch2 (v3) as its testing framework. Tests are organized using TEST_CASE and SECTION macros:

#include <catch2/catch_test_macros.hpp>

TEST_CASE("Mesh", "[mesh]") {
  // Setup code runs for each section
  const auto mesh = create_test_mesh();

  SECTION("Valid mesh construction") {
    REQUIRE(mesh.num_nodes() == 9);
    REQUIRE(mesh.num_elements() == 8);
  }

  SECTION("Coordinate access") {
    const auto& nodes = mesh.nodes();
    REQUIRE_THAT(nodes[0].location().x(), WithinAbs(-88.0, 1e-10));
  }
}

The test executable uses a custom main() to initialize Tpetra/Kokkos before running tests:

#define CATCH_CONFIG_RUNNER
#include <catch2/catch_all.hpp>
#include "Tpetra_Core.hpp"

int main(int argc, char* argv[]) {
  Tpetra::ScopeGuard tpetra_scope_guard(&argc, &argv);
  return Catch::Session().run(argc, argv);
}

Building and Running Tests

# Build with tests (enabled by default)
mkdir build && cd build
cmake ..
make -j8

# Run all tests via CTest
ctest --output-on-failure

# Run the test executable directly
./test/cocoa_tests

# Run specific test cases by tag
./test/cocoa_tests "[mesh]"
./test/cocoa_tests "[gwce]"

# Run specific test case by name
./test/cocoa_tests "Mesh"

# List all available tests
./test/cocoa_tests --list-tests

Test Files

Test files are located in the test/ directory:

File	Description
`test_main.cpp`	Entry point with Tpetra/Kokkos initialization
`test_mesh.cpp`	Mesh construction and access
`test_neighbor_table.cpp`	Node connectivity and neighbor tables
`test_point2d.cpp`	2D point and geometry operations
`test_projection.cpp`	Coordinate projection (CPP)
`test_datetime.cpp`	DateTime and TimeDelta classes
`test_timestepper.cpp`	Time stepping iteration
`test_temporal_field.cpp`	Multi-level temporal field storage
`test_gwce_consistent.cpp`	GWCE consistent (implicit) solver and matrix assembly
`test_gwce_lumped.cpp`	GWCE lumped (explicit) solver
`test_wetdry.cpp`	Wetting and drying algorithm
`test_tide_potential.cpp`	Tidal potential forcing
`test_tidal_boundary.cpp`	Tidal boundary conditions
`test_momentum.cpp`	Momentum solver
`test_bottom_friction.cpp`	Bottom friction computation
`test_coriolis.cpp`	Coriolis force computation
`test_advection.cpp`	Advection terms
`test_pressure_gradient.cpp`	Pressure gradient computation
`test_inline_lateral_stress.cpp`	Inline lateral stress computation
`test_smagorinsky.cpp`	Smagorinsky turbulence model
`test_basis_functions.cpp`	Finite element basis functions
`test_coordinate_rotation.cpp`	Coordinate rotation for global meshes
`test_ramp_function.cpp`	Forcing ramp functions
`test_configuration_reader.cpp`	YAML configuration reader
`test_netcdf_io.cpp`	NetCDF I/O operations
`test_logger.cpp`	Logging system
`test_tide_astronomic.cpp`	Astronomical tide potential
`test_forcing_manager.cpp`	Forcing manager orchestration
`test_boundary_type.cpp`	Boundary type classification
`test_flow_boundary.cpp`	Flow boundary conditions (\(Q_{\text{force}}\) kernels, time series, segment integration)
`test_schema_validation.cpp`	YAML configuration schema validation
`test_distributed.cpp`	Distributed computing (serial mode)
`test_distributed_mpi.cpp`	MPI distributed computing (multi-rank)

Test Data

Test data and helper utilities are organized as follows:

test/
├── TestMeshData.hpp              # Test mesh generation utilities
├── TestPrecision.hpp             # Mixed-precision test tolerance helpers
├── test_*.cpp                    # Test source files
├── run_integration_test.cmake    # Integration test runner script
└── data/
    ├── test_config.yaml          # ConfigurationReader test fixture
    └── wnat/                     # WNAT integration test data
        ├── wnat.nc                             # Mesh file
        ├── cocoa_config_integration_test.yaml  # Tides-only config
        ├── compare_tides_reference.py          # Reference comparison script
        ├── plot_comparison.py                  # Comparison plot generator
        ├── wnat_reference_fp64.nc              # Serial reference
        └── wnat_reference_mpi_fp64.nc          # MPI reference

examples/
├── flow_channel/             # Specified flux (type 22) channel
├── flow_channel_radiation/   # Flux + radiation (type 32) channel
├── global_case/              # Global ocean tidal case
└── wnat/                     # WNAT example configurations and met data

See Test Cases for detailed descriptions of each test case, including expected results and verification plots.

The TestMeshData.hpp helper provides functions to create small test meshes:

#include "TestMeshData.hpp"

using TestData::TestMeshData;

TEST_CASE("Example", "[example]") {
  const auto [x_coords, y_coords] = TestMeshData::create_coordinates();
  const auto elevation = TestMeshData::create_elevation();
  const auto triangles = TestMeshData::create_triangles();
  const auto boundaries = TestMeshData::empty_boundaries();

  // Construct HostMesh (held by unique_ptr), then wrap in Mesh
  auto host_mesh = std::make_unique<Geometry::HostMesh>(
      x_coords, y_coords, elevation, triangles, boundaries,
      TestMeshData::origin, TestMeshData::projection_type);
  const Cocoa::Mesh mesh(std::move(host_mesh));
}

Catch2 Assertions

Common assertions used in Cocoa tests:

Basic Assertions:

REQUIRE(expr);           // Fatal assertion
CHECK(expr);             // Non-fatal assertion
REQUIRE_FALSE(expr);     // Require expression is false
REQUIRE_NOTHROW(expr);   // Require no exception thrown
REQUIRE_THROWS_AS(expr, ExceptionType);  // Require specific exception

Floating-Point Comparisons:

#include <catch2/catch_approx.hpp>
#include <catch2/matchers/catch_matchers_floating_point.hpp>

using Catch::Approx;
using Catch::Matchers::WithinAbs;
using Catch::Matchers::WithinRel;

REQUIRE(value == Approx(expected));              // Default tolerance
REQUIRE(value == Approx(expected).epsilon(1e-6)); // Custom tolerance
REQUIRE_THAT(value, WithinAbs(expected, 1e-10)); // Absolute tolerance
REQUIRE_THAT(value, WithinRel(expected, 1e-6)); // Relative tolerance

Sections for Test Organization:

TEST_CASE("Component", "[tag]") {
  // Setup code runs before each section
  auto component = create_component();

  SECTION("First behavior") {
    // Test first behavior
    REQUIRE(component.first_method() == expected);
  }

  SECTION("Second behavior") {
    // Test second behavior (fresh component)
    REQUIRE(component.second_method() == expected);
  }
}

Writing New Tests

Create or edit a test file in test/:

#include <catch2/catch_test_macros.hpp>
#include "your_header.hpp"

TEST_CASE("ComponentName", "[component][tag]") {
  SECTION("Descriptive behavior name") {
    // Arrange
    auto obj = create_object();

    // Act
    auto result = obj.method();

    // Assert
    REQUIRE(result == expected);
  }
}

Add the file to test/CMakeLists.txt if creating a new file:

set(COCOA_TEST_SOURCES
    test_main.cpp
    # ... existing files ...
    test_your_new_file.cpp)

Rebuild and run:

make cocoa_tests
./test/cocoa_tests "[your_tag]"

Compile-Time Tests

For constexpr functionality, use static_assert:

// Compile-time tests for constexpr functions
static_assert(TimeDelta::fromDays(1).totalHours() == 24,
              "Day to hours conversion failed");
static_assert(TimeDelta::fromHours(24) == TimeDelta::fromDays(1),
              "Equality comparison failed");

Integration Tests

In addition to unit tests, Cocoa includes end-to-end integration tests that run a complete simulation on the Western North Atlantic (WNAT) mesh and compare results against known-good reference solutions. These tests verify that the full simulation pipeline (mesh loading, tidal forcing with SAL, time stepping, output) produces bitwise-reproducible results.

Output is always delivered on a background writer thread, so every run exercises the asynchronous write path; there is no I/O mode to select.

Test Name	Ranks	Description
`Integration_WNAT_tides`	1	Serial simulation compared against serial reference
`Integration_WNAT_tides_mpi`	2	Two compute ranks (partitioned mesh, MPI-specific reference)

Each test runs a 1-day tides-only simulation (M2 boundary + tide potential + SAL) with \(\Delta t\) = 20 s and compares the output against the serial reference solution (wnat_reference_fp64.nc). The MPI test uses a separate reference (wnat_reference_mpi_fp64.nc) because mesh partitioning changes floating-point operation order.

All integration tests use a tolerance of 1e-10, enforcing near-exact agreement with the reference. The comparison script also verifies that wet/dry state (NaN locations) matches between test and reference at every timestep.

After comparison, a Cartopy-based plotting script generates a visual summary (wnat_comparison.png) with peak water level and velocity maps, difference maps, and time series at probe points. In CI, this plot is uploaded as a GitHub Actions artifact.

Prerequisites:

Integration tests require Python 3 with numpy, xarray, netCDF4, and cartopy. If these modules are not available, the integration tests are silently disabled at CMake configure time.

CMake options:

Integration tests are off by default and must be explicitly enabled:

# Enable integration tests
cmake .. -Dcocoa_ENABLE_INTEGRATION_TESTS=ON

In CI, integration tests are enabled for the serial build-and-test jobs and the coverage job. GPU compile-check jobs do not run tests.

Running integration tests selectively:

# Run only integration tests
ctest -R Integration

# Run only the serial integration test
ctest -R Integration_WNAT_tides$

# Run only the MPI integration test
ctest -R Integration_WNAT_tides_mpi

# Exclude integration tests (run unit tests only)
ctest -E Integration

Regenerating reference solutions:

If numerical methods change and produce different (but correct) results, the reference solutions must be regenerated:

cd test/data/wnat

# Serial reference
/path/to/cocoa -i cocoa_config_integration_test.yaml
mv cocoa_output.nc wnat_reference_fp64.nc

# MPI reference (2 compute ranks)
mpiexec -np 2 /path/to/cocoa -i cocoa_config_integration_test.yaml
mv cocoa_output.nc wnat_reference_mpi_fp64.nc

Debugging Failed Tests

# Run with verbose output
./test/cocoa_tests -s "[failing_tag]"

# Run under debugger
lldb ./test/cocoa_tests -- "[failing_tag]"

# Run specific named test
./test/cocoa_tests "Exact Test Name"

# Show test durations
./test/cocoa_tests -d yes