================ Docker Container ================ Cocoa is distributed as Docker containers pre-built for a range of GPU and CPU architectures. This is the easiest way to get started without compiling from source. .. note:: Cocoa computes in double precision; a few bandwidth-sensitive fields are stored in single precision internally (see :doc:`../theory/numerical_methods`). This is built in and requires no configuration, so there is a single build per architecture rather than separate precision variants. Image Families and Naming ------------------------- Cocoa images are split into three families by accelerator toolchain, identified by the tag suffix. ```` is the Cocoa release (for example ``latest`` or a version such as ``1.0``): .. list-table:: :header-rows: 1 :widths: 30 25 45 * - Image - Hardware - Base * - ``zcobell/cocoa:-cpu`` - CPU only (x86-64 and ARM64) - Debian 12 * - ``zcobell/cocoa:-cuda`` - NVIDIA GPUs (CUDA) - ``nvidia/cuda`` (Ubuntu) * - ``zcobell/cocoa:-rocm`` - AMD GPUs (ROCm/HIP) - ``rocm/dev-ubuntu`` The families are kept separate because the CUDA and ROCm toolkits are mutually exclusive at the hardware level and each is large; a host is either NVIDIA or AMD, so a combined image would only add bloat. Splitting the CPU build out of the GPU images also keeps CPU and CI users from pulling a multi-gigabyte GPU toolkit they will never use. The ``-cpu`` image is a single **multi-architecture** tag: it carries both ``linux/amd64`` and ``linux/arm64`` variants behind one name, and ``docker pull`` automatically selects the one matching your host. The ARM64 variant runs natively on Apple Silicon Macs through Docker Desktop, as well as on AWS Graviton, Ampere Altra, and NVIDIA Grace. The ``-cuda`` and ``-rocm`` images are **GPU-only**: they do not include a CPU fallback build. Use the ``-cpu`` image to run on a node without a supported GPU. The Trilinos base images follow the same convention: ``zcobell/trilinos_base:-cpu``, ``-cuda``, and ``-rocm``. Supported Architectures ----------------------- **CPU (``-cpu`` image, x86-64 and ARM64):** .. list-table:: :header-rows: 1 :widths: 20 30 35 * - Architecture - Backend - ISA tuning * - ``serial`` - Single-threaded - Haswell (x86-64) / Neoverse-N1 (ARM64) * - ``openmp`` - Multi-threaded - Haswell (x86-64) / Neoverse-N1 (ARM64) **NVIDIA GPUs (``-cuda`` image):** .. list-table:: :header-rows: 1 :widths: 20 30 30 * - Architecture - Hardware - Compute Capability * - ``volta70`` - NVIDIA Volta (V100) - 7.0 * - ``turing75`` - NVIDIA Turing (T4) - 7.5 * - ``ampere80`` - NVIDIA Ampere (A100) - 8.0 * - ``ampere86`` - NVIDIA Ampere (A10) - 8.6 * - ``ada89`` - NVIDIA Ada (L40S) - 8.9 * - ``hopper90`` - NVIDIA Hopper (H100) - 9.0 * - ``blackwell100`` - NVIDIA Blackwell (B100) - 10.0 **AMD GPUs (``-rocm`` image):** .. list-table:: :header-rows: 1 :widths: 20 30 30 * - Architecture - Hardware - GFX ISA * - ``mi300`` - AMD Instinct MI300X / MI300A - gfx942 * - ``mi200`` - AMD Instinct MI210 / MI250 / MI250X - gfx90a Within an image, the variant name is simply the architecture name, e.g., ``ampere80``, ``mi300``, or ``openmp``. Running the Container --------------------- **CPU (x86-64 or ARM64, auto-selected):** .. code-block:: bash docker run -it -v $(pwd):/workspace zcobell/cocoa:latest-cpu **NVIDIA GPU (requires the NVIDIA Container Toolkit):** .. code-block:: bash docker run -it --gpus all -v $(pwd):/workspace zcobell/cocoa:latest-cuda **AMD GPU (requires the ROCm kernel driver on the host):** .. code-block:: bash docker run -it --device=/dev/kfd --device=/dev/dri \ --group-add video --security-opt seccomp=unconfined \ -v $(pwd):/workspace zcobell/cocoa:latest-rocm The ``-v $(pwd):/workspace`` flag mounts your current directory into the container's working directory so Cocoa can access your mesh and configuration files. Selecting a Variant ------------------- Each image bundles the variants for its own family and selects a sensible default: the ``-cpu`` image defaults to ``serial``, while the GPU images default to a representative architecture (``ampere80`` for ``-cuda``, ``mi200`` for ``-rocm``). Use the ``select_cocoa`` command to switch at runtime: .. code-block:: bash # List the variants available in this image source select_cocoa --help # NVIDIA: select the A100 backend (in the -cuda image) source select_cocoa ampere80 # AMD: select the MI300 backend (in the -rocm image) source select_cocoa mi300 # CPU: select the multi-threaded build (in the -cpu image) source select_cocoa openmp # Verify selection which cocoa A variant exists only in the image for its family -- for example ``ampere80`` is present only in ``-cuda`` and ``mi300`` only in ``-rocm``. The selection persists for the duration of the shell session. To set it at launch, pass the ``COCOA_ARCH`` environment variable: .. code-block:: bash # Run on an A100 docker run -it --gpus all -e COCOA_ARCH=ampere80 \ -v $(pwd):/workspace zcobell/cocoa:latest-cuda Running a Simulation -------------------- Once inside the container with the appropriate architecture selected: .. code-block:: bash cocoa -i your_config.yaml An example simulation is included in the container at ``/opt/cocoa/examples``: .. code-block:: bash cp -r /opt/cocoa/examples/* . source select_cocoa serial cocoa -i simple.yaml See :doc:`quick_start` for details on configuration files and expected output. Converting ADCIRC Meshes ------------------------ The container includes the ``convert_adcirc_format.py`` utility for converting ADCIRC model files to Cocoa's NetCDF mesh format. Python 3 with ``netCDF4`` and ``numpy`` are pre-installed. **Basic mesh conversion (fort.14 only):** .. code-block:: bash python3 /opt/cocoa/utils/convert_adcirc_format.py \ --mesh fort.14 \ --output mesh.nc **With nodal attributes (fort.13):** .. code-block:: bash python3 /opt/cocoa/utils/convert_adcirc_format.py \ --mesh fort.14 \ --attributes fort.13 \ --output mesh.nc **With self-attraction and loading (fort.24):** .. code-block:: bash python3 /opt/cocoa/utils/convert_adcirc_format.py \ --mesh fort.14 \ --attributes fort.13 \ --sal fort.24 \ --output mesh.nc .. list-table:: Conversion Script Options :header-rows: 1 :widths: 25 15 60 * - Flag - Required - Description * - ``--mesh`` - Yes - Path to ADCIRC fort.14 mesh file * - ``--output`` - Yes - Path for output NetCDF file * - ``--attributes`` - No - Path to ADCIRC fort.13 nodal attributes file * - ``--sal`` - No - Path to self-attraction/loading file (fort.24 ASCII or NetCDF) See :doc:`../user_guide/mesh_preparation` for details on the NetCDF mesh format and supported nodal attributes. Mounting Data Volumes --------------------- Mount your simulation directory into the container so input files are accessible and output files persist after the container exits: .. code-block:: bash # Mount a single directory docker run -it --gpus all \ -v /path/to/simulation:/workspace \ zcobell/cocoa:latest-cuda # Mount input and output separately docker run -it --gpus all \ -v /path/to/meshes:/data/meshes:ro \ -v /path/to/output:/workspace \ zcobell/cocoa:latest-cuda .. tip:: Use ``:ro`` (read-only) for input data mounts to prevent accidental modification of source files. Non-Interactive Execution ------------------------- Run a simulation without entering the container interactively: .. code-block:: bash docker run --gpus all \ -v $(pwd):/workspace \ -e COCOA_ARCH=ampere80 \ zcobell/cocoa:latest-cuda \ cocoa -i config.yaml Building the Container ---------------------- Each family is built in two stages: first the Trilinos base image, then the Cocoa image on top of it. The three families share one build context per image and select the family with a per-family Dockerfile (``Dockerfile.cpu``, ``Dockerfile.cuda``, ``Dockerfile.rocm``), so the build scripts and entrypoints are not duplicated. Substitute the family suffix throughout. **1. Build the Trilinos base image** (example: CUDA): .. code-block:: bash cd containers/base_trilinos_container docker build -f Dockerfile.cuda -t zcobell/trilinos_base:latest-cuda . **2. Build the Cocoa image:** .. code-block:: bash cd containers/cocoa_container DOCKER_BUILDKIT=1 docker build --ssh default \ -f Dockerfile.cuda -t zcobell/cocoa:latest-cuda . The ``--ssh default`` flag forwards your SSH agent for private repository access during the build. Ensure your SSH agent is running with the appropriate key loaded (``ssh-add``). The ``-cpu`` family is multi-architecture. Build and push both platform variants under one tag with ``buildx``: .. code-block:: bash cd containers/cocoa_container docker buildx build --platform linux/amd64,linux/arm64 \ -f Dockerfile.cpu -t zcobell/cocoa:latest-cpu --push . On a cluster the build is driven by the SLURM batch scripts in ``containers/slurm/``. ``submit_all.sh`` queues every family, making each Cocoa image depend on its Trilinos base; see those scripts for the exact ``buildx`` invocation and push steps. .. note:: Building a GPU family compiles Trilinos and Cocoa once per architecture in that family, which is resource-intensive and may take several hours. The AMD (``-rocm``) images are currently validated by compilation; runtime validation on AMD hardware is ongoing. The ARM64 half of the ``-cpu`` build runs natively on an ARM64 builder or, more slowly, under ``qemu`` emulation.