============ Output Files ============ Output Configuration -------------------- Configure output in the YAML file: .. code-block:: yaml output: global: step_interval: 1h # Output every hour (or integer step count) filename: cocoa_output.nc # Output filename (optional) mode: split # "split" (default) or "combined" variables: # Selectively disable time-series variables pressure: false wind: false ``output.global`` configures the global output file, which holds every mesh node. By default it is named ``cocoa_output.nc`` in the working directory; use the ``filename`` parameter to change it. Point time series go to their own file, configured under ``output.stations`` (see :ref:`station-output`). ``output.compression`` applies to both. **output.global parameters:** .. list-table:: :header-rows: 1 :widths: 25 15 15 45 * - Parameter - Type - Default - Description * - ``step_interval`` - int or duration - 100 - Interval between output writes. Accepts a plain integer (number of time steps) or a duration string with suffix: ``s``, ``m``, ``h``, ``d`` (e.g., ``15m``, ``1h``). See :ref:`duration-intervals`. * - ``start_time`` - datetime - simulation start - First time-series slice. See :ref:`output-time-window`. * - ``end_time`` - datetime - simulation end - Last time a time-series slice may be written. See :ref:`output-time-window`. * - ``filename`` - string - ``cocoa_output.nc`` - Output NetCDF filename * - ``mode`` - string - ``split`` - Output mode for peak variables. ``split`` writes peak data to a separate file; ``combined`` writes everything to one file. See :ref:`output-mode`. * - ``variables`` - map - *(all enabled)* - Selectively disable time-series variables. See :ref:`output-variable-selection`. .. _output-time-window: Time Window ----------- ``start_time`` and ``end_time`` limit time-series output to part of the run: .. code-block:: yaml output: global: step_interval: 1h start_time: 2005-08-28T00:00:00 end_time: 2005-08-30T00:00:00 Slices are written at ``start_time``, ``start_time + step_interval``, and so on, up to and including ``end_time``. The schedule starts at ``start_time``, not at the simulation start: with a 1 h interval and ``start_time`` at 00:30, slices fall on the half hour. Either bound may be omitted, and either may lie outside the simulation period. Both must be a whole number of time steps from ``simulation.start_time``. The window applies to time-series variables only. Peak variables still cover the whole run, and checkpoints are written on their own schedule. If the window contains no output time within the run, Cocoa logs a warning and the file holds the mesh and peak variables with no time slices. On a restart the schedule continues where it left off, because it is counted from the original cold start. .. _output-compression: Compression ----------- Every output file, the station file included, uses NetCDF-4 deflate compression with one shared level and HDF5 byte-shuffle setting: .. code-block:: yaml output: compression: level: 1 # Deflate level 0-9; 0 disables compression entirely shuffle: false # Byte-shuffle filter (small ratio gain, slower writes) Compression runs on the background writer thread, one snapshot at a time. When output is frequent enough that compressing a snapshot takes longer than the simulation takes to reach the next output step, the writer falls behind and the simulation stalls waiting for it. If frequent output is slowing the model down, reduce ``level``, or set ``level: 0`` to write uncompressed files with no compression cost. Files remain valid NetCDF-4 at every setting, and readers do not need any configuration to match. .. _output-variable-selection: Variable Selection ------------------ By default, all available time-series variables are written except the forcing diagnostics ``tide_potential``, ``bpg_effective`` and ``water_level_correction``, which are opt-in. You can selectively disable individual variables (or enable the opt-in ones) using the ``variables`` section: .. code-block:: yaml output: global: variables: pressure: false # Disable atmospheric pressure output wind: false # Disable wind velocity output tide_potential: true # Enable tidal potential output (off by default) bpg_effective: true # Enable effective BPG output (off by default) **Rules:** - Except for the opt-in diagnostics, only variables explicitly set to ``false`` are disabled. Setting such a variable to ``true`` or omitting it has the same effect: it remains enabled. - ``tide_potential``, ``bpg_effective`` and ``water_level_correction`` are written only when explicitly set to ``true``. - Only time-series variables can be disabled. Peak variables are always written regardless of this setting. - Meteorological variables (``pressure``, ``wind``) are only available when meteorological forcing is enabled, and ``ice_concentration`` only when ice is enabled and a meteorological source carries it (see :ref:`ice-concentration`). - The ``tide_potential`` variable is only available when tidal potential forcing is enabled, the ``bpg_effective`` pair only when baroclinic forcing is enabled, and ``water_level_correction`` only when the data assimilation correction is enabled. **Available time-series variables:** .. list-table:: :header-rows: 1 :widths: 20 20 50 * - YAML Name - NetCDF Variable(s) - Description * - ``zeta`` - ``zeta`` - Water surface elevation (m) * - ``velocity`` - ``ucx``, ``ucy`` - Eastward and northward velocity (m/s) * - ``pressure`` - ``pressure`` - Atmospheric pressure at mean sea level (Pa). Requires meteorological forcing. * - ``wind`` - ``wind_u``, ``wind_v`` - Eastward and northward wind velocity (m/s). Requires meteorological forcing. * - ``ice_concentration`` - ``ice_concentration`` - Sea ice area fraction (0-1). Requires a meteorological source that carries ice. * - ``tide_potential`` - ``tide_potential`` - Tidal potential displacement (m). Requires tidal potential forcing. Off by default; set to ``true`` to write it. * - ``bpg_effective`` - ``bpg_effective_x``, ``bpg_effective_y`` - Eastward and northward effective baroclinic pressure gradient (m/s2), the file value plus the free-surface correction. Requires baroclinic forcing. Off by default; set to ``true`` to write it. * - ``water_level_correction`` - ``water_level_correction`` - The data assimilation water level correction the model applies (m), after the ramp and multiplier. Requires ``forcing.water_level_correction``. Off by default; set to ``true`` to write it. Output Variables ---------------- Time-Series Variables ^^^^^^^^^^^^^^^^^^^^^ Time-varying variables are written at each output interval. Dry nodes are masked using a fill value of ``-99999.0``. .. list-table:: :header-rows: 1 * - Variable - Description - Units - Dimensions * - ``zeta`` - Water surface elevation - meters - (time, node) * - ``ucx`` - Eastward velocity component - m/s - (time, node) * - ``ucy`` - Northward velocity component - m/s - (time, node) * - ``pressure`` - Atmospheric pressure at mean sea level - Pa - (time, node) * - ``wind_u`` - Eastward wind velocity component - m/s - (time, node) * - ``wind_v`` - Northward wind velocity component - m/s - (time, node) * - ``ice_concentration`` - Sea ice area fraction - 1 - (time, node) * - ``tide_potential`` - Tidal potential displacement - m - (time, node) * - ``bpg_effective_x`` - Eastward effective baroclinic pressure gradient - m s-2 - (time, node) * - ``bpg_effective_y`` - Northward effective baroclinic pressure gradient - m s-2 - (time, node) * - ``water_level_correction`` - Data assimilation water level correction applied - m - (time, node) Which of these are present depends on the forcing and the ``variables`` selection (see :ref:`output-variable-selection`). .. _output-peak-variables: Peak Variables ^^^^^^^^^^^^^^ Peak (maximum) values are accumulated over the entire simulation and written once at the end. A restarted run starts its peaks afresh, so its peak fields cover that run only (see :doc:`checkpoint`). Nodes that were never wet contain the fill value ``-99999.0``. Peak variables are always written and cannot be disabled. .. list-table:: :header-rows: 1 * - Variable - Description - Units - Dimensions * - ``zeta_max`` - Peak water surface elevation - meters - (node) * - ``velocity_max`` - Peak velocity magnitude - m/s - (node) * - ``pressure_min`` - Minimum atmospheric pressure - Pa - (node) * - ``wind_max`` - Peak wind speed - m/s - (node) .. note:: The ``pressure_min`` and ``wind_max`` variables are only present when meteorological forcing is enabled. .. _output-mode: Output Mode ----------- The ``mode`` parameter controls whether peak variables are written to the same file as time-series data or to a separate file. Split Mode (Default) ^^^^^^^^^^^^^^^^^^^^ .. code-block:: yaml output: global: mode: split In split mode, Cocoa writes two files: - ``cocoa_output.nc``, the main file, holds the time-series variables. - ``cocoa_output_peak.nc``, the peak file, holds the peak variables. The peak filename is derived from the main filename by inserting ``_peak`` before the ``.nc`` extension. For example: .. list-table:: :header-rows: 1 * - Main Filename - Peak Filename * - ``cocoa_output.nc`` - ``cocoa_output_peak.nc`` * - ``gulf_simulation.nc`` - ``gulf_simulation_peak.nc`` Split mode is the default because peak files are small and easy to transport independently of the larger time-series file. Combined Mode ^^^^^^^^^^^^^ .. code-block:: yaml output: global: mode: combined In combined mode, both time-series and peak variables are written to a single file. .. _station-output: Station Output -------------- Station output records time series at listed points, usually more often than the global file and at a small fraction of its size. It goes to a separate file with its own interval, time window and variable selection. .. code-block:: yaml output: stations: step_interval: 6m locations: {include: stations.yaml} ``stations.yaml`` is a list of names and locations: .. code-block:: yaml - name: Battery, NY, NOAA location: [-74.0142, 40.7006] - name: 8761724 location: [-89.957, 29.263] The list can also be written inline under ``locations``. Locations use the mesh file's coordinate system: longitude and latitude in degrees for a geographic mesh. Names must be unique. A relative ``include`` path is resolved against the directory of the configuration file. .. list-table:: :header-rows: 1 :widths: 25 15 20 40 * - Parameter - Type - Default - Description * - ``step_interval`` - int or duration - ``output.global.step_interval`` - Interval between station samples. Defaults to the global output interval. * - ``locations`` - list - *(required)* - Stations, as ``name`` and ``location: [x, y]`` entries. * - ``filename`` - string - ``cocoa_station_output.nc`` - Station NetCDF filename; must differ from ``output.global.filename`` and, in split mode, from the peak file. * - ``start_time``, ``end_time`` - datetime - simulation start, end - Station time window, with the same rules as :ref:`output-time-window`. * - ``variables`` - map - *(as* ``output.global`` *)* - The same toggles and defaults as ``output.global.variables`` (see :ref:`output-variable-selection`), set independently. A variable is written only when its forcing is active. * - ``partially_wet`` - string - ``nearest_wet_depth`` - Value reported when the station's element is partly wet. See :ref:`station-wet-dry`. .. _station-wet-dry: Wet/Dry Stations ^^^^^^^^^^^^^^^^ A station takes the value interpolated from the corners of the element that contains it, with the model's own basis: linear on a triangle, bilinear on a quadrilateral. A station on an edge or node shared by several elements uses the lowest-numbered of them, so the result does not depend on how the mesh is partitioned. If every corner is wet the value is interpolated. If every corner is dry the fill value ``-99999.0`` is written. A partly wet element follows ``partially_wet``, which ports ADCIRC's station options: - ``nearest_wet_depth``: the value at the corner nearest the station, if that corner is wet and the water depth there (its elevation plus the bathymetric depth at the station) exceeds ``0.8 * physics.h0``; otherwise fill. This is ADCIRC's ``StatPartWetFix`` with ``How2FixStatPartWet = 0``. - ``nearest_wet``: the nearest corner's value if that corner is wet; otherwise fill (``How2FixStatPartWet = 1``). - ``fill``: always fill, which is ADCIRC's default. Cocoa defaults to ``nearest_wet_depth``, unlike ADCIRC, so that stations near the shoreline report data. A station outside the mesh is reported in a startup warning and kept in the file with fill values, so the file's station order always matches the list. Station File ^^^^^^^^^^^^ The station file is a CF-1.8 discrete sampling geometry with ``featureType`` ``timeSeries``. Its dimensions are ``time`` and ``station``. ``station_name`` (the ``timeseries_id``), ``lon``, ``lat`` and ``depth`` (bathymetry interpolated to the station) describe each station. Each output component is a ``(time, station)`` float variable with the name, units and attributes it has in the global file, so vectors are geographic east/north components and time is in seconds since 1970-01-01. .. code-block:: python import xarray as xr ds = xr.open_dataset("cocoa_station_output.nc") ds = ds.set_coords("station_name").swap_dims(station="station_name") ds.zeta.sel(station_name="Battery, NY, NOAA").plot() Samples are held on the device and written in blocks: when the buffer fills, at every global output time, and at the end of the run. While the global output window is open, station data in the file therefore lags the model by at most one global output interval. Outside that window, or when every global time-series variable is disabled, it can lag by up to a full buffer of 512 samples. A run stopped by an instability still writes every buffered sample. NetCDF File Structure --------------------- Output files follow CF-1.8 and UGRID-1.0 conventions. Example structure (combined mode): .. code-block:: text dimensions: time = UNLIMITED ; node = 1000 ; nfaces = 1800 ; nvertex = 3 ; variables: double time(time) ; time:units = "seconds since 1970-01-01 00:00:00" ; time:calendar = "standard" ; double lon(node) ; lon:units = "degrees_east" ; double lat(node) ; lat:units = "degrees_north" ; double depth(node) ; depth:units = "m" ; depth:positive = "down" ; float zeta(time, node) ; zeta:units = "m" ; zeta:long_name = "Water surface elevation" ; float ucx(time, node) ; ucx:units = "m s-1" ; ucx:long_name = "water velocity, eastward component" ; ucx:standard_name = "eastward_sea_water_velocity" ; float ucy(time, node) ; ucy:units = "m s-1" ; ucy:long_name = "water velocity, northward component" ; ucy:standard_name = "northward_sea_water_velocity" ; float zeta_max(node) ; zeta_max:units = "m" ; zeta_max:long_name = "Peak water surface elevation" ; float velocity_max(node) ; velocity_max:units = "m s-1" ; velocity_max:long_name = "Peak velocity magnitude" ; Field variables are stored as single precision (``float``): ~7 significant digits, ample for model output products, at half the file size and write cost of double. Coordinates, bathymetry, and the time axis remain double. In split mode, the main file omits the peak variables (``zeta_max``, ``velocity_max``, etc.), and they appear in the separate peak file instead. Reading Output Files -------------------- Python (xarray) ^^^^^^^^^^^^^^^ .. code-block:: python import xarray as xr ds = xr.open_dataset("cocoa_output.nc") zeta = ds['zeta'] # Time-varying elevation zeta_max = ds['zeta_max'] # Peak elevation (combined mode) # Plot time series at a specific node zeta.isel(node=100).plot() For split mode, open the peak file separately: .. code-block:: python peak = xr.open_dataset("cocoa_output_peak.nc") zeta_max = peak['zeta_max'] Python (netCDF4) ^^^^^^^^^^^^^^^^ .. code-block:: python from netCDF4 import Dataset nc = Dataset("cocoa_output.nc", "r") zeta = nc.variables['zeta'][:] # Shape: (time, node) time = nc.variables['time'][:] nc.close() # For split mode peak data: peak = Dataset("cocoa_output_peak.nc", "r") zeta_max = peak.variables['zeta_max'][:] # Shape: (node,) peak.close()