Skip to content
This repository was archived by the owner on Jul 8, 2026. It is now read-only.

SPEC.md

Latest commit

 

History

History
645 lines (512 loc) · 34.2 KB

SPEC.md

File metadata and controls

645 lines (512 loc) · 34.2 KB
Raw

deck-gl-datamesh-layers — Library Specification

Overview

@oceanum/deck-gl-datamesh-layers is a wrapper library that provides deck.gl layers for visualising gridded data from pre-cached zarr datasets hosted on the Oceanum Datamesh. It combines @oceanum/deck-gl-grid (rendering) with @oceanum/datamesh Dataset.zarr() (data fetching) into a seamless API.

Zarr datasets are accessed directly via a root service URL, with individual datasets as subpaths (e.g. https://zarr.datamesh.oceanum.io/oceanum_wave_glob05). The library uses Dataset.zarr() from @oceanum/datamesh to open datasets, which handles consolidated metadata parsing, coordinate discovery, and chunk-level data access via DataVar.get(). Because zarr is a chunked, partitioned format, only the chunks required for the current view are fetched on demand. Chunk selection is driven by three axes: time and level (from props) and spatial extent (from the deck.gl viewport). The CachedHTTPStore in @oceanum/datamesh handles fetching and IndexedDB caching transparently — panning to a previously visited area or returning to an earlier time step serves chunks from cache.

Goals

  • Provide deck.gl layers that connect directly to zarr datasets via a service URL
  • Open the zarr dataset via Dataset.zarr() (metadata + coordinate discovery); fetch data chunks on demand via DataVar.get() driven by time, level, and viewport extent
  • Use deck.gl CompositeLayer properties and state to manage dimension selection, viewport tracking, async chunk loading, and re-rendering
  • Leverage zarr's chunked architecture — only fetch chunks that intersect the current viewport at the current time/level; previously fetched chunks are served from the IndexedDB cache so panning back or revisiting a time step is instant
  • Support core visualisation types from @oceanum/deck-gl-grid: pcolor, particle, partmesh (vector arrows), and contour
  • Keep the API minimal — users should not need to understand the internal data pipeline

Architecture

User Props (datasource, time, level, colormap, ...)
        │
        ▼
┌───────────────────────────────────────────────┐
│   DatameshLayer (CompositeLayer)              │  ← This library
│                                               │
│  initializeState():                           │
│    1. Resolve instance (explicit or latest)   │
│    2. Open dataset via Dataset.zarr(url, hdr) │
│    3. Store Dataset in layer state            │
│    4. Fetch coord arrays via DataVar.get()    │
│    5. Request initial slice for current       │
│       viewport (triggers chunk fetch)         │
│                                               │
│  updateState():                               │
│    - On datasource/instance change → re-open  │
│    - On time/level change → resolve nearest   │
│      index, request slice for current viewport│
│    - On viewport change → compute new bbox,   │
│      request slice for new spatial extent     │
│      (only fetches newly visible chunks)      │
│                                               │
│  renderLayers():                              │
│    1. Read sliced 2D data from state          │
│    2. Build data + datakeys                   │
│    3. Return inner grid layer                 │
│                                               │
└───────────────┬───────────────────────────────┘
                │ data (2D slice as xarray-like object)
                ▼
┌───────────────────────────────────────────────┐
│  @oceanum/deck-gl-grid layer                  │
│  (PcolorLayer, ParticleLayer, etc.)           │
└───────────────────────────────────────────────┘

        ┌───────────────────────────────────────────────┐
        │  CachedHTTPStore (datamesh)                    │
        │  URL: serviceUrl/datasource[/instance]         │
        │                                                │
        │  Remote chunks ←→ IndexedDB cache              │
        │                                                │
        │  Chunks addressed by (time, [level], lat, lon) │
        │  Only chunks intersecting the current viewport │
        │  at the selected time/level are fetched        │
        └───────────────────────────────────────────────┘

Each wrapper layer is a deck.gl CompositeLayer that:

  1. On initialisation (or when datasource / instance changes), opens the zarr archive, resolves the instance group (explicit or latest), and fetches its metadata (dimensions, coordinates, chunk layout) — no data chunks yet
  2. Reads the time, level, latitude, and longitude coordinate arrays from the dataset metadata and stores them in state along with chunk sizes
  3. When time, level, or the viewport changes, computes the required slice: nearest time/level index + lat/lon index range from the viewport bbox. The zarr library fetches only the chunks that intersect this slice (or serves them from IndexedDB cache if previously visited)
  4. Once the slice resolves, stores the 2D data in state and renderLayers() passes it to the corresponding @oceanum/deck-gl-grid layer

Layers

Wrapper Layer Inner Layer Purpose
DatameshPcolorLayer PcolorLayer Pseudocolor grid cells
DatameshParticleLayer ParticleLayer Animated vector field particles
DatameshPartmeshLayer PartmeshLayer Mesh-based vector field arrows
DatameshContourLayer ContourLayer Contour lines with labels

Props

Common Props (all layers)

Prop Type Required Description
serviceUrl string Yes Root URL of the zarr service. Datasets are at serviceUrl/datasource
authHeaders object No HTTP headers for authentication (e.g. { Authorization: 'Bearer ...' }). Default {}
datasource string Yes Dataset name (subpath of the service URL)
instance string No Zarr group name within the archive. If omitted, defaults to the latest instance (last when sorted lexicographically). See Zarr Instance Groups
time string | Date No ISO 8601 string (e.g. '2024-01-15T00:00:00Z') or JavaScript Date for time selection. Resolved to the nearest available time step. If omitted, first time step is used
level number No 0-based index into the level dimension. If omitted, index 0 is used. Ignored if the dataset has no level dimension
colormap object No { scale: string[], domain: number[] } — colour scale and value domain
opacity number No Layer opacity (0–1). Default 1.0
altitude number No Altitude offset. Default 0.0
globalWrap boolean No Wrap data across the antimeridian. Default false
scale number No Value multiplier before colormapping. Default 1.0
offset number No Value offset before colormapping. Default 0.0
pickable boolean No Enable picking. Default false
visible boolean No Layer visibility. Default true
viewportPadding number No Fraction of viewport extent to pad beyond visible edges when fetching spatial chunks. Default 0.1 (10%)
debounceWait number No Debounce delay in ms before a slice request is dispatched after time, level, or viewport changes. Default 100
onDataLoad function No Callback when dataset metadata loads. Receives { dataset, times, nlevels, instance } where times is an array of Date objects, nlevels is the number of available levels, and instance is the resolved group name
onError object No Error hook object — one callback per error state. See Error Hooks below

DatameshPcolorLayer Props

Renders a scalar variable as coloured grid cells.

Prop Type Required Description
variable string Yes Scalar variable name in the zarr dataset (e.g. 'hs', 'temperature') → mapped to datakeys.c
color [r,g,b] No Fallback colour when no colormap. Default [200,200,200]
material boolean | object No Phong material for lighting. Default false

DatameshParticleLayer Props

Renders animated particles flowing through a vector field. Requires either u/v components or magnitude/direction variables.

Prop Type Required Description
uVariable string * U (eastward) vector component variable name → mapped to datakeys.u
vVariable string * V (northward) vector component variable name → mapped to datakeys.v
magnitudeVariable string * Magnitude variable name → mapped to datakeys.m
directionVariable string * Direction variable name → mapped to datakeys.d
speed number No Particle animation speed multiplier. Default 1.0
npart number No Number of particles. Default 1000
size number No Particle size in pixels. Default 3
length number No Particle trail length. Default 12
directionConvention string No Direction convention: 'NAUTICAL_FROM', 'NAUTICAL_TO', 'CARTESIAN_RADIANS'. Default 'NAUTICAL_FROM'
color [r,g,b] No Fallback colour. Default [200,200,200]

* Provide either (uVariable + vVariable) or (magnitudeVariable + directionVariable). The layer validates that exactly one pair is provided.

DatameshPartmeshLayer Props

Renders mesh-based arrows for a vector field. Same variable selection as ParticleLayer.

Prop Type Required Description
uVariable string * U (eastward) vector component variable name → mapped to datakeys.u
vVariable string * V (northward) vector component variable name → mapped to datakeys.v
magnitudeVariable string * Magnitude variable name → mapped to datakeys.m
directionVariable string * Direction variable name → mapped to datakeys.d
speed number No Animation speed multiplier. Default 1.0
size number No Arrow size in pixels. Default 3
directionConvention string No Direction convention. Default 'NAUTICAL_FROM'
color [r,g,b] No Fallback colour. Default [200,200,200]

* Provide either (uVariable + vVariable) or (magnitudeVariable + directionVariable).

DatameshContourLayer Props

Renders contour lines for a scalar variable.

Prop Type Required Description
variable string Yes Scalar variable name in the zarr dataset → mapped to datakeys.c
levels number[] No Contour level values. Default []
labelSize number No Label font size. Default 12
labelColor [r,g,b,a] No Label colour. Default [255,255,255,255]
smoothing boolean No Smooth contour lines. Default false
numLabels number No Labels per contour line. Default 1
color [r,g,b] No Fallback line colour. Default [200,200,200]

Error Hooks

The onError prop accepts an object with a callback for each distinct error state. Each callback receives a structured error object. If a hook is not provided for a given error state, the error is logged to the console as a warning.

onError: {
  onMetadataError:   (error) => { ... },
  onChunkError:      (error) => { ... },
  onVariableError:   (error) => { ... },
  onValidationError: (error) => { ... },
}

Error States

Hook When Error Object Layer Behaviour
onMetadataError The initial lazy open of the zarr dataset fails (network error, 404, auth, missing _coordinates, etc.) { type: 'metadata', datasource, message, cause } Layer renders nothing — no data is available
onChunkError A chunk fetch fails during slicing (network timeout, corrupted chunk, HTTP error) { type: 'chunk', datasource, time, level, message, cause } Layer continues rendering the last valid slice
onVariableError A requested variable name is not found in the zarr dataset metadata { type: 'variable', datasource, variable, availableVariables, message } Layer renders nothing for the missing variable
onValidationError Invalid prop combination detected (e.g. neither u/v nor magnitude/direction provided for ParticleLayer, or both pairs provided) { type: 'validation', message } Layer renders nothing — fired synchronously during updateState

Error Object Fields

Field Type Description
type string Error state identifier: 'metadata', 'chunk', 'variable', 'validation'
datasource string The datasource ID (present on all except validation)
time Date | null The time being requested when the error occurred (only on chunk)
level number | null The level index being requested (only on chunk)
variable string The variable name that was not found (only on variable)
availableVariables string[] List of valid variable names in the dataset (only on variable)
message string Human-readable description of the error
cause Error | null The underlying JavaScript error, if any

Default Behaviour

When no onError hook is provided for a given state:

  • metadata — console.warn with the message; layer renders nothing
  • chunk — console.warn; layer keeps showing the last valid slice (no blank frame)
  • variable — console.warn; layer renders nothing
  • validation — console.error; layer renders nothing (this is a programming error)

Usage

new DatameshPcolorLayer({
  id: 'sst',
  serviceUrl: SERVICE_URL,
  authHeaders: AUTH_HEADERS,
  datasource: 'oceanum_sst_glob',
  variable: 'sst',
  time: '2024-01-15T00:00:00Z',
  onError: {
    onMetadataError: (err) => {
      showToast(`Failed to load ${err.datasource}: ${err.message}`);
    },
    onChunkError: (err) => {
      showToast(`Data loading error at ${err.time?.toISOString()}: ${err.message}`);
    },
    onVariableError: (err) => {
      showToast(`Variable "${err.variable}" not found. Available: ${err.availableVariables.join(', ')}`);
    },
    onValidationError: (err) => {
      console.error('Layer configuration error:', err.message);
    }
  }
})

Dimension Handling

Datamesh zarr datasets have dimensions (time, [level], latitude, longitude). The actual variable names for each dimension are discovered from the _coordinates attribute in the zarr root metadata (see above). The wrapper layers produce a 2D spatial subset by slicing along all four axes:

  • Time — selected by the time prop (ISO 8601 string or Date), resolved to the nearest available time step via binary search on the time coordinate array. The time coordinate values in the zarr store are numeric timestamps; the layer converts the prop value to a timestamp for comparison
  • Level — selected by the level prop as a direct 0-based index. There is no nearest-match or unit conversion — levels have no standard unit or convention, so the user addresses them positionally
  • Latitude and longitude — selected by the current deck.gl viewport bbox, resolved to the index range that covers the visible area

Because zarr is a chunked format, slicing is a combination of local index arithmetic and on-demand remote chunk fetches — the zarr library resolves which chunks intersect the requested index ranges and fetches only those (or serves them from cache).

Viewport-Based Spatial Slicing

The layer reads the current viewport from this.context.viewport (provided by deck.gl to all layers). On each updateState where the viewport has changed:

  1. Extract the viewport bounding box: [west, south, east, north]
  2. Map the bbox to index ranges in the latitude and longitude coordinate arrays: 
    const lonRange = indexRange(lonCoords, west, east);   // [startLon, endLon]
const latRange = indexRange(latCoords, south, north); // [startLat, endLat]
  3. Request the zarr slice at [timeIndex, levelIndex?, latRange, lonRange]
  4. The zarr library maps these index ranges to chunk keys — only chunks that overlap the viewport are fetched

Padding: The viewport bbox is expanded by a configurable margin (default: 10% of the viewport extent on each side) so that data is available slightly beyond the visible edges. This prevents blank strips appearing at the edges during slow pans.

Lifecycle

On dataset open (in initializeState / updateState when datasource or instance changes):

  1. Resolve the instance group: use the instance prop if provided, otherwise discover instances from root .zmetadata, sort lexicographically, and select the last one
  2. Open the zarr archive via Dataset.zarr(url, authHeaders) — fetches consolidated metadata and discovers all variables and their dimensions
  3. Read coordinate mappings from dataset.coordkeys (parsed from _coordinates in .zattrs) for x, y, t, z
  4. Store the Dataset object in layer state: this.setState({ dataset })
  5. Fetch coordinate arrays via DataVar.get() — time (t), latitude (y), and longitude (x); level count from dataset.dimensions
  6. Resolve the initial time index from the time prop (nearest match) or default to 0; use the level prop directly as index or default to 0
  7. Compute initial lat/lon index ranges from the current viewport
  8. Request the initial slice (triggers async chunk fetch — see below)
  9. Call onDataLoad callback with { dataset, times, nlevels, instance } where times is an array of Date objects, nlevels is the number of available levels, and instance is the resolved group name

On time/level prop change (in updateState, detected via changeFlags.propsChanged):

  1. Time: convert the time prop (ISO 8601 string or Date) to a numeric timestamp, then find the nearest index in the time coordinate array
  2. Level: use the level prop directly as the index (clamped to valid range)
  3. If the index has changed, request a new slice at the current viewport extent
  4. The zarr library determines which chunks cover the requested slice:
    • If those chunks are already in the IndexedDB cache (e.g. revisiting a previous time step) → served locally, effectively instant
    • If not cached → fetched from the remote zarr store over HTTP
  5. The slice request is async. While it resolves, the layer continues rendering the previous slice (no flicker)
  6. When the slice resolves, store the 2D data in state and trigger a re-render

On viewport change (in updateState, detected via changeFlags.viewportChanged):

  1. Compute the new padded viewport bbox
  2. Map to lat/lon index ranges
  3. If the new index ranges are already contained within the previously fetched ranges → no fetch needed, the existing data covers the view
  4. If new chunks are needed → request a new slice covering the expanded viewport
  5. While loading, continue rendering the previous data (may show blank edges briefly)
  6. When the slice resolves, update state with the new data and coordinate subset

On render (in renderLayers):

  1. Read slicedData and datakeys from state
  2. If slicedData is null (still loading initial data), return no layers
  3. Otherwise return the inner @oceanum/deck-gl-grid layer with the sliced data: 
    {
  coords: {
    [xName]: { data: lonSubset },  // Only the lon values in the viewport range
    [yName]: { data: latSubset }   // Only the lat values in the viewport range
  },
  data_vars: {
    [varName]: { data: slicedArray2D },  // (latRange × lonRange) subset
    ...
  }
}

Time Index Resolution

The time prop accepts an ISO 8601 string or JavaScript Date. It is converted to a numeric timestamp and matched against the time coordinate array using nearest-match (binary search for efficiency since the time coordinate is monotonically increasing):

function nearestTimeIndex(timeCoords, target) {
  const t = (target instanceof Date ? target : new Date(target)).getTime();
  let lo = 0, hi = timeCoords.length - 1;
  while (lo < hi) {
    const mid = (lo + hi) >> 1;
    if (timeCoords[mid] < t) lo = mid + 1;
    else hi = mid;
  }
  // Check neighbours to find true nearest
  if (lo > 0 && Math.abs(timeCoords[lo - 1] - t) < Math.abs(timeCoords[lo] - t)) {
    return lo - 1;
  }
  return lo;
}

Level Index

The level prop is used directly as a 0-based index into the level dimension. No conversion or nearest-match is performed — the user addresses levels positionally. The index is clamped to [0, nlevels - 1].

Slicing

Given a dataset variable with dimensions discovered from _coordinates and a shape of (ntime, [nlevel], nlat, nlon):

  • If time and level dimensions exist: data[timeIndex][levelIndex][latRange][lonRange]
  • If only time dimension exists: data[timeIndex][latRange][lonRange]
  • If only level dimension exists: data[levelIndex][latRange][lonRange]
  • If neither exists: data[latRange][lonRange]

The dimension names and order are determined from the variable's dims array in the zarr metadata, cross-referenced with the _coordinates attribute.

Coordinate Discovery via _coordinates

The zarr root metadata (.zattrs) contains a _coordinates attribute that maps dimension roles to variable names:

{
  "_coordinates": {
    "x": "longitude",
    "y": "latitude",
    "t": "time",
    "z": "level"
  }
}
  • x — longitude coordinate variable name
  • y — latitude coordinate variable name
  • t — time coordinate variable name (may be absent if no time dimension)
  • z — level coordinate variable name (may be absent if no level dimension)

On dataset open, the layer reads _coordinates and uses it to:

  1. Identify the coordinate arrays for viewport mapping (x, y), time selection (t), and level indexing (z)
  2. Build the datakeys object required by the inner @oceanum/deck-gl-grid layer by combining _coordinates (for spatial axes) with the layer's variable props (for data variables)

datakeys Construction

The datakeys object passed to the inner @oceanum/deck-gl-grid layer is built automatically from two sources:

Spatial axes — always derived from _coordinates:

const coords = dataset.attrs._coordinates;
datakeys.x = coords.x;  // e.g. 'longitude'
datakeys.y = coords.y;  // e.g. 'latitude'

Data variables — derived from each layer's variable props:

Layer Props datakeys
DatameshPcolorLayer variable { c: variable }
DatameshContourLayer variable { c: variable }
DatameshParticleLayer uVariable, vVariable { u: uVariable, v: vVariable }
DatameshParticleLayer magnitudeVariable, directionVariable { m: magnitudeVariable, d: directionVariable }
DatameshPartmeshLayer (same as ParticleLayer) (same as ParticleLayer)

The variable props also determine which zarr data variables to fetch chunks for — the layer only requests chunks for the named variables, not every variable in the dataset.

Zarr Instance Groups

A zarr archive on the Datamesh may contain multiple instance groups — each representing a distinct run, forecast cycle, or data version. The groups are named such that lexicographic sorting produces chronological order (e.g. ISO 8601 timestamps like 2024-01-15T00, or zero-padded identifiers like run_001).

archive.zarr/
├── .zattrs                  # Root metadata
├── 2024-01-14T00/           # Instance group
│   ├── .zattrs              # _coordinates, variable metadata
│   ├── longitude/
│   ├── latitude/
│   ├── time/
│   └── hs/
├── 2024-01-14T12/           # Instance group
│   └── ...
└── 2024-01-15T00/           # Instance group (latest)
    └── ...

instance Prop

The instance prop selects which group to open:

  • If provided — the layer opens the specified group directly (e.g. instance: '2024-01-14T12')
  • If omitted — the layer reads the root metadata to list available groups, sorts them lexicographically, and opens the last one (i.e. the latest instance)

Resolution During Metadata Open

During initializeState / updateState when datasource or instance changes:

  1. If instance prop is set → construct URL as serviceUrl/datasource/instance
  2. If instance prop is not set → fetch serviceUrl/datasource/.zmetadata, discover groups by looking for <name>/.zgroup entries, sort lexicographically, select the last one, construct URL as serviceUrl/datasource/instance
  3. Open the zarr archive at the resolved URL via Dataset.zarr(url, authHeaders)
  4. Coordinate mappings are available via dataset.coordkeys (parsed from _coordinates in .zattrs)
  5. Store the resolved instance name in state: this.state.instance

The onDataLoad callback includes the resolved instance name:

onDataLoad: ({ dataset, times, nlevels, instance }) => {
  console.log(`Loaded instance: ${instance}`);
}

Changing Instance

If the instance prop changes, the layer re-opens the dataset at the new group — this is a metadata re-fetch (same as changing datasource). The previous data continues rendering during the transition.

Debouncing

All slice requests — whether triggered by viewport changes, time prop changes, or level prop changes — are debounced. This prevents redundant chunk fetches when:

  • The user is scrubbing a time slider rapidly
  • The user is panning/zooming the map
  • Multiple props change in quick succession (e.g. time + level updated together in one React render)

Mechanism

The layer maintains a single debounced requestSlice() function. Any change to time, level, or viewport calls this function. Only the last invocation within the debounce window actually triggers a chunk fetch. While waiting for the debounce to settle, the layer continues rendering the previous slice.

time prop changes rapidly:  t1 → t2 → t3 → t4 → [debounce wait] → fetch(t4)
viewport panning:           v1 → v2 → v3 → v4 → [debounce wait] → fetch(v4)
combined changes:           t1+v1 → t2+v2 →      [debounce wait] → fetch(t2, v2)

Configuration

Prop Type Default Description
debounceWait number 100 Debounce delay in milliseconds before a slice request is dispatched

The default of 100ms is a reasonable balance — fast enough to feel responsive but long enough to coalesce rapid changes. For time animation playback at a fixed interval, the debounce is effectively a no-op since each time change is spaced beyond the debounce window.

Data Fetching

URL Construction

The zarr dataset URL is constructed from the layer props:

// Without instances:
url = `${serviceUrl}/${datasource}`

// With an explicit instance:
url = `${serviceUrl}/${datasource}/${instance}`

// With auto-resolved instance (latest):
url = `${serviceUrl}/${datasource}/${resolvedInstance}`

Metadata vs Chunk Fetching

There are two distinct phases of network activity:

  1. Metadata open — triggered when datasource, instance, or serviceUrl changes. Calls Dataset.zarr(url, authHeaders) which fetches the zarr consolidated metadata (.zmetadata) and coordinate arrays. This is lightweight and tells the layer what dimensions, times, levels, variables, and chunk layout are available.

  2. Chunk fetching — triggered whenever a slice is requested (initial load, time/level change, or viewport change). Uses DataVar.get(indexSpec) which maps the requested index ranges across all four dimensions (time, [level], lat, lon) to chunk keys and fetches them via HTTP. The CachedHTTPStore in @oceanum/datamesh caches chunks in IndexedDB, so:

    • First visit to a time step at a given viewport → HTTP fetch for the required chunks
    • Revisiting the same time step / viewport region → instant from IndexedDB cache
    • Panning slightly → only the newly revealed edge chunks are fetched; the rest are already cached
    • Adjacent time steps may share spatial chunks (depending on chunk layout), reducing fetch overhead when scrubbing through time

Loading States

  • this.state.loading — true during the initial metadata open
  • this.state.slicing — true while a slice request (chunk fetch) is in progress
  • this.state.error — the most recent error object, or null if no error
  • While slicing, the layer continues rendering the previous slice — no blank frame
  • onDataLoad({ dataset, times, nlevels, instance }) fires when metadata open completes — times is an array of Date objects, nlevels is the count of available levels, instance is the resolved group name. This allows the application to build UI controls (time sliders, level selectors, instance indicators, etc.)
  • On error, the corresponding onError hook is called (see Error Hooks). If no hook is defined for that error state, a console warning is emitted

CompositeLayer State Summary

this.state = {
  // Dataset (lazy — metadata loaded, chunks fetched on demand)
  dataset: null,        // Dataset from gateway.zarr(query, lazy=true)
  loading: false,       // True during initial metadata open
  slicing: false,       // True while chunk fetch for a slice is in progress
  error: null,          // Most recent error object ({ type, message, ... }) or null

  // Instance + coordinate metadata
  instance: null,       // Resolved zarr group name (string)
  coordNames: null,     // { x, y, t?, z? } — variable names from _coordinates

  // Coordinate arrays (extracted from dataset metadata on open)
  times: null,          // Float64Array of timestamps in ms (or null if no t dim)
  nlevels: 0,           // Number of levels (0 if no z dim)
  lats: null,           // Float64Array of latitude values
  lons: null,           // Float64Array of longitude values

  // Current dimension indices (updated when time/level props or viewport change)
  timeIndex: 0,         // Index into times array (nearest match from time prop)
  levelIndex: 0,        // Direct 0-based index from level prop
  latRange: [0, 0],     // [startIndex, endIndex] into lats array for current viewport
  lonRange: [0, 0],     // [startIndex, endIndex] into lons array for current viewport

  // Derived (populated when slice resolves)
  slicedData: null,     // 2D data object ready for the inner layer (viewport subset)
  datakeys: null        // Resolved datakeys mapping
};

Package Structure

deck-gl-datamesh-layers/
├── src/
│   ├── index.js                    # Public exports
│   ├── datamesh-pcolor-layer.js
│   ├── datamesh-particle-layer.js
│   ├── datamesh-partmesh-layer.js
│   ├── datamesh-contour-layer.js
│   └── utils/
│       ├── dataset-slice.js        # Nearest-index + slicing across all dimensions
│       ├── viewport.js             # Viewport bbox → lat/lon index ranges, debouncing, padding
│       └── coordinates.js          # Parse _coordinates attr, build datakeys
├── package.json
├── vite.config.js
├── SPEC.md
└── README.md

Dependencies

Peer Dependencies

  • @deck.gl/core ^9.2.0
  • @deck.gl/layers ^9.2.0
  • @luma.gl/core ^9.2.0
  • react ^19.0.0

Dependencies

  • @oceanum/datamesh ^2.0.0
  • @oceanum/deck-gl-grid ^9.2.0

Usage Example

import { DatameshPcolorLayer, DatameshParticleLayer } from '@oceanum/deck-gl-datamesh-layers';
import DeckGL from '@deck.gl/react';

const SERVICE_URL = 'https://zarr.datamesh.oceanum.io';
const AUTH_HEADERS = { Authorization: `Bearer ${token}` };

function App() {
  const [time, setTime] = useState('2024-01-15T00:00:00Z');
  const [level, setLevel] = useState(0);
  const [availableTimes, setAvailableTimes] = useState([]);
  const [nlevels, setNlevels] = useState(0);

  const layers = [
    new DatameshPcolorLayer({
      id: 'wave-height',
      serviceUrl: SERVICE_URL,
      authHeaders: AUTH_HEADERS,
      datasource: 'oceanum_wave_glob05',
      variable: 'hs',            // Scalar variable → datakeys.c
      time,                       // ISO 8601 string — resolved to nearest time step
      colormap: {
        scale: ['#313695', '#4575b4', '#74add1', '#abd9e9', '#fee090', '#fdae61', '#f46d43', '#d73027'],
        domain: [0, 1, 2, 3, 4, 5, 6, 8]
      },
      opacity: 0.8,
      pickable: true,
      onDataLoad: ({ times, nlevels }) => {
        setAvailableTimes(times);   // Array of Date objects
        setNlevels(nlevels);        // Number of available levels
      }
    }),
    new DatameshParticleLayer({
      id: 'wind-particles',
      serviceUrl: SERVICE_URL,
      authHeaders: AUTH_HEADERS,
      datasource: 'oceanum_era5_wind10m',
      uVariable: 'u10',          // U component → datakeys.u
      vVariable: 'v10',          // V component → datakeys.v
      time,                       // Can also pass a Date object
      level: 0,                   // 0-based index into level dimension
      npart: 5000,
      speed: 2.0,
      colormap: {
        scale: ['#f7fbff', '#6baed6', '#08306b'],
        domain: [0, 10, 25]
      }
    })
  ];

  return (
    <>
      <DeckGL layers={layers} /* viewState, controller, etc. */ />
      <TimeSlider times={availableTimes} value={time} onChange={setTime} />
      {nlevels > 1 && (
        <LevelSelector count={nlevels} value={level} onChange={setLevel} />
      )}
    </>
  );
}

Open Questions

  1. Time animation helpers: Should the library include utilities for stepping/animating through time (e.g. chunk prefetching for adjacent time steps, playback controls)?

  2. Chunk prefetching: Should the layer proactively prefetch chunks for adjacent time steps (t-1, t+1) or nearby spatial tiles to make scrubbing/panning feel instant? This trades bandwidth for perceived performance.