Weather GIS Platform — Architecture
| Description | Architecture documentation — clustered GeoServer on Kubernetes |
Table of contents
Weather GIS Platform — architecture documentation¶
Start here
New to this platform? Read the Vision first — the goals and the principles that keep it simple and durable, with links to the proofs. Not a technical reader? How it works explains the platform in plain language. Preparing the Foundation Workshop? The GIS contribution answers the workshop agenda point by point.
This documentation describes how to deploy GeoServer as a cluster on Kubernetes to serve meteorological, aeronautical and hydrological data: catalog and resources in the database (JDBCConfig / JDBCStore / JDBCStatus via JNDI), tile cache and rasters on S3, WPS, TimescaleDB + PostGIS database, image mosaic ingestion service.
Reading guide¶
| Section | Audience | Content |
|---|---|---|
| Architecture (arc42) | Architects, tech leads | The why and the what: constraints, context, building blocks, deployment, decisions |
| Guides | Ops, developers | The how: reproducible step-by-step instructions, derived from validated prototypes |
| Decisions (ADR) | Everyone | Each structuring choice, its alternatives and its consequences |
Progress status¶
| # | Sub-project | Status |
|---|---|---|
| 0 | Documentation foundation | ✅ in place |
| 1 | Database (TimescaleDB + PostGIS, GeoServer schema) | ✅ guide + PoC validated |
| 2 | GeoServer image (Tomcat, JNDI, plugins, JVM) | ✅ guide + PoC validated |
| 3 | Clustering (variable replica count, configuration propagation) | ✅ guide + PoC validated |
| 4 | S3 storage (tile cache, rasters) | ✅ guide + PoC validated |
| 5 | ImageMosaic ingestion (satellite time series, multi-dimensional models) | ✅ guide + PoC validated |
| 6 | Vector data (published TimescaleDB/PostGIS tables) | ✅ guide + prototype validated |
| 7 | Basemaps (downloaded sources, external services) | ✅ guide + prototype validated |
| 8 | Processing: vector → raster interpolation, custom WPS | ✅ guide + prototype validated |
Method¶
Each technical chapter is backed by a prototype executed on a test Kubernetes
cluster before being written. The artifacts (Dockerfile, SQL scripts,
manifests) live in poc/<NN>-<name>/ and are referenced from the guides.
Visual conventions¶
Screenshots are clickable (zoom). Example:
Vision — a GeoServer 3 platform, simple by construction, durable by design¶
Executive summary of the architecture documented on this site. Every claim below links to the decision record (ADR) or the step-by-step guide where it is demonstrated by a runnable prototype — nothing here is theoretical.
The goal¶
One platform serving meteorological, aeronautical and hydrological data — model fields, satellite and radar imagery, decoded messages (SYNOP, METAR, TAF, SIGMET…), lightning, hydrology — through open OGC standards (WMS, WMTS, WFS, WCS, WPS), with clients free to navigate through time and altitude, and with units treated as first-class data: a measurement always has a unit (ADR-0003).
Why GeoServer 3 now¶
- GeoServer 3.0.x is the current stable line (Tomcat 11, Jakarta EE, Spring 6, Java 17/21). The 2.28 maintenance line reaches end of life in 2027: starting a new platform on 2.x means paying the Jakarta migration within a year — starting on 3.x avoids migrating twice.
- Every module this architecture needs is published for 3.0.1 (catalog and resources in a database, clustering, S3 tile cache, COG, WPS, control-flow) — verified, not assumed, and exercised by the prototypes (guide 2).
- The risk of a young major version is contained by this project's method: each capability was proven on a disposable test bench before being documented, and the pitfalls found are recorded in the guides.
Simple by construction¶
1. Nobody configures anything by hand. A small in-house plugin configures GeoServer at startup from declarative Kubernetes ConfigMaps: workspaces, datasources, mosaics, observation layers, basemaps, styles, tile caching (ADR-0006). Adding a weather mosaic or an observation layer is one line in a ConfigMap — no UI clicks, no REST scripts, nothing to remember (guide 5, guide 6).
2. Servers are cattle, not pets. A GeoServer pod owns no durable state: its data directory is rebuilt at every start and the database is the single source of truth — catalog, resources, styles, security, even the master key (ADR-0005). Replicas discover each other by DNS with no fixed addresses (ADR-0008); operations scale the cluster with a single command. Measured on the bench: configuration changes propagate in 3 s, scaling 3→5→2 serves zero errors, a full cold start converges in under a minute (guide 3).
3. One pattern per data family — and only one.
| Family | Pattern | Proof |
|---|---|---|
| Model fields, imagery | COG on S3, PostGIS-indexed ImageMosaic, TIME / ELEVATION / run dimensions | guide 5 |
| Decoded observations | TimescaleDB hypertable + one SQL view per layer, TIME-enabled | guide 6 |
| Tiles | Shared S3 blobstore (cache survives any pod) | guide 4 |
| Basemaps | Hosted public-domain data first, cascaded WMS as a complement | guide 7 |
| Derived fields | In-house WPS / rendering transformations | guide 8 |
New data arrives by inserting rows and objects — never by reconfiguring servers: layers pick it up immediately, with no restart and no notification bus (ADR-0010).
Durable by design¶
- Open standards end to end: OGC services out; COG, CF conventions and WMO code tables in (ADR-0003, ADR-0009) — no proprietary formats, no lock-in, clients replaceable.
- Reproducible artifacts: the server image is built from pinned, released components; nothing is downloaded at runtime; community modules are resolved as versioned bundles (ADR-0007).
- Decisions are written down: 13 architecture decision records with context, rejected alternatives and consequences (decision log) — the "why" survives the people.
- Failures are loud: a missing precondition stops a component in seconds with an explicit message — never a silent fallback (measured: bad discovery DNS or unreachable S3 abort in 3 s).
- Everything is retestable: each guide ends with a runnable verification suite; the platform can be rebuilt from an empty cluster and re-proven at any time.
Where this stands, and what comes next¶
The eight foundation blocks are built, tested and documented on this site (database, server image, clustering, S3 storage, ingestion, observations, basemaps, processing — see the home page status table). The open items are listed honestly in Risks and technical debt: validity-interval time for area messages (SIGMET/AIRMET/GAMET, TAF), request-time raster palettes, observability, real GTS decoders, retention governance, and the workspace layout decision (ADR-0002) to be made against the real data inventory.
How it works — a plain-language tour¶
This page explains the platform without assuming any technical background. If you want the engineering details, every section links to them — but you can understand the whole system from this page alone.
What this platform does¶
Weather models, satellites, radars, observation stations and hydrological sensors all produce data about the atmosphere and rivers. This platform collects that data and turns it into maps and figures that any application can request — a web viewer, a mobile app, an expert tool — using open, standardised requests ("give me the temperature map over Europe at 6 pm, at flight level 300").
flowchart LR
A[Data producers<br/>models · satellites · stations · radars] --> B[The platform]
B --> C[Any application<br/>web maps · expert tools · scripts]
C -- "one standard language (OGC)" --> B
The key word is standard: applications don't need to know anything about our internals, and we can change our internals without breaking them.
The restaurant analogy¶
Think of the platform as a restaurant:
- The pantry — all the data lives in two places: a database (facts and figures: observations, catalogs, indexes) and an object store (big files: satellite and model images). Nothing valuable is ever stored inside a server itself.
- The kitchen — GeoServer is the chef: when an application orders "temperature map, Europe, 6 pm", it fetches the right ingredients from the pantry and cooks the image on the spot.
- Several identical kitchens — we run several copies of GeoServer behind a single entrance. If one breaks, the others keep serving; on busy days we simply add more. They are identical and disposable: any of them can be thrown away and replaced in under a minute, because nothing is stored inside them (the proof).
- The pass-through counter — maps that were already cooked are kept in a shared cache: the second person asking for the same map gets it instantly, whichever kitchen originally cooked it (the proof).
flowchart TB
subgraph Entrance["One entrance"]
LB[Requests arrive]
end
LB --> K1[GeoServer copy 1]
LB --> K2[GeoServer copy 2]
LB --> K3[GeoServer copy N]
K1 & K2 & K3 --> DB[(Database<br/>facts, catalogs, indexes)]
K1 & K2 & K3 --> S3[(Object store<br/>big image files)]
K1 & K2 & K3 --> C[(Shared map cache)]
How new data appears — with nobody doing anything¶
When a new model run or satellite image arrives, an ingestion service converts it to an efficient open format, drops the file into the object store, and adds one row to an index in the database — "this file covers Europe, 6 pm, flight level 300, temperature, in kelvin".
That's all. Every GeoServer copy consults the index live, so the new data is immediately visible to every application, with no restart, no deployment, no manual step (the proof). The same goes for observations: the decoder writes rows, the maps update.
Time, altitude and units¶
- Every piece of data carries its date, and where relevant its altitude level and model run: applications can rewind, compare runs, and browse levels — the platform answers "at what times/levels do you have data?" as a standard question.
- Every measurement carries its unit. Sources disagree (aviation reports temperature in Celsius, models in kelvin): the platform converts everything to one canonical unit on the way in and never loses track of the original (why this matters).
Why it stays simple over time¶
- Nobody configures servers by hand. The entire server setup is written down as short text files; servers configure themselves from those files at start-up. Adding a data layer is a one-line change, reviewed like code. There is no "the person who clicked the right buttons has left" risk.
- Everything is rebuildable. From an empty computer cluster, the whole platform can be reconstructed from the repository — and each chapter of this documentation ends with a test that proves its part still works.
- Every decision is written down with its reasons and the options we rejected (the decision log) — thirteen so far.
- Failures are loud. When something is missing or broken, the component says so explicitly and stops, instead of limping along silently. Problems are found in seconds, not discovered weeks later.
What runs where¶
The platform runs on Kubernetes, a system that supervises applications on a computer cluster: it restarts what crashes, spreads the load, and lets us say "run three GeoServers" as a wish that it continuously enforces. The database and the object store are services the platform uses but doesn't manage — they can be provided by any infrastructure team.
Want the full picture? Start with the Vision, then the architecture and the step-by-step guides.
GIS Contribution — Foundation Workshop¶
This page answers the Product & Technology Foundation Workshop agenda for the GIS / geospatial visualization capability. It follows the agenda's own numbering and uses the formats the agenda asks for: lessons as Problem → Root Cause → New Principle, technology decisions as Need → Alternatives → Decision → Rationale backed by lightweight ADRs.
Everything stated here is already built and tested, not proposed on paper: a working reference implementation runs on a Kubernetes cluster, every claim below links to the guide or ADR that proves it, and each guide ships with a rerunnable test suite.
If you read only two pages
Vision — the direction in one page. How it works — the platform explained without jargon.
Part 1 — WHAT: GIS in the product landscape¶
GIS is a platform-level capability, not a pillar product (§1, §2)¶
Applying the Four-Pillar Architecture Check: geospatial visualization is needed by any pillar that touches maps, weather, aeronautics or hydrology. It therefore belongs in the Common Platform, below the pillar products:
flowchart TB
subgraph CP["Common Platform"]
GIS["GIS services<br/>(OGC APIs: maps, features, processing)"]
REF["Shared referential<br/>(parameters, units, level types)"]
STORE["Shared data services<br/>(spatio-temporal DB, object storage)"]
end
subgraph PP["Pillar Products"]
P1["Pillar 1 product"]
P2["Pillar 2 product"]
P34["Pillars 3 / 4 (future)"]
end
P1 --> GIS
P2 --> GIS
P34 -.-> GIS
GIS --> REF
GIS --> STORE
What the GIS capability provides to every product on top of it:
| Capability | What products get | Standard |
|---|---|---|
| Map rendering | Styled maps of any published layer, any projection | OGC WMS |
| Tiled maps | Fast, cacheable map tiles for web/mobile clients | OGC WMTS / TMS |
| Feature access | Raw vector data (stations, observations, zones) as GeoJSON/GML | OGC WFS |
| Geospatial processing | Server-side computation (e.g. interpolation of station networks) | OGC WPS |
| Time navigation | Every time-aware layer answers TIME= requests |
WMS dimensions |
| Vertical navigation | Model layers answer ELEVATION= (pressure level / altitude) |
WMS dimensions |
| Run selection | Model layers distinguish forecast runs (reference time) | WMS custom dimension |
| Styling | Palettes and symbolization applied server-side at request time | SLD |
Because everything is exposed through OGC standards, any client — our future UI, QGIS, a customer's existing tool — consumes the same services. No customer-specific branches at the GIS layer: customer needs are met with configuration (layers, styles, workspaces), not forks.
Target data landscape (§2)¶
The capability is designed against the real operational inventory, not a toy dataset (details):
- Observations from the GTS network: SYNOP, METAR — dense surface networks; TEMP, AMDAR — sparse upper-air.
- Aeronautical area products with validity periods: SIGMET, AIRMET, GAMET, TAF.
- Imagery: satellite (per satellite/sensor, time series), radar, lightning.
- NWP models: multi-run, multi-validity, multi-level rasters.
- Hydrology: stations, discharge/level series, basins, flood-related layers.
Client applications must navigate this data in time and altitude, and apply palettes at request time — both are native WMS mechanisms in the design (dimensions + SLD), not custom code.
What unlocks several pillars first (§3)¶
The highest-leverage foundation pieces, in build order — this is exactly the sequence the reference implementation followed (guides):
- Shared spatio-temporal database (schemas, roles, PostGIS/TimescaleDB) — everything else reads from it.
- The parameter/unit referential — one table answering "what is this quantity and in which unit is it stored" (ADR-0003). A measurement without a unit is a bug; K vs °C confusion must be structurally impossible.
- The GIS service itself, clustered and configured entirely by code.
- Ingestion conventions (file naming, index tables) so any producer can feed the platform without touching the GIS service.
Part 2 — HOW: lessons, principles, stack¶
§4 Lessons learned (Problem → Root Cause → New Principle)¶
From previous operational experience with GeoServer platforms, validated again during the reference build:
| Problem | Root Cause | New Principle |
|---|---|---|
| GIS server config drifted between environments; nobody could rebuild it | Configuration lived in a UI, applied by hand, stored on a local disk | Nobody configures by hand. All configuration is code (a startup plugin + one ConfigMap); the database is the single source of truth (ADR-0006) |
| Adding/replacing a server instance was a manual, risky operation | Instances were "pets": fixed IPs, local state, hand-tuned | Cattle, not pets. Instances boot from an empty disk, register via DNS discovery, and are disposable (ADR-0005, ADR-0008) |
| Failures surfaced days later as wrong maps, not at deploy time | Components fell back silently on missing preconditions | Fail loud. A missing database, bucket or DNS entry stops startup with an explicit exit code — never a degraded service |
| Same physical quantity stored in different units by different feeds | No shared referential; each decoder chose its own conventions | Units are first-class. Canonical unit per parameter, conversion at ingestion, source unit kept (ADR-0003) |
| Upgrades feared and postponed; versions rotted | Environment not reproducible, no way to test an upgrade safely | Everything retestable. Pinned artifacts, scripted builds, rerunnable test suites per subsystem — an upgrade is a branch + a test run |
| Understanding official specs (WMO/ICAO) from text alone led to wrong implementations | Figures and tables in the standards carry meaning the text does not repeat | Study the source documents, figures included, before modelling data |
These are offered as input to the workshop's initial Engineering Principles.
§5 Architecture principles — GIS stance¶
Position against each relevant workshop principle, with the evidence:
| Workshop principle | GIS stance | Evidence |
|---|---|---|
| API-first | Everything is an OGC API (WMS/WMTS/WFS/WPS); the UI is just another client | All guides test via API only |
| Standards-based interfaces | OGC + SQL + S3 protocol; no proprietary interface anywhere | ADR-0009 |
| Cloud-native but on-premise capable | Runs on plain Kubernetes (validated on k3s); S3 protocol served by self-hostable storage; no cloud-vendor dependency | Guide 3, Guide 4 |
| Containerization | Single custom image built on the official GeoServer image, plugins baked in | Guide 2 |
| Configuration outside application code | One ConfigMap declares layers/styles/mosaics; secrets in K8s Secrets; zero config in the image | ADR-0006 |
| Scalability | Replica count is a number in a manifest; scaling up/down measured with zero request errors | Guide 3 |
| Upgradeability | Ephemeral instances + DB as source of truth ⇒ upgrade = new image tag + rollout | ADR-0005 |
| Observability by design | Honest gap — health probes exist; metrics/tracing/dashboards are the next subsystem to design | Risks |
| Event-driven where appropriate | Deliberately not event-driven at the GIS boundary: ingestion writes files + one DB row; the GIS service discovers data through the DB index, no notification bus needed. Simpler, and one less moving part | ADR-0010 |
| No customer-specific forks | Customer variation = workspaces, layers, styles — configuration, never code | §2 above |
§6 Technology stack (Need → Alternatives → Decision → Rationale)¶
| Need | Alternatives considered | Decision | Rationale |
|---|---|---|---|
| GIS / visualization server | GeoServer 3, GeoServer 2.x, MapServer, thin tile servers (pg_tileserv/martin) | GeoServer 3.0.1 | Only candidate covering the full need (WMS+WFS+WPS+time/elevation dimensions+SLD+plugin ecosystem); v3 = supported line (Tomcat 11/Spring 6/JDK 21); prior team production experience; thin tile servers cover only a fraction of the need |
| Relational + geospatial + time-series | PostgreSQL+PostGIS, +TimescaleDB, dedicated TSDB (Influx…) | PostgreSQL + PostGIS + TimescaleDB (shared, operator-run) | One database engine for geospatial and time-series; hypertables for observation volumes; the platform DB is shared — GIS gets its own schemas/roles (Guide 1) |
| GIS configuration storage | File data dir on shared volume (RWX), DB-backed config | DB-backed (jdbcconfig/jdbcstore) | Shared-filesystem data dirs were the root cause of past drift and locking issues; DB gives one source of truth + instant propagation (measured ~3 s across replicas) |
| Object / file storage | S3-compatible self-hosted (SeaweedFS, MinIO), filesystem | S3 protocol; SeaweedFS for the bench | Rasters as Cloud-Optimized GeoTIFF read directly from S3; tile cache in S3; MinIO rejected (AGPL licensing risk); final production storage = platform-level decision, anything S3-compatible works (ADR-0009) |
| Clustering / session sharing | Fixed-member config, K8s-native DNS discovery | Hazelcast with DNS discovery | Replica count becomes a free variable; UI sessions shared across replicas; no fixed IPs anywhere (ADR-0008) |
| Dense-network interpolation (SYNOP/METAR) | IDW, splines | IDW as WPS rendering transformation | Computed server-side at render time, styled by SLD like any raster (Guide 8) |
| Sparse-network interpolation (TEMP/AMDAR) | IDW, Barnes, RBF, first-guess + corrections | First-guess + corrections | Comparative study on realistic sparse/clustered stations: RMSE 1.36 K vs 4.40 (IDW), 4.93 (Barnes), 6.77 (RBF) (ADR-0013) |
| In-house plugin distribution | Ad-hoc copies, company artifact repository | Company Nexus/Artifactory (to confirm) | Community plugins are multi-JAR bundles; in-house plugins are Maven builds — both need a proper artifact channel (ADR-0007) |
Full decision log: 13 ADRs, all in the lightweight format the agenda proposes.
Part 3 & 4 — Organizing and starting¶
§8 What GIS needs from the initial toolchain¶
- Artifact/container registry — the single hard dependency: the custom GeoServer image and in-house plugin JARs must live somewhere (ties into ADR-0007).
- CI able to run the existing per-subsystem test suites (plain bash + kubectl against a throwaway cluster).
- Nothing exotic: the whole reference stack runs on a single-node k3s.
§9 Sprint 0 — the vertical slice already exists¶
The agenda asks Sprint 0 to demonstrate Source → Ingestion → Messaging/Processing → Storage → API → UI. The reference implementation already covers most of it, end-to-end and tested:
flowchart LR
SRC["Source<br/>(model runs, sat images,<br/>observations, basemaps)"] --> ING["Ingestion<br/>(convert to COG,<br/>write index row)"]
ING --> ST["Storage<br/>(S3 + PostGIS/Timescale)"]
ST --> API["API<br/>(OGC: WMS/WFS/WPS,<br/>time + elevation + runs)"]
API --> UI["UI<br/>(any OGC client)"]
style UI stroke-dasharray: 5 5
Proven today (each item = a rerunnable test suite): database bootstrap, image build, 3-replica cluster with config propagation and shared sessions, S3 tiles + COG rasters, model/satellite mosaic ingestion with time+elevation+run dimensions, observation tables + views, basemaps, server-side interpolation. The full chain was re-deployed from scratch on a second cluster using only the written runbooks, with zero deviation — the strongest durability evidence we have.
Deliberately not built yet (platform-level decisions the workshop should own):
| Gap | Why it waits |
|---|---|
| Messaging/eventing | GIS doesn't need it (ADR-0010); if the platform adopts a bus, ingestion plugs into it upstream |
| Identity / RBAC integration | GIS must federate with the platform IdP — the IdP choice comes first |
| UI / visualization client | Any OGC client works today; the common UI framework choice belongs to the workshop |
| Observability stack | Needs the platform-wide monitoring choice; biggest known gap |
| Data lifecycle / retention | Mechanism is ready to build (time-indexed tables), but the retention owner and policy are not defined yet |
| Real GTS decoders | Ingestion conventions are fixed; decoders (SYNOP/METAR/BUFR/GRIB) are the first real product work |
§10 30/90-day view (GIS slice)¶
- First 30 days: registry + CI in place, reference stack redeployed on the team's environment (the runbooks are written and proven), workspace organization decided against the real inventory (ADR-0002 — deliberately still open), identity integration designed.
- First 90 days: first real GTS decoders feeding the platform, area products with validity periods (SIGMET/AIRMET/GAMET/TAF) as interval-based time layers, request-time palette mechanism generalized, observability subsystem.
Where to see it¶
- Documentation — this site: vision, plain-language explanation, arc42 architecture, 8 runbook guides, 13 ADRs.
- Live demo — a 3-replica cluster serving the full stack (model layers with time/elevation/run navigation, satellite mosaics, observations, interpolated surfaces, cached tiles) is deployed and can be shown during the workshop.
Architecture
1. Introduction and goals¶
System purpose¶
A platform for disseminating meteorological and geographic data, built around GeoServer deployed as a cluster on Kubernetes. It exposes OGC services (WMS, WMTS, WFS, WCS, WPS) for three domains — meteorology, aeronautics, hydrology — whose data share three dimensions: time, space and elevation (vertical level). It serves:
- rasters produced by weather models (several dimensions: model run, forecast validity time, vertical level) and by satellite imagery (time series);
- vector data stored in TimescaleDB + PostGIS;
- basemaps (downloaded sources or external services).
Documentation goals¶
- Describe a reproducible architecture: every component is configured through versionable artifacts (image, scripts, manifests), not through manual actions in the UI.
- Describe a GeoServer that is genuinely clustered: variable number of replicas, no fixed IP addresses, shared configuration in the database.
- Provide step-by-step guides validated by prototype for each building block.
Target data inventory (collected 2026-09-02)¶
Expected sources, mostly from the GTS network plus dedicated feeds — the workspace and layer organization (ADR-0002) will be decided against this real inventory, not against any assumption:
| Family | Products | Notes |
|---|---|---|
| Surface observations | SYNOP, METAR | point features, observation instant |
| Aeronautical messages | TAF, AIRMET / SIGMET / GAMET | TAF: aerodrome forecasts; the others are area products with a validity period (time intervals, not instants) |
| Imagery | satellite images, radar | radar has a fast refresh cycle |
| Events | lightning | high-volume point events |
| Models | NWP fields | run × validity × level |
| Hydrology | dedicated hydrological layers | to be specified |
Client applications navigating this data must be able to move through time and altitude, and to apply palettes to rasters at request time.
Stakeholders¶
| Role | Expectation |
|---|---|
| Architect | Overall view, decisions and their rationale |
| Ops / platform | Deployment, scalability, monitoring, recovery procedures |
| Developer | Layer publishing, data ingestion, APIs |
| Business user | Stable, high-performance OGC services |
2. Constraints¶
Technical¶
| Constraint | Detail |
|---|---|
| Platform | Kubernetes (no specific distribution mandated); the number of GeoServer replicas is driven by ops (kubectl scale, k9s, Rancher…) |
| Clustering | At least 3 replicas, variable count; no configuration based on fixed IP addresses |
| Configuration persistence | Database rather than file system: JDBCConfig, JDBCStore, JDBCStatus plugins |
| Database access | A JNDI resource declared in Tomcat; URL, user and password injected through environment variables / Secrets |
| Database | TimescaleDB with the PostGIS extension; dedicated GeoServer schema with a dedicated user and a configured search_path (the GeoServer schema is not the only schema in the database) |
| Object storage | S3 (or compatible) for the GeoWebCache tile cache and for rasters |
| GeoServer extensions | WPS mandatory; S3 GWC and S3 raster plugins; in-house plugins (configuration initialization, WPS interpolation process) built and embedded in the image |
| Administration UI | Usable in a cluster: UI user sessions must survive routing across replicas (affinity or session replication) |
| Best practices | Follow the official "GeoServer in production" recommendations |
| Dimensions | Every piece of data carries at least a temporal and/or vertical dimension in addition to space; these dimensions must be exposed by the OGC services (TIME, ELEVATION, custom dimensions) |
| Client navigation | Client applications must navigate through time and altitude across all layers, including area messages whose temporal dimension is a validity interval (AIRMET/SIGMET/GAMET, TAF) |
| Raster palettes | Palettes must be applicable to rasters at request time (named styles and/or SLD env() substitution) — future work item |
| Data lifetime | A retention mechanism is expected, but the owner of the retention decision is not identified yet — record, do not invent policies |
| Units of measurement | A measurement always has a unit. The same quantity may arrive in different units depending on the source (temperature in kelvin or degrees Celsius, pressure in Pa or hPa, wind in m/s or knots). The unit must be known, stored with the data, and kept under control all the way to rendering and service outputs |
Organizational¶
- Documentation in English, structured following arc42, maintained together with the code.
- Every technical chapter is validated by a prototype before it is written.
Conventions¶
- Generic naming of components (
geoserver,db,s3,ingestion). - One structuring decision = one ADR.
3. Context and scope¶
Business context¶
flowchart LR
subgraph Clients
WEB[Web mapping applications]
SIG[Desktop GIS clients]
API[Automated processing]
end
subgraph Platform
GS[GeoServer ×N<br/>WMS · WMTS · WFS · WCS · WPS]
ING[Ingestion service]
end
subgraph Storage
DB[(TimescaleDB + PostGIS<br/>GeoServer catalog · vectors · granule index)]
S3[(S3<br/>rasters · tile cache)]
end
subgraph Sources
NWP[Weather models]
SAT[Satellite imagery]
VEC[Vector feeds]
BASE[External basemaps]
end
WEB & SIG & API --> GS
GS <--> DB
GS <--> S3
NWP & SAT --> ING
ING --> S3
ING --> DB
VEC --> DB
BASE -.cascade.-> GS
Technical context¶
| Interface | Protocol | Direction |
|---|---|---|
| Clients → GeoServer | HTTP/HTTPS, OGC services | inbound |
| GeoServer → database | JDBC via JNDI (Tomcat) | outbound |
| GeoServer → S3 | S3 API (HTTPS) | outbound |
| Ingestion → S3 / database | S3 API, SQL | outbound |
| GeoServer ↔ GeoServer | configuration propagation (see chapter 8) | internal |
Out of scope¶
- Production of the source data (model execution, satellite reception).
- Client applications.
4. Solution strategy¶
To be written as sub-projects 1 to 7 progress: a one-page summary of the main choices (database as the source of truth, S3 for bulky data, configuration as code, clustering without local state).
5. Building block view¶
5.1 Database (sub-project 1)¶
| Responsibility | Host the GeoServer configuration schema (catalog, resources, WPS statuses) and the data schemas of the producing services |
| Ours | geoserver role and schema, search_path, reader roles on the data schemas, initialization Jobs |
| Operator's | PostgreSQL/TimescaleDB instance, PostGIS extension if the provided account is not superuser, backups, high availability |
| Interfaces | JDBC via JNDI (jdbc/geoserver, jdbc/<data>); SQL via psql for the Jobs |
| Artifacts | poc/01-database/, guide 1, ADR-0004 |
5.2 GeoServer image (sub-project 2)¶
| Responsibility | Serve the OGC services; load its entire configuration from the database; initialize the catalog through code |
| Contents | Official GeoServer 3.0.1 + wps, wps-jdbc, gwc-s3, control-flow, gs-jdbcconfig, gs-jdbcstore, geoserver-init plugin, JNDI server.xml, bootstrap-datadir.sh |
| Interfaces | HTTP 8080 (OGC, REST, UI); JNDI jdbc/geoserver (R/W) and jdbc/meteo (R) |
| Local state | Nothing durable: emptyDir data dir regenerated at boot |
| Artifacts | poc/02-geoserver-image/, guide 2, ADR-0005, ADR-0006 |
5.3 Cluster (sub-project 3)¶
| Responsibility | Make N replicas operate as a single GeoServer: change propagation, IP-free discovery, UI sessions, shared WPS |
| Contents | gs-hz-cluster module (Maven bundle), headless Service geoserver-hz, affinity-enabled Ingress, WPS RWX PVC, PDB, geoserver-cluster-env ConfigMap |
| Interfaces | Hazelcast 5701 between pods; HTTP 8080 via Service/Ingress |
| Artifacts | poc/03-clustering/, guide 3, ADR-0008 |
5.4 S3 object storage (sub-project 4)¶
| Responsibility | Shared tile cache (gwc-tiles bucket) and COG rasters (rasters bucket) |
| Ours | Default S3 blobstore and COG stores created by the plugin; COG production (ingestion) |
| Operator's | S3-compatible service, buckets, credentials, retention |
| Interfaces | S3 API (path-style) from the pods; gwc-s3 and S3 RangeReader (IIO_S3_AWS_*) |
| Artifacts | poc/04-s3/, guide 4, ADR-0009 |
5.5 Ingestion service and mosaics (sub-project 5)¶
| Responsibility | Produce the COGs on S3, maintain the granule index and the quantity reference table, create the mosaic views |
| Contents | Python + GDAL container (ingest.py), mosaic schema, Jobs; on the GeoServer side: plugin (MOSAICS) |
| Interfaces | S3 (write), SQL (ingestion account); GeoServer reads the index through jdbc/meteo and the COGs through range reads |
| Artifacts | poc/05-ingestion/, guide 5, ADR-0010 |
5.6 Decoded observations (sub-project 6)¶
| Responsibility | Store decoded surface observations (canonical unit, source unit kept) and expose them as time-qualified layers |
| Contents | Schema obs (station, observation hypertable, publication views), decoder simulator, VECTOR_LAYERS plugin publication |
| Interfaces | SQL (decoder account, write); jdbc/meteo (GeoServer, read-only); WMS TIME / WFS / CQL |
| Artifacts | poc/06-vector/, guide 6, ADR-0011 |
5.7 Basemaps (sub-project 7)¶
| Responsibility | Background layers: hosted reference data (Natural Earth) and optional cascaded external WMS, both S3-tile-cached |
| Contents | Schema basemap + ogr2ogr load Job; plugin: per-schema datastore, code-managed SLD style, WMSStore cascade |
| Interfaces | SQL (basemap role, load); jdbc/meteo (read); external WMS (cascade); WMS/WMTS out |
| Artifacts | poc/07-basemaps/, guide 7, ADR-0012 |
5.8 Processing (sub-project 8)¶
| Responsibility | Turn point observations into continuous fields; host the in-house processes |
| Contents | meteo-wps module (meteo:IDWInterpolation, WPS + rendering transformation), obs-idw-temperature style, sparse-interpolation study |
| Interfaces | WPS Execute; SLD <Transformation> on observation layers |
| Artifacts | poc/08-processing/, poc/02-geoserver-image/meteo-wps/, guide 8, ADR-0013 |
| Future work | Production first-guess + corrections process combining mosaics and observations (ADR-0013) |
6. Runtime view¶
6.1 Startup of a GeoServer replica (sub-project 2)¶
sequenceDiagram
participant K as Kubernetes
participant B as bootstrap-datadir.sh
participant DB as Database (geoserver schema)
participant T as Tomcat
participant GS as GeoServer (JDBCConfig, JDBCStore)
participant I as geoserver-init
K->>B: starts the container (env, Secrets, emptyDir)
B->>B: variable checks
B->>DB: SELECT to_regclass('geoserver.object'), ('geoserver.resources')
DB-->>B: present? → initdb/import=false (otherwise true)
B->>B: writes jdbcconfig/jdbcstore/status/controlflow .properties
B->>T: exec startup.sh (server.xml envsubst, admin, SKIP_DEMO_DATA)
T->>T: binds jdbc/geoserver and jdbc/meteo (pools)
T->>GS: starts the webapp
GS->>DB: loads catalog (object) and resources (resources, including security/)
GS->>I: GeoServerInitializer.initialize()
I->>GS: workspace / JNDI datastore / layer: created or verified
GS-->>K: /geoserver/index.html 200 (probes)
First boot: initdb/import=true, JDBCConfig and JDBCStore create their
tables and import the initial data dir (including security/). Subsequent
boots: everything is re-read from the database; the local data dir only
contains the files regenerated by the bootstrap.
6.2 Scaling from 3 to 5 replicas (sub-project 3)¶
sequenceDiagram
participant Ops as Ops (kubectl scale / k9s)
participant K as Kubernetes
participant DNS as Headless Service DNS
participant N as New pod
participant C as Hazelcast cluster (3 members)
participant DB as Database
Ops->>K: replicas 3 → 5
K->>N: creates the pod (emptyDir, env)
K->>DNS: publishes the pod IP (publishNotReadyAddresses)
N->>DNS: resolves geoserver-hz (bootstrap, pre-check)
N->>N: regenerates cluster/*, jdbc*/*.properties
N->>DB: loads catalog and resources
N->>C: joins (5701) → size=4, then 5
N-->>K: Ready (probes) → receives traffic
Note over C: a configuration change on one node is published to the others (event)
Measured: 5 pods ready in 25 s, 0 errors on in-flight requests; scale-down
5 → 2 without errors (graceful shutdown, terminationGracePeriodSeconds).
6.3 Tile request (sub-project 4)¶
sequenceDiagram
participant C as WMTS client
participant P as GeoServer pod (any of them)
participant G as GeoWebCache
participant S3 as gwc-tiles bucket
participant R as Rendering engine
C->>P: GetTile (layer, gridset, z/x/y)
P->>G: tile?
G->>S3: GET object
alt HIT
S3-->>G: tile
else MISS
G->>R: rendering (internal WMS)
R-->>G: image
G->>S3: PUT object
end
G-->>C: tile + header geowebcache-cache-result: HIT|MISS
The cache is shared: a tile rendered by one pod is served as a HIT by all the others, including pods created afterwards.
6.4 Arrival of a new granule (sub-project 5)¶
sequenceDiagram
participant I as Ingestion service
participant S3 as rasters bucket
participant DB as mosaic index (PostGIS)
participant G as GeoServer (any pod)
participant C as Client
I->>I: field → canonical unit → COG
I->>S3: PUT models/…/arpege_t_isobaric85000_run_valid.tif
I->>DB: INSERT model_granule (time, elevation, reference_time, source_unit)
C->>G: GetCapabilities
G->>DB: SELECT DISTINCT time, elevation… FROM mosaic.arpege_t
G-->>C: up-to-date dimensions (new forecast validity time included)
C->>G: GetMap TIME=… ELEVATION=…
G->>DB: matching granules
G->>S3: range reads of the COG
G-->>C: image
No restart and no notification: measured, the new forecast validity time is visible on all pods as soon as the row is inserted.
7. Deployment view¶
7.1 Kubernetes topology (state after sub-project 3)¶
flowchart LR
U[Clients / administrators] --> ING[Ingress geoserver<br/>cookie affinity]
ING --> SVC[Service geoserver :8080]
SVC --> D[Deployment geoserver<br/>replicas N ≥ 3<br/>emptyDir data dir]
D --- HZ[Headless Service geoserver-hz :5701]
D --- PVC[(PVC RWX geoserver-wps)]
D --> DBS[(TimescaleDB Service<br/>existing, shared)]
D --> S3[(S3 object storage<br/>gwc-tiles · rasters)]
subgraph Config
S1[(Secret geoserver-db)]
S2[(Secret meteo-reader)]
S3[(Secret geoserver-admin)]
C1[ConfigMap geoserver-env]
C2[ConfigMap geoserver-cluster-env]
end
Config -.-> D
J[Jobs init-geoserver-db / grant-reader] --> DBS
SA[(Secret db-admin)] -.-> J
PDB[PodDisruptionBudget minAvailable 1] -.-> D
| Object | Role | Sub-project |
|---|---|---|
Jobs init-geoserver-db, grant-meteo-reader + Secret db-admin |
Database initialization (pre-install / PreSync hook) | 1 |
Secrets geoserver-db, meteo-reader, geoserver-admin |
Application accounts | 1, 2 |
ConfigMaps geoserver-env, geoserver-cluster-env |
Non-secret variables (JNDI, control-flow, cluster, WPS) | 2, 3 |
Deployment geoserver |
N replicas, reproducible image, emptyDir data dir, probes |
2, 3 |
Service geoserver |
ClusterIP HTTP | 2 |
Headless Service geoserver-hz |
Hazelcast discovery through DNS | 3 |
Ingress geoserver |
Exposure + session affinity | 3 |
PVC geoserver-wps (RWX) |
Shared WPS results | 3 |
| PodDisruptionBudget | At least 1 replica during maintenance operations | 3 |
Secret s3-credentials |
S3 endpoint, keys, buckets (optional: without it, ephemeral local cache) | 4 |
To be completed: ingestion service (5).
7.2 Platform requirements¶
- Kubernetes with an Ingress controller supporting session affinity (or
HZ_SESSION_SHARING=true). - RWX storage for asynchronous WPS.
- Internal DNS access to the headless Service (standard).
- Existing TimescaleDB/PostGIS database, with an admin account provided for the initialization Job.
- S3-compatible object storage (path-style) with two buckets and one key pair.
- Image registry and artifact repository (ADR-0007).
8. Cross-cutting concepts¶
To be written together with sub-projects 2 to 6: configuration persistence (initialization code rather than REST), secret injection, health probes, JVM settings, logging and monitoring.
8.0 Data sources and secrets¶
flowchart TB
subgraph Secrets
A[(db-admin<br/>operator)]
B[(geoserver-db)]
C[(meteo-reader)]
end
A --> J[Initialization Jobs]
B --> J
C --> J
B --> GS[GeoServer ×N]
C --> GS
GS -- "JNDI jdbc/geoserver · R/W" --> S1[(geoserver schema)]
GS -- "JNDI jdbc/meteo · R" --> S2[(decoder schema)]
DEC[Decoding service] -- "its own account · W" --> S2
J -- "creates roles, schema, grants" --> S1
J -- "reader role" --> S2
- One Secret per account; the admin Secret is only mounted in the Jobs.
- One JNDI resource per (URL, account) pair; GeoServer stores reference the JNDI name. Details: ADR-0004.
8.0 bis Configuration persistence and JVM¶
- The database is the single source of truth: catalog (JDBCConfig),
resources including
security/(JDBCStore), WPS statuses (status). A pod's data dir is ephemeral and regenerated at startup (ADR-0005). - Reproducible configuration through a Java plugin
GeoServerInitializer, idempotent, driven by the environment; REST/UI reserved for manual operations (ADR-0006). - JVM:
-XX:MaxRAMPercentage=70— the pod memory limit is the only value to tune;ExitOnOutOfMemoryErrorso that an OOM restarts the pod rather than leaving a zombie process behind.
8.1 Parameters, dimensions and granule index¶
Status
Model from ADR-0003, implemented in sub-project 5 for rasters (mosaic schema: parameter, parameter_code, level_type, model_granule, sat_granule, per-mosaic views). The observations part (OBSERVATION) is implemented in sub-project 6 (schema obs: station, observation hypertable, publication views — see guide 6).
Quantity reference table and granule index¶
erDiagram
PARAMETER ||--o{ PARAMETER_CODE : "known as"
PARAMETER ||--o{ MODEL_GRANULE : "quantity"
PARAMETER ||--o{ OBSERVATION : "quantity"
LEVEL_TYPE ||--o{ MODEL_GRANULE : "level type"
SAT_GRANULE }o--|| PARAMETER : "quantity"
PARAMETER {
text param_key PK "e.g. air_temperature"
text cf_standard_name
text canonical_unit "UCUM: K, Pa, m/s, %"
text label_fr
}
PARAMETER_CODE {
text param_key FK
text scheme "grib2 | eccodes | bufr | metar | iwxxm | cf"
text code "0-0-0 | t | 0 12 101 | M03 …"
text unit_in_scheme "K | K | K | Cel"
}
LEVEL_TYPE {
smallint code PK "GRIB2 table 4.5"
text name "surface, isobaric, height above ground…"
text unit "Pa, m"
}
MODEL_GRANULE {
int fid PK
geometry the_geom "footprint, EPSG:4326"
text location "s3://bucket/prefix/file.tif"
text model "arpege, arome, ifs…"
text param_key FK
timestamptz reference_time "model run → DIM_REFERENCE_TIME"
timestamptz time "validity → TIME"
numeric elevation "level value → ELEVATION"
smallint level_type FK
text level_label "e.g. FL300"
text source_unit "actual unit of the file"
timestamptz ingested_at
}
SAT_GRANULE {
int fid PK
geometry the_geom
text location
text satellite "OSCAR/Space, e.g. meteosat-12"
text instrument "e.g. fci"
text product "e.g. ir105, rgb-natural"
text param_key FK "e.g. brightness_temperature"
timestamptz time "→ TIME"
text source_unit
timestamptz ingested_at
}
OBSERVATION {
text station_id
timestamptz time "TimescaleDB hypertable"
text param_key FK
double value "in canonical unit"
text source_unit "K, Cel, kt, hPa…"
text message_type "SYNOP | METAR | SPECI | TAF"
}
Rules¶
- One GeoServer layer = one quantity = one unit = one style. One
ImageMosaic per (model, quantity) pair or per (satellite, instrument,
product) triple, filtered by an SQL view over the granule table
(
UseExistingSchema=true,AdditionalDomainAttributesfor the model run). - Exposed dimensions:
TIME(forecast validity time or observation instant),DIM_REFERENCE_TIME(model run),ELEVATION(vertical level). Two different level types = two layers. - File naming: redundant with the table, never the source of truth (patterns in ADR-0003).
8.2 Units of measurement¶
Principle: a measurement always has a unit.
- Each quantity in the reference table carries a canonical unit (K, Pa, m/s…), chosen based on the CF standard names.
- The same quantity arrives in different units depending on the carrier
(kelvin in GRIB2 and BUFR, degrees Celsius in METAR, hectopascal for QNH,
knots for aeronautical wind): the ingestion converts to the canonical
unit and keeps the source unit (
source_unit) next to the value. - The unit of a layer is published in its metadata and its legend; service outputs (GetFeatureInfo, WFS, WCS, WPS) return values in the canonical unit. Exact mechanism to be validated in sub-projects 5 and 6.
9. Architecture decisions¶
Structuring decisions are recorded as ADRs: see the decision index.
10. Quality requirements¶
| Requirement | Scenario | Measurement on the test bench (sub-project 3) |
|---|---|---|
| Horizontal scalability | kubectl scale 3 → 5 → 2 during continuous GetMap requests |
0 errors / 95 requests; 5 pods ready in 25 s |
| Configuration consistency | Catalog change on one node | Visible on the others within 3 s, without restart |
| Resilience to pod replacement | Deletion of a pod | New pod ready in ~20 s, configuration re-read from the database (sub-project 2) |
| Full restart | 0 → 3 simultaneous replicas | 3 pods ready in ~55 s, a single cluster |
| Explicit failure | Discovery impossible (wrong DNS) | Stops within 3 s with a message |
| UI usability in a cluster | 10 authenticated requests in round-robin | 10/10 with Ingress affinity or session sharing |
| Reproducibility | Replayable image and Jobs | Reproducible build, idempotent Jobs (sub-projects 1-2) |
11. Risks and technical debt¶
| Risk | Impact | Response |
|---|---|---|
| PostGIS not installable (admin account not superuser, extension missing) | Init Job fails, GeoServer does not start | Checked first; explicit failure with exit code 3 and the command to hand over to the operator, nothing is created (sub-project 1) |
DEFAULT PRIVILEGES granted to the wrong role |
New decoder tables invisible to GeoServer | The script reads the schema owner; re-running grant-reader catches up on existing objects (sub-project 1) |
Pre-existing cluster-level geoserver role, created by another account |
ALTER ROLE refused (PostgreSQL 16+: only the creator or a superuser may modify it) |
Explicit PostgreSQL error; convention: the role is created by the init Job with the provided admin account (sub-project 1) |
| Password rotation not propagated | GeoServer loses its connection after rotation | Procedure: Secret → Job → GeoServer restart; JNDI pool read at startup (sub-project 2) |
| Loss of the catalog encryption key | Store passwords undecryptable, catalog unusable | Covered: security/ (master key, keystore) carried in the database by JDBCStore, identical for all replicas (sub-project 2, ADR-0005); the database backup includes the key |
| Divergence between database and disk | Inconsistent behavior across replicas | Covered: ephemeral data dir regenerated, no durable write to disk (sub-project 2) |
| Configuration made through REST/UI not persisted or not propagated | Lost on restart, divergent replicas | Covered: configuration through the GeoServerInitializer plugin (ADR-0006); REST reserved for manual operations |
| GeoServer admin password rotation | GEOSERVER_ADMIN_PASSWORD is only imported at first boot |
Covered: PUT /rest/security/self/password on one node, effective on all (sub-project 3, test 07) — to be integrated into the operations procedure |
Problem while reinitializing Logging from JDBC Config message on subsequent boots |
Logging possibly left at the default level | Observed as non-blocking; to be qualified (sub-project 3) |
| Clustering blocked at startup | Replicas unavailable | Covered: DNS discovery without RBAC, pre-check and explicit stop within 3 s (sub-project 3, test 04) |
| RWX storage unavailable on the target platform | Asynchronous WPS results unreadable from the other nodes | Documented platform requirement (ch. 7.2); fallback: disable result storage |
| Loss of the node holding a UI session (affinity) | User logged out | Acceptable for an administration UI; HZ_SESSION_SHARING=true validated if needed |
| Keystore password (master key) left at its default | Warning in the UI; secrets encrypted with a known key | To address: change via PUT /rest/security/masterpw (or the UI) after the first boot, persisted in the database by JDBCStore; to be integrated into the initialization plugin or the operations procedure (sub-project 4 or later) |
| S3 tile cache without an expiration policy | Unbounded bucket growth | To define: retention by prefix/age on the S3 side, or GWC seeding/truncation (out of scope for sub-project 4) |
| Granule retention (S3 + index) undefined | Unbounded growth | To define together with the retention policy (per quantity/model); consistent S3 + index deletion by the ingestion service |
| Observation retention (hypertable) not defined | Unbounded table growth | To be defined with the data owners (TimescaleDB drop_chunks policy) — sub-project 6 |
| Retention decision ownership unclear | Policies cannot be committed | Open: identify who owns data-lifetime decisions (per family) before implementing any purge |
| Request-time raster palettes not implemented | Client requirement (2026-09-02) unmet | Future work: named styles per palette and/or env()-parameterized ColorMaps |
| Validity-interval time dimension (AIRMET/SIGMET/GAMET, TAF) not modelled | Area messages cannot be time-navigated yet | Future work: interval-based TIME (start/end attributes) on vector layers |
| Cascaded basemap provider unavailable or terms changed | Background layer degraded | Optional by design (ConfigMap), hosted basemap is the default; S3 cache keeps recent tiles |
| Dependency on public repositories (SourceForge, OSGeo Nexus) to build the image | Build impossible if an artifact is withdrawn | Corporate artifact repository (ADR-0007, proposed) |
12. Glossary¶
| Term | Definition |
|---|---|
| AMDAR / TEMP | Upper-air observations: AMDAR (sensors aboard commercial aircraft), TEMP (radiosonde soundings) — scattered data, vertical profiles |
| ADR | Architecture Decision Record — written record of an architecture decision |
| arc42 | Architecture documentation template in 12 chapters |
| BUFR | WMO binary format (FM 94) for observations; its descriptors (Table B) identify the quantities |
| Barnes (objective analysis) | Interpolation method from scattered points to a grid using iterative Gaussian weighting; built into GeoServer (vec:BarnesSurface) |
| Blobstore | GeoWebCache tile storage backend (disk, S3…); a default blobstore receives the layers without an explicit assignment |
| GeoServer catalog | The set of workspaces, stores, layers and styles known to GeoServer |
| COG (Cloud Optimized GeoTIFF) | GeoTIFF organized in internal tiles and overviews, readable through HTTP range requests directly from object storage |
| Cascade (WMS) | GeoServer republishing layers of a remote WMS through a WMSStore; the remote service does the rendering |
| CF standard name | Vocabulary of the CF conventions: standardized name of a physical quantity and its canonical unit |
| DEFAULT PRIVILEGES | PostgreSQL privileges automatically applied to future objects created by a given role in a schema |
| Domain | Business family of data: meteorology, aeronautics, hydrology |
| Data dir | GeoServer configuration directory; ephemeral here, regenerated at every startup, the database being the source of truth |
| GeoServerInitializer | GeoServer extension point invoked after the catalog is loaded; where configuration as code lives |
| Granule | Elementary raster file (one instant, one level) indexed in an ImageMosaic |
| Hazelcast | In-memory data grid used by the hz-cluster module to broadcast configuration events between replicas |
| GWC | GeoWebCache — the tile cache embedded in GeoServer |
| Granule (index) | Row of the PostGIS index describing a COG: S3 location, footprint, dimensions, source unit |
| Hypertable | TimescaleDB table transparently partitioned by time; backs the observation store |
| ImageMosaic | GeoServer store aggregating granules along dimensions (time, elevation, custom dimensions) |
| First guess (model background) | Model field used as the starting point of an analysis, corrected with observed innovations (ADR-0013) |
| IDW | Inverse Distance Weighting — interpolation weighted by inverse distance, suited to dense networks (SYNOP, METAR) |
| IWXXM | XML exchange model for aeronautical messages (METAR, TAF, SIGMET…), ICAO Annex 3 standard |
| JDBCConfig | GeoServer extension storing the catalog in the database |
| JDBCStore | GeoServer extension storing resources (styles, icons, configuration files) in the database |
| JDBCStatus | GeoServer extension storing task states/statuses in the database (shared between replicas) |
| JNDI | Java resource directory; here, the data sources declared in Tomcat (jdbc/geoserver, jdbc/meteo) |
| Master password (master key) | GeoServer secret encrypting the stored passwords (keystore); must be identical on all replicas |
| METAR / SPECI / TAF | Aeronautical observation messages (METAR, SPECI) and aerodrome forecast messages (TAF), ICAO Annex 3 |
| OGC | Open Geospatial Consortium — WMS, WMTS, WFS, WCS, WPS standards |
| Rendering transformation | GeoTools process invoked from an SLD <Transformation>; computes derived data (e.g. an interpolated surface) at rendering time |
| Replica | GeoServer instance (pod) in the cluster |
| Model run | Execution of a weather model at a given time (reference time); produces forecast validity times |
| search_path | Ordered list of the PostgreSQL schemas in which an unqualified name is looked up; set here on the role |
| Superuser | Unrestricted PostgreSQL role; required for CREATE EXTENSION postgis |
| Headless Service | Kubernetes Service without an address (clusterIP: None) whose DNS returns the IP of each pod; the basis of Hazelcast discovery |
| Session affinity | Routing of all requests from the same client to the same pod (cookie set by the Ingress) |
| PDB (PodDisruptionBudget) | Minimum number of pods to keep during voluntary operations (drain, update) |
| SYNOP | Surface weather observation from a station, nowadays exchanged in BUFR |
| TAC | Traditional Alphanumeric Code — WMO textual codes (METAR, TAF, legacy SYNOP FM 12) |
| Canonical unit | Reference unit chosen for a quantity within the platform (e.g. kelvin for temperature); any incoming data in another unit is converted or labeled |
| Validity | The instant for which a forecast is valid |
| Elevation / level | Vertical dimension of a piece of data (altitude, pressure level, depth) |
| WAFS | World Area Forecast System — global aeronautical grids (wind, temperature, icing, turbulence) distributed as GRIB2 |
| WMTS | Web Map Tile Service — pre-cut tile service, served here by GeoWebCache |
| WPS | Web Processing Service — execution of geospatial processes |
Guides
Guides¶
The guides are reproducible step-by-step walkthroughs. Each one:
- starts with a Prerequisites section;
- relies on the artifacts in the repository's
poc/<NN>-<name>/folder; - ends with a Verification section: commands to run and the expected result.
| Guide | Sub-project | Artifacts |
|---|---|---|
| Database | 1 | poc/01-database/ |
| GeoServer image | 2 | poc/02-geoserver-image/ |
| Clustering | 3 | poc/03-clustering/ |
| S3 storage | 4 | poc/04-s3/ |
| ImageMosaic ingestion | 5 | poc/05-ingestion/ |
| Vector data | 6 | poc/06-vector/ |
| Basemaps | 7 | poc/07-basemaps/ |
| Processing and interpolation | 8 | poc/08-processing/ |
Database — GeoServer configuration schema¶
The TimescaleDB database (PostgreSQL + timescaledb + postgis) is a
required, pre-existing, shared and independent service: other services use
it and another team operates it. This guide describes what GeoServer installs
into it and how, without ever touching the rest of the database.
flowchart LR
OPS[Database operator] -- "provides" --> SA[(db-admin Secret)]
SA --> JOB[init-geoserver-db Job]
SG[(geoserver-db Secret)] --> JOB
JOB -- "creates / refreshes" --> DB[(Shared TimescaleDB<br/>geoserver schema · PostGIS · search_path)]
SG --> GS[GeoServer ×N<br/>JNDI jdbc/geoserver]
GS --> DB
DEC[Decoding service] -- "owns" --> DS[(data schema)]
SR[(meteo-reader Secret)] --> JOB2[grant-reader Job]
JOB2 -- "read-only" --> DS
SR --> GS
GS -. "JNDI jdbc/meteo (read)" .-> DS
Related decision: ADR-0004.
Artifacts: poc/01-database/ (scripts, manifests, README).
Prerequisites¶
- A PostgreSQL 14+ database reachable from the cluster, with an admin
account provided by the operator. If PostGIS is not yet installed in the
database, this account must be a superuser — otherwise ask the operator to
run
CREATE EXTENSION postgis SCHEMA postgis;. kubectlon the target namespace; thepostgres:17-alpineimage reachable (it is only used to providepsql).
To test without an existing database, a test bench is provided
(k8s/test-timescaledb.yaml, k8s/test-demo-schema.yaml) — it is not a
reference deployment.
What the initialization script does¶
scripts/init-geoserver-db.sql, driven by init-geoserver-db.sh, in six
idempotent steps. The PostGIS precondition is checked first: on failure,
nothing has been created.
| # | Action | Detail |
|---|---|---|
| 1 | PostGIS | Detects the schema where the extension is already installed (shared database: often public). Absent → installation into postgis if superuser, otherwise explicit failure (exit code 3) |
| 2 | geoserver role |
CREATE ROLE if absent, then ALTER ROLE … PASSWORD on every run (rotation = re-run); NOSUPERUSER NOCREATEDB NOCREATEROLE |
| 3 | geoserver schema |
CREATE SCHEMA IF NOT EXISTS … AUTHORIZATION geoserver — the JDBCConfig / JDBCStore / WPS JDBC plugins will create their tables there on GeoServer's first startup |
| 4 | Privileges | USAGE on the PostGIS schema and on public |
| 5 | search_path |
ALTER ROLE geoserver SET search_path = geoserver, <PostGIS schema>, public — set on the role, hence effective for every connection, whatever the JDBC URL |
| 6 | Summary | Role, schema, PostGIS schema and version, TimescaleDB version, role settings |
Everything is parameterized through environment variables (table in the
README); no value is hard-coded. The SQL uses \gexec with
format('%I' / '%L') — no shell interpolation, hence no injection through the
names.
No silent fallback
PostGIS missing without superuser, missing variable, non-existent data
schema: the script stops with a message and a dedicated exit code
(2, 3, 4). A Job in Error state is information, not noise.
Two psql pitfalls hit while prototyping
\gseton a query that returns no rows is an error underON_ERROR_STOP. Wrap the query in a scalar subquery (SELECT (SELECT …) AS var): it always returns one row, andNULLleaves the variable undefined (:{?var}).ALTER ROLE … SET search_pathonly applies at connection time. After aSET ROLEin a session opened by another account, thesearch_pathremains that of the initial account — hence a puzzlingtype "geometry" does not exist. Issue an explicitSET search_path.
Multiple data sources¶
GeoServer uses one JNDI resource per (URL, account) pair:
| Resource | Account | Schema | Privileges | Secret |
|---|---|---|---|---|
jdbc/geoserver |
geoserver |
geoserver |
read / write (owner) | geoserver-db |
jdbc/meteo (example) |
geoserver_meteo_reader |
decoding service's schema | read-only | meteo-reader |
The grant-reader.sql script creates the reader role on an existing
schema: USAGE, SELECT on the tables and sequences present, and
ALTER DEFAULT PRIVILEGES FOR ROLE <owner> for future tables.
DEFAULT PRIVILEGES: classic pitfall
Default privileges only apply to objects created by the specified role
(FOR ROLE). The script reads the schema owner and uses it; it must
therefore be run by a superuser or by that owner. If the decoding service
creates its tables with another role, re-run grant-reader (the
GRANT … ON ALL TABLES statements catch up on existing objects).
The order between the decoding service and GeoServer is irrelevant: if the decoder installs PostGIS first, the GeoServer init detects it; otherwise the init installs it and the decoder finds it. Both paths were exercised on the test bench.
The declaration of the JNDI resources on the Tomcat side is covered in guide 2.
Steps¶
1. Test bench (optional)¶
K="kubectl -n meteo-gis"
kubectl apply -f k8s/namespace.yaml -f k8s/test-timescaledb.yaml
$K rollout status statefulset/timescaledb --timeout=300s
kubectl apply -f k8s/test-demo-schema.yaml
$K wait --for=condition=complete job/test-demo-schema --timeout=120s
The timescaledb-ha image initializes then restarts the server on first
launch: pg_isready returns true in between. Wait for a real, stable query
(SELECT 1) before launching the Jobs.
2. Secrets¶
Copy the k8s/secret-*.example.yaml templates, replace the changeme-*
values, apply. The db-admin Secret is referenced only by the Jobs;
geoserver-db and meteo-reader will also be referenced by GeoServer.
3. Scripts and Jobs¶
kubectl apply -f k8s/configmap-scripts.yaml
kubectl apply -f k8s/job-init-geoserver-db.yaml
$K wait --for=condition=complete job/init-geoserver-db --timeout=120s
$K logs job/init-geoserver-db
kubectl apply -f k8s/job-grant-reader.yaml
$K wait --for=condition=complete job/grant-meteo-reader --timeout=120s
$K logs job/grant-meteo-reader
If the data schema does not exist yet (the producer service has not run), the Job fails explicitly (exit code 4):
4. Integration into the deployment¶
- Helm: annotate the Job with
helm.sh/hook: pre-install,pre-upgradeandhelm.sh/hook-delete-policy: before-hook-creation; it runs before the GeoServer Deployment on every release. - ArgoCD:
argocd.argoproj.io/hook: PreSync. - Password rotation: update the Secret, re-run the Job, restart GeoServer (the JNDI pool reads the Secret at startup — sub-project 2).
Verification¶
From an ephemeral psql pod (replace host, database and password):
kubectl -n meteo-gis run psql-check --rm -i --restart=Never --image=postgres:17-alpine -- \
sh -c 'PGPASSWORD=<geoserver> psql -h <host> -U geoserver -d <database> \
-c "SHOW search_path" -c "SELECT postgis_full_version()" \
-c "CREATE TABLE t_check (id int, g geometry(Point,4326))" -c "\dt" -c "DROP TABLE t_check"'
Expected: search_path = geoserver, postgis, public (or the actual PostGIS
schema), PostGIS version displayed without a schema prefix, table
t_check listed in geoserver.
Wrong password → connection refused (psql exit code 2):
Reader role: read OK, write refused, geoserver schema inaccessible:
Expected error case — PostGIS missing and non-superuser account: failure before any creation, exit code 3:
Idempotence: re-run the Job (kubectl delete job … && kubectl apply …) →
Completed, identical summary ("PostGIS already installed").
Cleaning up the test bench¶
kubectl delete namespace meteo-gis
GeoServer image — Tomcat, JNDI, plugins, ephemeral data dir¶
Goal: a reproducible GeoServer image, configured solely through environment variables and Secrets, with all durable configuration living in the database. Version: GeoServer 3.0.1 (Tomcat 11, JDK 21).
flowchart LR
subgraph Build["docker build (multi-stage)"]
A[plugins stage<br/>SourceForge extensions<br/>+ OSGeo Nexus community modules] --> C
B[build stage<br/>Maven: geoserver-init plugin] --> C
C[final stage<br/>docker.osgeo.org/geoserver:3.0.1<br/>+ JARs + JNDI server.xml + bootstrap]
end
subgraph Run["pod startup"]
D[bootstrap-datadir.sh<br/>env → ephemeral data dir] --> E[image startup.sh<br/>server.xml envsubst, admin]
E --> F[Tomcat 11: JNDI jdbc/geoserver · jdbc/meteo]
F --> G[GeoServer: JDBCConfig + JDBCStore<br/>load the catalog from the database]
G --> H[geoserver-init: workspace, JNDI store, layer]
end
C -. image .-> D
Decisions: ADR-0005 (image,
data dir, security/), ADR-0006
(configuration via plugin), ADR-0007
(plugin provisioning). Artifacts: poc/02-geoserver-image/.
Prerequisites¶
- Sub-project 1 deployed (database,
db-admin,geoserver-db,meteo-readerSecrets). - Docker with BuildKit; an image registry reachable from the cluster (for the
local test bench:
registry:2on127.0.0.1:5000, which containerd accepts over HTTP without any configuration). - Outbound access to SourceForge,
repo.osgeo.organd Maven Central at build time.
Image contents¶
| Component | Type | Role | Source |
|---|---|---|---|
| GeoServer 3.0.1 | official image | Tomcat 11, JDK 21, hardening, startup.sh / install-extensions.sh scripts, admin management |
docker.osgeo.org/geoserver:3.0.1 |
wps |
extension | Web Processing Service | SourceForge |
wps-jdbc |
extension | WPS execution status in the database (shared across replicas) | SourceForge |
gwc-s3 |
extension | S3 blobstore for GeoWebCache (used in sub-project 4) | SourceForge |
control-flow |
extension | Limits the number of concurrent requests | SourceForge |
gs-jdbcconfig |
community | Catalog in the database | OSGeo Nexus (org/geoserver/community) |
gs-jdbcstore |
community | Resources (styles, security, GWC…) in the database | OSGeo Nexus |
geoserver-init |
in-house | Idempotent catalog initialization | geoserver-init/ (Maven) |
postgresql-client |
package | psql for first-boot detection |
apt |
The extension list is a build argument (STABLE_EXTENSIONS,
COMMUNITY_JARS): sub-projects 3 (cluster) and 4 (S3 rasters) extend it
without touching anything else.
Environment variables¶
| Variable | Source | Role |
|---|---|---|
DB_HOST, DB_PORT, DB_NAME |
db-admin Secret (these 3 keys only) |
Configuration database |
GEOSERVER_DB_USER, GEOSERVER_DB_PASSWORD, GEOSERVER_DB_SCHEMA |
geoserver-db Secret |
jdbc/geoserver resource |
METEO_DB_HOST, METEO_DB_PORT, METEO_DB_NAME |
geoserver-env ConfigMap |
Server / database holding the decoded data |
READER_USER, READER_PASSWORD |
meteo-reader Secret |
jdbc/meteo resource (read-only) |
GEOSERVER_ADMIN_USER, GEOSERVER_ADMIN_PASSWORD |
geoserver-admin Secret |
GeoServer administrator account |
PROXY_BASE_URL |
ConfigMap | Public URL in the capabilities |
GEOSERVER_INIT_WORKSPACE, GEOSERVER_INIT_DEMO_LAYER, GEOSERVER_INIT_METEO_JNDI, GEOSERVER_INIT_METEO_SCHEMA |
ConfigMap | Initialization plugin |
CONTROLFLOW_* |
ConfigMap | control-flow rules |
JNDI_*_MAX_TOTAL |
ConfigMap | Pool sizes |
EXTRA_JAVA_OPTS |
image (default) | JVM: MaxRAMPercentage=70, G1, ExitOnOutOfMemoryError, headless, forceXY, UTC |
The database admin account never enters GeoServer
The Deployment reads DB_HOST, DB_PORT, DB_NAME from the db-admin
Secret key by key; DB_ADMIN_PASSWORD is never referenced.
JNDI resources¶
Two Tomcat <Resource> entries, one per (URL, account) pair — fragment
config/jndi-resources.xml:
jdbc/geoserver:geoserveraccount, read/write, used by JDBCConfig, JDBCStore and the WPS status (jndiName=java:comp/env/jdbc/geoserverin their.properties);jdbc/meteo: reader account,?readOnly=true, used by the PostGIS datastores (jndiReferenceName).
Pool: SELECT 1 validation on borrow and while idle, 30 s eviction, 120 s
abandon, maxTotal set by variable. The PostgreSQL driver is in
$CATALINA_HOME/lib (provided by the official image) — a requirement for a
Tomcat JNDI pool to see it.
Pitfall hit while prototyping: conf/context.xml is ignored
The official image's server.xml declares the GeoServer context with
override="true" and its own jdbc/postgres resource: anything put
in conf/context.xml is ignored (symptom:
NameNotFoundException: Name [geoserver] is not bound, then the modules
falling back to localhost:5432/gsstore). The solution is an
overridden server.xml (/opt/config_overrides/server.xml),
generated by gen-server-xml.sh from the image's template by splicing in
our fragment, and installed by startup.sh with envsubst. Regenerate it
whenever the image version changes.
Data dir bootstrap (at every startup)¶
config/bootstrap-datadir.sh is the image's entrypoint; it ends with
exec bash /opt/startup.sh (official image).
| # | Action |
|---|---|
| 1 | Variable check (exit 2 with the list of missing variables) |
| 2 | Connects to the database with the geoserver account (exit 3 if unreachable); first boot? = do the object (JDBCConfig) and resources (JDBCStore) tables exist in the schema? |
| 3 | jdbcconfig/jdbcconfig.properties and jdbcstore/jdbcstore.properties: enabled=true, jndiName=java:comp/env/jdbc/geoserver, initdb/import = true only on first boot; the modules' SQL scripts copied from the image |
| 4 | jdbcstatusstore.props: dbtype=postgis, jndiReferenceName=java:comp/env/jdbc/geoserver, schema=geoserver |
| 5 | controlflow.properties from CONTROLFLOW_* |
| 6 | Exports the variables for the server.xml envsubst |
| 7 | Summary without secrets, then startup.sh (admin from GEOSERVER_ADMIN_*, SKIP_DEMO_DATA=true) |
The data dir is an emptyDir: nothing written to it is durable. On first
boot, JDBCStore imports the initial data dir (including the security/
created by the image: master key, keystore, users.xml) into the resources
table; on subsequent boots (import=false) everything is read back from the
database, including security/ — which makes the master key and the admin
account identical on all replicas.
Pitfalls hit at the prototype stage
- Do not pre-create
security/in the data dir: the image only copies the default folder there when it is absent (otherwiseusers.xmlnot found). - The message
ERROR [jdbcconfig.config] - Problem while reinitializing Logging from JDBC Configappears on subsequent boots; logging works (default levels). Observed, non-blocking, to be monitored. - On first boot,
WARN [gwc.config] Cannot read resource gwc-gs.xml: GeoWebCache applies its defaults then persists them to the database; the message no longer appears afterwards. - The WPS status table name is
status(notwpsstatus).
The geoserver-init plugin¶
A GeoServerInitializer (official extension point, called after the catalog
is loaded) which, idempotently:
- creates the
${GEOSERVER_INIT_WORKSPACE}workspace and its namespace; - creates or updates the
meteo-dbdatastore of type PostGIS (JNDI) (jndiReferenceName,schema,Expose primary keys); - publishes the demonstration table as a layer.
The log states what it did (created / exists / verified). The following
sub-projects extend this plugin (S3 stores, mosaics, styles): all
reproducible configuration goes through it, never through the UI or REST
(ADR-0006).
"GeoServer in production" checklist¶
Manual recommendation (production/) |
Where it is applied |
|---|---|
| Java: sized heap, G1, headless | EXTRA_JAVA_OPTS (MaxRAMPercentage=70); the pod's limits.memory is the single source of truth |
| Container: no demo data, unnecessary applications removed, non-deterministic shutdown | official image (SKIP_DEMO_DATA=true, i_am_a_teapot) |
| Configuration: admin password changed | geoserver-admin Secret |
| Configuration: proxy base URL | PROXY_BASE_URL |
| Configuration: control-flow | controlflow.properties |
| Data: PostGIS via JNDI, validated pool | server.xml |
| Logging to standard output | Tomcat/K8s (kubectl logs) |
| Identifiable nodes (multi-instance) | sub-project 3 |
Steps¶
cd poc/02-geoserver-image
./gen-server-xml.sh # once per image version
IMAGE=<registry>/meteo-gis/geoserver:3.0.1-poc ./build.sh --push
IMAGE=<registry>/meteo-gis/geoserver:3.0.1-poc ./deploy.sh # admin Secret, ConfigMap, Service, Deployment (1 replica)
Private registry: deploy.sh references imagePullSecrets: regcred (adapt as
needed). The local test bench uses registry:2 on 127.0.0.1:5000.
Verification¶
Startup log (bootstrap, JDBC modules, plugin, control-flow):
Tables and views created by the modules in the geoserver schema (object,
resources, status…):
Services: layer present in the capabilities, GetMap, GetFeature, WPS (196
processes), asynchronous execution accepted, admin account from the Secret,
default one rejected, demo data absent:
Administration UI: JNDI datastore and layer created by the plugin, preview:
Resilience: pod deleted → new pod with a pristine emptyDir, import=false,
everything read back from the database, admin unchanged:
Deviations observed at the prototype stage¶
conf/context.xmlignored (override="true") → overriddenserver.xml.security/must not be pre-created by the bootstrap.- WPS status table:
status. - The PostgreSQL driver is already in the official image's
$CATALINA_HOME/lib. python3is absent from the image: theserver.xmlgeneration happens on the host side (gen-server-xml.sh), which in any case keeps the artifact reviewable.
Clustering — variable replica count without fixed addresses¶
Goal: N GeoServer replicas (≥ 3, any number: kubectl scale, k9s, Rancher…)
that share the same configuration in the database, discover each other
without fixed IP addresses, propagate their configuration changes, keep the
administration UI usable and serve asynchronous WPS from any node.
flowchart TB
subgraph K8s["Namespace"]
ING[Ingress<br/>GSROUTE cookie affinity] --> SVC[geoserver Service<br/>ClusterIP :8080]
SVC --> P1[GeoServer pod 1]
SVC --> P2[GeoServer pod 2]
SVC --> P3[GeoServer pod N]
HZ[(Headless Service geoserver-hz<br/>clusterIP: None<br/>publishNotReadyAddresses: true)]
P1 <-. "Hazelcast :5701<br/>DNS discovery" .-> P2
P2 <-. "Hazelcast :5701" .-> P3
P1 <-. "Hazelcast :5701" .-> P3
P1 & P2 & P3 --- HZ
PVC[(RWX PVC geoserver-wps)]
P1 & P2 & P3 --- PVC
PDB[PodDisruptionBudget minAvailable: 1]
end
DB[(Shared TimescaleDB<br/>geoserver schema)]
P1 & P2 & P3 --> DB
Decisions: ADR-0008.
Artifacts: poc/03-clustering/ (manifests, deploy.sh, tests/) plus
evolutions of poc/02-geoserver-image/ (bundle, bootstrap, plugin).
Prerequisites¶
- Sub-projects 1 and 2 (database, image, Secrets,
geoserver-envConfigMap). - An Ingress controller (ingress-nginx on the test bench; Traefik documented).
- RWX storage for asynchronous WPS results (Longhorn on the test bench). Without RWX: disable result storage, or accept that a result is only readable from the node that produced it.
How it works¶
Propagation: the gs-hz-cluster module (Hazelcast)¶
- Each pod is a member of a Hazelcast cluster (port 5701). Any change to the
catalog or the configuration on one node is published to the others, which
refresh their in-memory copy (
sync_method=event). - Module prerequisites: JDBCConfig + JDBCStore (in place since sub-project 2).
- The bundle (module + Hazelcast 5.3.8 + dependencies) is resolved by
Maven from the release POM on the OSGeo Nexus and filtered against the
JARs already present in the image (
bundles/hz-cluster/pom.xml,ARG WITH_HZ_CLUSTER) — the method from ADR-0007.
Discovery through Kubernetes DNS (no IPs, no RBAC)¶
The headless Service geoserver-hz (clusterIP: None) publishes one DNS
entry per pod; Hazelcast queries it (<kubernetes><service-dns>). Two
non-negotiable points, both hit at the prototype stage:
publishNotReadyAddresses: true: pods that are still starting (not yetReady) must be visible, otherwise on a cold boot each pod only sees itself and forms its own cluster.- Check before Tomcat: the bootstrap resolves
HZ_SERVICE_DNS; if it does not resolve, the container stops with an explicit message (exit code 4) within a few seconds — no startup stuck in a loop.
The configuration (cluster/cluster.properties, cluster/hazelcast.xml) is
regenerated at every startup from the HZ_* variables; the cluster
directory is added to JDBCStore's ignoreDirs, otherwise the module reads an
(empty) configuration from the database and falls back to its defaults.
| Variable | Default | Role |
|---|---|---|
HZ_ENABLED |
true |
enables the module |
HZ_DISCOVERY |
kubernetes |
kubernetes (DNS) or tcp-ip (loopback, Docker tests) |
HZ_SERVICE_DNS |
— | geoserver-hz.<namespace>.svc.cluster.local |
HZ_CLUSTER_NAME |
geoserver |
cluster name |
HZ_SYNC_METHOD / HZ_SYNC_DELAY |
event / 2 |
propagation |
HZ_SESSION_SHARING / HZ_SESSION_STICKY |
false / true |
Wicket session sharing (see below) |
WPS_STORAGE_DIR |
— | shared WPS directory (declared by the plugin) |
Administration UI sessions¶
Without any mechanism, behind a round-robin Service, the UI logs out on every request (measured: 0/10 authenticated requests). Two validated solutions:
| Solution | Result | When |
|---|---|---|
Session affinity at the Ingress (GSROUTE cookie, ingress-nginx; Traefik: traefik.ingress.kubernetes.io/service.sticky.cookie) |
10/10 | Default: no code, sessions not replicated, standard |
HZ_SESSION_SHARING=true (Wicket sessions replicated by Hazelcast, HzSessionShareFilter filter injected into web.xml by the bootstrap) |
10/10 under true round-robin | Without a sticky-capable Ingress, or to survive the loss of the node holding the session |
Shared WPS¶
Execution status in the database (wps-jdbc, sub-project 2) + results on the
RWX PVC mounted by all pods (/opt/geoserver_wps, declared in the WPS
configuration by the geoserver-init plugin): an execution submitted on one
node can be tracked and read from any other one.
Node identification¶
GEOSERVER_NODE_OPTS=id:${HOSTNAME} (production guide "make cluster nodes
identifiable"): the pod name appears in the UI header.
Steps¶
cd poc/02-geoserver-image && IMAGE=<registry>/meteo-gis/geoserver:3.0.1-poc3 ./build.sh --push
cd ../03-clustering && IMAGE=<registry>/meteo-gis/geoserver:3.0.1-poc3 ./deploy.sh
kubectl -n meteo-gis scale deployment/geoserver --replicas=5 # or k9s, Rancher…
deploy.sh applies: headless Service, WPS PVC, PodDisruptionBudget, Ingress,
geoserver-cluster-env ConfigMap, Deployment with 3 replicas (preferred
anti-affinity, port 5701, WPS volume, terminationGracePeriodSeconds: 60).
Verification (tests in poc/03-clustering/tests/)¶
Cluster formation at deployment time (DNS discovery, size 1 → 2 → 3):
01 Propagation: workspace created through REST on pod A, visible on B within
3 s, deleted on B, absent on A:
02 Scale 3 → 5 → 2 → 3 with continuous GetMap requests: 0 errors out of
95:
03 Cold boot 0 → 3 simultaneously: 3 pods ready in ~55 s, a single cluster:
04 Discovery failure (wrong DNS): explicit stop within 3 s:
05 Sessions: Ingress affinity 10/10; round-robin without sharing 0/10; with
HZ_SESSION_SHARING=true 10/10:
06 Cross-node WPS: submitted on A, ProcessSucceeded and the result
(939 bytes) read on B, files on the shared volume:
07 Admin password rotation: effective on the other node, old password
rejected:
Node identifier in the UI:
Deviations and pitfalls observed¶
-
Very first deployment on an empty configuration database: all replicas run the first-boot initialization (schema creation + import) concurrently; one wins, the others can exceed the startup probe and get restarted once before converging. Harmless but noisy — for the initial deployment, start with
replicas: 1and scale up once the first pod is Ready (observed on a fresh environment; every later cold boot is fast since the database is already initialized).A concurrent first boot can also leave duplicate catalog entries (e.g. the same coverage/layer created twice by two racing initializers). Symptom: the layer works through the workspace virtual service (
/geoserver/<ws>/wms) but the global endpoint (/geoserver/ows, used by Layer Preview) returnsLayerNotDefined, and GetCapabilities lists the layer twice. Cleanup:DELETE /rest/workspaces/<ws>/coveragestores/<store>?recurse=truefor the affected stores, then restart one pod — its initializer recreates them once, cleanly. Restart any remaining pod that still serves a stale reader (Failed to create readerin the WMS exception). -
cluster/must be in JDBCStore'signoreDirs(otherwisegsEventCluster/reloaddefaults apply). - Hazelcast on JDK 21: add the recommended
--add-opens/--add-modules java.se(done inEXTRA_JAVA_OPTS). HzSessionShareListenerdoes not exist in 3.0.1: only the filter needs to be declared (initialization is handled by a Spring bean).- The
Failed to publish resource notification, cluster not initialized (yet)warnings at startup are normal (before the cluster forms). - Tests:
kubectl port-forward svc/…pins one pod (it does not exercise round-robin);kubectl run --rm -isometimes loses the output — the tests go throughkubectl execin a GeoServer pod.
S3 storage — tile cache and rasters¶
Goal: move everything bulky or shared off the pods' disks — the GeoWebCache tile cache and the rasters — to S3 (or compatible) object storage, configured by code and shared by all replicas.
flowchart LR
C[WMTS / WMS client] --> P1[GeoServer pod A]
C --> P2[GeoServer pod B]
P1 -- "GWC: tile? MISS → render → PUT" --> S3T[(gwc-tiles bucket)]
P2 -- "GWC: GET → HIT" --> S3T
P1 -- "COG: range reads (Range)" --> S3R[(rasters bucket<br/>demo/demo-temperature.tif)]
P2 -- "COG" --> S3R
SEC[(s3-credentials Secret)] -.-> P1 & P2
DB[(Database: S3 blobstore, tile layers,<br/>COG store in JDBCStore/JDBCConfig)] -.-> P1 & P2
Decision: ADR-0009. Artifacts:
poc/04-s3/ plus evolutions of poc/02-geoserver-image/ (COG bundle, plugin,
bootstrap).
Prerequisites¶
- Sub-projects 1 to 3 deployed.
- An S3-compatible object store reachable from the pods, two buckets (tiles,
rasters) and one key pair. Test bench: SeaweedFS (Apache 2.0),
k8s/test-seaweedfs.yaml— not a reference deployment. s3-credentialsSecret (templatek8s/secret-s3.example.yaml):
| Key | Role |
|---|---|
S3_ENDPOINT, S3_REGION |
Endpoint (http/https) and region |
S3_ACCESS_KEY, S3_SECRET_KEY |
Credentials |
S3_PATH_STYLE |
true for most S3-compatible stores (SeaweedFS, MinIO, Ceph) |
S3_GWC_BUCKET |
Tile cache bucket |
S3_RASTER_BUCKET, COG_DEMO_KEY |
Rasters bucket and key of the demonstration COG |
The Deployment references this Secret with optional: true: without S3,
GeoServer runs with a local, ephemeral tile cache.
Tile cache: default S3 blobstore¶
gwc-s3extension (in the image since sub-project 2).- The
geoserver-initplugin creates or updates at every startup anS3BlobStoreInfo(s3-tiles):bucket,endpoint,useHTTPSderived from the URL, credentials,maxConnections,default=true— throughBlobStoreAggregator(org.geowebcache.storage), never through REST/UI (ADR-0006). With a custom endpoint, the client automatically switches to path-style. - The GWC configuration (
gwc/blobstores.xml,gwc-layers/*.xml) lives in the database via JDBCStore: it survives pod replacement and is common to all replicas. - The tile layers are created explicitly by the plugin
(
GWC.add(GeoServerTileLayer)) in aContextLoadedEventlistener — GeoWebCache is not yet initialized when theGeoServerInitializers run, and auto-creation upon layer addition did not happen at the prototype stage.
Rasters: COGs read directly from S3¶
- Community modules
gs-cog-core+gs-cog-s3(bundle resolved by Maven, ADR-0007): HTTP range requests against a Cloud Optimized GeoTIFF — no local copy, no volume. - GeoTIFF store created by the plugin: URL
cog://s3://<bucket>/<key>, metadataCogSettings.Key→CogSettingsStore(S3 RangeReader, credentials encrypted in the database). - Endpoint, region and path-style of the S3 RangeReader through
environment variables exported by the bootstrap:
IIO_S3_AWS_ENDPOINT,IIO_S3_AWS_REGION,IIO_S3_AWS_FORCE_PATH_STYLE(without it, the client triesbucket.endpoint: "Unable to execute HTTP request" — hit at the prototype stage),IIO_S3_AWS_USER/IIO_S3_AWS_PASSWORDas a fallback. - This is the mode required for the ImageMosaic on S3 of sub-project 5
(
Cog=true,CogRangeReader=…S3RangeReaderinindexer.properties).
The demonstration COG (data/make-cog.sh, GDAL in a container) is a
synthetic temperature in kelvin (north-south gradient), 256×256, EPSG:4326,
tiled 128, DEFLATE — metadata units=K, param_key=air_temperature
(ADR-0003).
Pre-check and failure modes¶
The bootstrap requires S3_ENDPOINT to answer over HTTP (any status code)
before starting Tomcat; otherwise explicit stop with exit code 5
(measured: 3 s). A missing bucket or wrong credentials show up in the
GeoServer log (blobstore or COG store as a warning, service intact).
Steps¶
cd poc/04-s3
kubectl apply -f k8s/test-seaweedfs.yaml # test bench only
./data/make-cog.sh && kubectl -n meteo-gis create configmap demo-cog --from-file=demo-temperature.tif=data/demo-temperature.tif
kubectl apply -f k8s/secret-s3.example.yaml -f k8s/job-seed-rasters.yaml # buckets + COG
IMAGE=<registry>/meteo-gis/geoserver:3.0.1-poc4 ./deploy.sh
Verification (tests in poc/04-s3/tests/)¶
Buckets and COG seeded (seed-s3 Job); startup: S3 blobstore verified, COG
store and layer, tile layers:
01 First tile MISS (rendered then written to S3), second HIT;
objects in the bucket:
02 Same tile on another pod: HIT (no rendering):
03 Pod replaced: blobstore still present and still the default, tile HIT:
04 COG layer on S3, GetMap from two pods:
05 Scale 3 → 2 → 3: all three pods serve the tile as a HIT:
06 S3 endpoint unreachable: explicit stop (exit code 5):
UI: blobstores and cached layers:
Deviations and pitfalls observed¶
BlobStoreAggregatorlives inorg.geowebcache.storage(GWC 2.0.1) and theS3BlobStoreInfosetters takeStrings.GWC.get()isnullwhile theGeoServerInitializers run → tile layers created onContextLoadedEvent; do not look upCatalog.classby type (several beans) but keep theGeoServerreference frominitialize().- A listener that throws an exception prevents the webapp from starting
("One or more listeners failed to start", details in the container's
logs/localhost.*.log, not on stdout). kubectl applyrejects a ConfigMap with more than 256 KB of annotation: usekubectl createfor the COG.- Same image tag republished:
imagePullPolicy: Always+rollout restart.
ImageMosaic ingestion — time series and multi-dimensional models¶
Goal: continuously publish weather rasters (models: model run × forecast validity time × vertical level; satellites: time series) as GeoServer ImageMosaic layers, COGs on S3 indexed in PostGIS, visible to all replicas as soon as they are ingested, with the quantity's unit known and displayed.
flowchart LR
SRC[Sources<br/>models, satellites] --> ING[Ingestion service<br/>Python + GDAL]
ING -- "COG (canonical unit)" --> S3[(rasters bucket)]
ING -- "INSERT index + one view per mosaic" --> DB[(mosaic schema<br/>parameter · model_granule · sat_granule)]
GS[GeoServer ×N] -- "jdbc/meteo (read)" --> DB
GS -- "range reads" --> S3
PLUG[geoserver-init plugin<br/>MOSAICS ConfigMap] -- "indexer/datastore.properties (Resource API)<br/>coverage + dimensions + title (unit)" --> GS
C[WMS clients] -- "TIME · ELEVATION · DIM_REFERENCE_TIME" --> GS
Decisions: ADR-0003
(accepted), ADR-0010.
Artifacts: poc/05-ingestion/ (schema, service, Jobs, tests).
Prerequisites¶
Sub-projects 1 to 4 (database, image, cluster, S3). Secrets: ingestion-db
(ingestion service account), s3-credentials, meteo-reader.
Data model (mosaic schema, owned by the ingestion service)¶
| Table | Role |
|---|---|
parameter |
One physical quantity = one row: param_key, cf_standard_name, canonical_unit, label |
parameter_code |
Known codes per scheme (grib2, eccodes, bufr, metar, cf) with the unit in that scheme |
level_type |
Level types (GRIB2 table 4.5) |
model_granule |
One row per model COG: the_geom, location (s3://…), model, param_key, reference_time (model run), time (validity), elevation, level_type, level_label, source_unit, ingested_at |
sat_granule |
One row per satellite image: satellite, instrument, product, param_key, time, source_unit |
views arpege_t, meteosat12_fci_ir105… |
One view per mosaic (filter on model+quantity or satellite+instrument+product); gt_pk_metadata declares fid as the key for GeoTools |
The schema is created by a Job (admin account), GeoServer's reader receives
SELECT through grant-reader (sub-project 1); datastore.properties
specifies schema=mosaic.
Ingestion service¶
ghcr.io/osgeo/gdal container + Python (boto3, psycopg, numpy),
CLI ingest.py model | sat. For each granule:
- field (synthetic at the prototype stage — actual GRIB/NetCDF decoding is out of scope);
- conversion to the canonical unit of the reference table (demonstrated:
source
Cel→K),source_unitpreserved; - writing of a COG (GDAL, DEFLATE, 256 tiles) with metadata
param_key,units,time,reference_time, level; - S3
put_objectfollowing the ADR-0003 convention (models/arpege/t/arpege_t_isobaric85000_20260901T00Z_20260901T06Z.tif,sat/meteosat-12/fci/meteosat-12_fci_ir105_20260901T1200Z.tif); INSERT … ON CONFLICT DO UPDATEinto the index andCREATE OR REPLACE VIEWof the mosaic.
Run as a Job (k8s/job-ingest-demo.yaml, ingest.sh); CronJob or
orchestrator in production. A quantity absent from the reference table =
explicit error ("add it to mosaic.parameter").
Real-data example: GOES-19 from NOAA Open Data¶
fetch_goes.py (+ k8s/job-ingest-goes.yaml) ingests real satellite
imagery through the exact same conventions: GOES-19 ABI channel 13
(infrared 10.3 µm brightness temperature, kelvin) from the public NOAA
bucket https://noaa-goes19.s3.amazonaws.com — anonymous HTTPS, one
full-disk granule every 10 minutes, reproducible by anyone. Each granule is
converted from the geostationary NetCDF to an EPSG:4326 COG (GDAL Warp),
uploaded as sat/goes-19/abi/goes-19_abi_ir103_<time>.tif and indexed with
its real acquisition time and footprint. Publication is then the usual
one-line MOSAICS entry (goes19_abi_ir103|…) — run the ingestion Job
before declaring the mosaic. Two pitfalls: the ubuntu-small GDAL image
has no netCDF driver (the ingestion image uses ubuntu-full), and the
"Unhandled X/Y axis unit rad" warning is harmless (the warped bounds are
correct).
Publication by the plugin (declarative)¶
geoserver-mosaics ConfigMap: MOSAICS="name|view|kind|param_key|title;…".
For each entry, at each pod's startup:
mosaics/<name>/indexer.properties(Cog=true,CogRangeReader=…S3RangeReader,UseExistingSchema=true,CanBeEmpty=true,TypeName=<view>,TimeAttribute=time,ElevationAttribute=elevation,AdditionalDomainAttributes=reference_timefor models) anddatastore.properties(PostgisNGJNDIDataStoreFactory,jndiReferenceName=java:comp/env/jdbc/meteo,schema=mosaic) — written through theResourceAPI, hence stored in the database by JDBCStore and shared;ImageMosaicstore (file:mosaics/<name>), coverage and layer;- dimensions enabled:
time(LIST, default MAXIMUM),elevation(Pa), custom dimensionreference_time; - title = label + canonical unit read from
mosaic.parameterthrough the JNDI resource, abstract and keywordsunit:K,param:air_temperature.
The S3 credentials of the RangeReader come from the environment
(IIO_S3_AWS_*, sub-project 4): no secret in the mosaic files.
No notification needed
ImageMosaic queries the database index on every request, dimension
domains included: a granule inserted by the ingestion is visible in the
GetCapabilities and served by all pods immediately, without any
restart or message (test 03). File-based mosaics would have required a
per-node harvest — that is the reason behind the "index in the
database" choice.
Verification (tests in poc/05-ingestion/tests/)¶
Schema and reference table, demonstration ingestion (6 model granules, 3 satellite ones):
01 Capabilities: titles carrying the unit, dimensions and their values:
02 GetMap by TIME × ELEVATION: different images (md5), defaults when no
dimension is given:
03 New forecast validity time ingested → 4 time values on both pods,
GetMap served, pods unchanged:
04 Satellite mosaic per instant:
05 Pod replaced: mosaics still published:
Deviations and pitfalls observed¶
- The reader role now serves several schemas:
grant-reader.sqlno longer redefines an existingsearch_path; the stores specifyschema=. - Views as the index: virtual key
fiddeclared ingt_pk_metadata. - The mosaic configuration files go through the
ResourceAPI (JDBCStore): shared between pods, regenerated at every startup.
Vector data — decoded surface observations (TimescaleDB / PostGIS)¶
Goal: publish continuously decoded surface observations (SYNOP, METAR…) as WMS layers with a TIME dimension and as WFS features, readable by every replica the moment they are inserted, with the canonical unit carried through (ADR-0003).
flowchart LR
SRC[SYNOP · METAR · …] --> DEC[Decoding service<br/>simulator in this prototype]
DEC -- "INSERT (canonical unit,<br/>source_unit kept)" --> DB[(schema obs<br/>station · observation hypertable<br/>publication views)]
GS[GeoServer ×N] -- "jdbc/meteo (read-only,<br/>datastore obs-db, schema=obs)" --> DB
PLUG[geoserver-init plugin<br/>VECTOR_LAYERS ConfigMap] -- "feature types + TIME dimension<br/>title/unit from reference data" --> GS
C[WMS TIME · WFS · CQL] --> GS
Decision: ADR-0011.
Artifacts: poc/06-vector/ (schema, Jobs, tests) and the geoserver-init
plugin (ensureVectorLayers).
Prerequisites¶
Sub-projects 1 to 5 (database, image, cluster, S3, reference data in
mosaic.parameter). Secrets: decoder-db (decoding-service account),
meteo-reader.
Data model (schema obs, owned by the decoding service)¶
| Object | Role |
|---|---|
station |
One row per station: identifier (ICAO/WMO), name, geom Point 4326, elevation |
observation |
TimescaleDB hypertable, PK (station_id, time, param_key) + surrogate fid ; value in the canonical unit, source_unit kept (METAR temperatures arrive in Celsius), message_type |
obs_air_temperature (view) |
Full series joined with stations — backs the TIME-enabled layer |
obs_latest_air_temperature (view) |
Latest value per station (DISTINCT ON … ORDER BY time DESC) — backs the map-friendly layer |
obs.gt_pk_metadata (view) |
Exposes the virtual key fid of the views to GeoTools |
The schema is created by a Job run with the operator account
(k8s/job-init-obs-schema.yaml); GeoServer's reader gets SELECT through the
grant-reader Job (DATA_SCHEMA=obs). The two init Jobs may race: the grant
Job simply retries until the schema exists (backoff).
Decoder simulator¶
ingest.py obs --from 2026-09-02T06:00Z --hours 3 --step-min 10 (same
container as the raster ingestion) upserts 10 demo stations across Europe and
Asia and generates one observation per station and per 10-minute step:
synthetic 2 m temperature in Celsius, converted to kelvin on insert
(source_unit=Cel), alternating METAR/SYNOP message types. Real
SYNOP/BUFR/TAC decoding is out of scope of the prototype. Run it as a Job
(k8s/job-ingest-obs.yaml, via ../05-ingestion/ingest.sh); a CronJob or the
real decoding service takes over in production.
Publication by the plugin (declarative)¶
VECTOR_LAYERS="name|view|timeAttribute(- for none)|param_key|title;…" in the
geoserver-mosaics ConfigMap. On each pod start the plugin ensures:
- a second datastore
obs-dbon the same JNDI resourcejdbc/meteowithschema=obs(one datastore per schema; the stores always name a JNDI resource, never a URL); - one feature type + layer per view, with
TIMEenabled on the given attribute (continuous interval presentation — listing every observation instant would be meaningless), default = most recent; - title and abstract carrying the canonical unit read from
mosaic.parameter(Surface observations — air temperature (K)).
No GWC tile layer is created for observation layers: requests are time-qualified and served straight by WMS.
Verification (tests poc/06-vector/tests/)¶
Schema, grant (retry visible), 180 observations ingested:
Startup: datastore and layers created/verified:
01 Capabilities: titles with the unit, continuous TIME dimension
(start/end/PT1S, default = latest):
02 The TIME dimension really filters: an in-range instant renders the
stations; an out-of-range instant is rejected with an explicit
ServiceException (fail loud — consistent with the project rule); WFS
counts per instant: 10, 10, 0. Two valid instants render identical images by
design (the default point style does not encode the value — thematic styling
comes with sub-project 8):
03 One more hour ingested → TIME extent and feature counts grow on both
pods, no restart, no notification (the views read the live tables):
04 The reader stays read-only (permission denied on INSERT) and the layers
survive a pod replacement:
Deviations and pitfalls observed¶
- A WMS request outside the TIME domain returns a
ServiceException(Could not find a match for 'time' value) rather than an empty map — document it to client teams. - Identical images for two valid instants are expected with the default style; the real proof of filtering is WFS/CQL counts (test 02).
gt_pk_metadatamust be visible inside the datastore's schema: theobsschema exposes a filtered view over the shared table.
Basemaps — hosted data and cascaded external services¶
Goal: give every map a background, with a deliberate choice between two patterns — reference data downloaded once and hosted locally (recommended for production), and an external WMS cascaded through GeoServer (quick to set up, but bound by third-party licence and availability). Both are cached in the shared S3 tile cache.
flowchart LR
NE[Natural Earth archive<br/>public domain] -- "ogr2ogr Job (once)" --> DB[(schema basemap<br/>countries)]
GS[GeoServer ×N] -- "jdbc/meteo · basemap-db" --> DB
EXT[External WMS provider] -. "cascade (WMSStore)" .-> GS
GS -- "GWC tiles" --> S3[(bucket gwc-tiles)]
PLUG[geoserver-init plugin<br/>VECTOR_LAYERS · BASEMAP_WMS_* · styles as code] --> GS
Decision: ADR-0012.
Artifacts: poc/07-basemaps/.
Hosted basemap (recommended)¶
- Source: Natural Earth 1:110m country boundaries — public domain, no
licence constraints. The load Job reads the published archive directly
(
ogr2ogr /vsizip//vsicurl/…) intobasemap.countries(schema owned by a dedicatedbasemaprole). On the target platform the archive would come from the internal artifact repository (ADR-0007), not from the internet. - Publication: one more entry in
VECTOR_LAYERSusing theschema.viewsyntax (countries|basemap.countries|-|-|Country boundaries (Natural Earth)|basemap-countries) — the plugin maintains one JNDI datastore per schema (basemap-db) and assigns the code-managed SLD stylebasemap-countries(the SLD body ships with the plugin and is written through the Resource API, so it is stored in the database and shared by all replicas). - Caching:
EXTRA_TILE_LAYERS=countries,osm— tiles go to the default S3 blobstore like any other layer.
Cascaded external WMS (optional)¶
BASEMAP_WMS_URL (capabilities URL), BASEMAP_WMS_LAYER (remote name) and
BASEMAP_WMS_NAME (local name) drive the plugin: it creates a WMSStore and
a cascaded layer. Empty URL = disabled. Caveats (ADR-0012): third-party terms
of use, availability and latency; the S3 tile cache softens the load on the
provider but the dependency remains.
Verification (tests poc/07-basemaps/tests/)¶
Schema init, Natural Earth load (177 features), reader grant:
Startup: per-schema datastore, layers, style and cascade verified:
01 Hosted countries layer: WFS 177 features, GetMap with the code-managed
default style:
02 Cascaded layer served through GeoServer:
03 Basemap tiles in the shared S3 cache: MISS → HIT on the same pod, HIT
from another pod (both hosted and cascaded layers):
04 Layers, style and cascade survive a pod replacement; tiles still HIT:
Deviations and pitfalls observed¶
ogr2ogrin the GDAL "ubuntu-small" image includes the PostgreSQL driver;-lco FID=fidgives the table a real primary key, so nogt_pk_metadataentry is needed for plain tables.- The cascaded layer's remote name (
OSM-WMS) comes from the provider's capabilities; the local name (osm) is ours.
Processing — vector → raster interpolation and in-house WPS¶
Goal: turn point observations into continuous fields. Two very different situations, two answers:
- dense networks (SYNOP, METAR): an in-house IDW process, exposed both as a WPS process and as an SLD rendering transformation (colored surface computed on the fly from the observation layer);
- sparse, badly distributed networks (TEMP, AMDAR — a handful of stations, one vertical level at a time): a comparative study on a known synthetic field, concluded by ADR-0013.
Artifacts: poc/08-processing/ (study, tests) and
poc/02-geoserver-image/meteo-wps/ (process module).
In-house process module (meteo-wps)¶
meteo:IDWInterpolation— Inverse Distance Weighting of a point attribute onto a grid (data,valueAttr,power, plus the rendering-transformation contractoutputBBOX/outputWidth/outputHeight).- Registered through the standard GeoTools SPI
(
META-INF/services/org.geotools.process.ProcessFactory→ a factory extendingAnnotatedBeanProcessFactoryunder themeteonamespace). - The style
obs-idw-temperature(managed as code, like every style) invokes the process as a<Transformation>and paints the result with a kelvin color ramp; it is attached as an additional style ofobs_latest_air_temperature(EXTRA_LAYER_STYLES).
GetMap …&layers=meteo:obs_latest_air_temperature&styles=obs-idw-temperature
Sparse-network study (TEMP/AMDAR case)¶
Protocol (study/compare.py, runnable in a plain Python container): a known
synthetic truth field is sampled at 5 stations clustered in one corner;
four reconstructions are scored against the truth (RMSE, same color scale):
| Method | RMSE (K) |
|---|---|
| IDW p=2 | 4.40 |
| Barnes 2-pass | 4.93 |
| RBF thin-plate | 6.77 |
| model first guess + corrections | 1.36 |
| (raw first guess, for reference) | 2.37 |
Readings: pure spatial methods hallucinate everywhere the stations are not — and RBF, often praised for sparse data, extrapolates worst with clustered stations. Starting from an (imperfect) model field and correcting it with the observed innovations beats everything by a wide margin — and even improves on the raw model. Hence ADR-0013.
Verification (tests poc/08-processing/tests/)¶
01 The process is described and listed by WPS:
02 The rendering transformation produces the colored surface on two pods
(and the point rendering stays available as the default style):
03 Process, style and attachment survive a pod replacement:
Deviations and pitfalls observed (rendering transformations)¶
- Process factories are discovered through the GeoTools SPI, not through
Spring beans: ship a
META-INF/services/org.geotools.process.ProcessFactoryentry (a bean alone is silently ignored). AnnotatedBeanProcessFactorynames processes after the class name minus "Process" (IDWInterpolationProcess→meteo:IDWInterpolation).- Implement
invertQuery— without it the renderer pushes an unbounded envelope down to PostGIS (POLYGON((-Infinity …parse error). - In
invertQuery, request all properties (setPropertyNames(null)): the renderer only fetches attributes referenced by symbolizers, and the process's value attribute is a literal it knows nothing about — the symptom is an empty collection while the plain layer renders fine. - IDW is global: the data query deliberately keeps all features (documented performance caveat for very large networks).
Decisions
Architecture decisions¶
Every structuring decision is recorded in an ADR (Architecture Decision Record), based on the template.
| ADR | Title | Status |
|---|---|---|
| 0001 | Documentation tooling and structure | accepted |
| 0002 | GeoServer workspace organization | proposed |
| 0003 | Parameter identification and granule naming | accepted |
| 0004 | Database initialization by a standalone Job and one JNDI resource per source | accepted |
| 0005 | Official image, plugins at build time, ephemeral data dir, security/ in the database |
accepted |
| 0006 | Reproducible configuration via a Java plugin, not via REST | accepted |
| 0007 | Provisioning of plugins and artifacts | proposed |
| 0008 | Hazelcast clustering via Kubernetes DNS discovery; UI sessions | accepted |
| 0009 | S3 object storage for the tile cache and the rasters (COG) | accepted |
| 0010 | Ingestion: COG on S3, PostGIS index, views, no notification | accepted |
| 0011 | Observations: TimescaleDB hypertable, one view per layer, canonical values | accepted |
| 0012 | Basemaps: hosted reference data first, cascaded WMS as a complement | accepted |
| 0013 | Sparse upper-air interpolation: model first guess + observed corrections | accepted |
ADR-0001 — Documentation tooling and structure¶
Date: 2026-09-01 · Status: accepted
Context¶
The project starts with no mandated documentation tool. We need complete documentation, with diagrams and screenshots, that can be handed over to colleagues and exported to PDF.
Decision¶
MkDocs + Material theme, Markdown content, structured according to
arc42, complemented by step-by-step guides and ADRs. Diagrams as
inline Mermaid. PDF export via the print-site plugin.
Rejected alternatives¶
| Alternative | Why rejected |
|---|---|
| AsciiDoc + Antora + PlantUML | More powerful but heavier to install and maintain; Markdown is more widely shared |
| Astro Starlight | Depends on Node.js; oriented towards websites rather than architecture documentation |
| Corporate wiki (Confluence, etc.) | None is mandated today; the Markdown will migrate easily if a tool is chosen later |
| draw.io diagrams / images | Binary or manual export, not diffable; Mermaid is versioned text |
Consequences¶
- Documentation versioned with the code, reviewed by diff.
- A single tool to install (
pip install -r requirements.txt). - Mermaid diagrams are limited to the supported types (flowchart, sequence, class, state, ER, gantt) — sufficient for arc42.
- Migration to another tool remains possible without rewriting the content.
ADR-0002 — GeoServer workspace organization¶
Date: 2026-09-01 · Status: proposed (to be decided with sub-projects 6 and 7)
Context¶
The layers span three domains (meteorology, aeronautics, hydrology) and four
kinds of data (basemaps and static data, satellite imagery, multi-dimensional
weather models, time series in the database). In GeoServer, a workspace is at
once a namespace, a virtual service (/geoserver/<workspace>/wms), a
security boundary (per-workspace access rules) and a grouping of stores.
The chosen split therefore shapes URLs, permissions and operations.
Update 2026-09-02 — decision basis¶
The decision must be made against the real target inventory (GTS network and dedicated feeds): SYNOP, METAR, TAF, AIRMET/SIGMET/GAMET (area products with validity periods), satellite imagery, radar, lightning, NWP models and hydrology-dedicated layers — explicitly not against the layout of any small prior project. The candidate workspace sets below are to be re-examined with that inventory and the future rights model.
Options¶
| Option | Workspaces | Pros | Cons |
|---|---|---|---|
| A — by data nature | static, satellites, models, meteo (TimescaleDB time series) |
Homogeneous stores per workspace (same type, same dimensions, same ingestion pipeline); simple operations and automation | A business client (e.g. aeronautics) consumes several workspaces; per-domain permissions are harder |
| B — by business domain | meteorology, aeronautic, hydrology |
URLs and permissions aligned with the consumers; one virtual service per business domain | Heterogeneous stores in the same workspace; a single source (e.g. a weather model) useful to several domains would have to be duplicated or referenced |
| C — hybrid | Workspaces by nature (A) + layer groups or per-domain naming prefixes; or the reverse | Combines both axes | Two conventions to maintain |
Decision¶
Not taken. To be settled after the ingestion (sub-project 5), vector data (6) and basemap (7) prototypes, when the actual number of stores and the permission requirements are known.
Consequences (expected)¶
- Prototypes 1 to 5 use provisional workspace names and must not hard-code anything against them.
- The decision will be recorded as an update to this ADR (status "accepted").
ADR-0003 — Parameter identification and granule naming¶
Date: 2026-09-01 · Status: accepted (validated by prototype 5; amendments: one view per mosaic instead of a table, source_unit per granule, no replica notification — see ADR-0010)
Context¶
The platform serves physical quantities (temperature, pressure, wind, humidity, wave height…) that arrive through several vectors, each with its own code system and its own units:
| Vector | Format | Code registry | Temperature, for example |
|---|---|---|---|
| Weather models (grids) | GRIB2 | WMO GRIB2 code table 4.2 — discipline‑category‑number triplet; ECMWF Parameter DB / ecCodes — paramId + shortName |
0-0-0 · 130 / t · K |
| SYNOP surface observations | BUFR (FM 94) | WMO BUFR Table B — F‑X‑Y descriptor | 0 12 101 · K |
| METAR / SPECI / TAF aeronautical observations and forecasts | Traditional alphanumeric code (FM 15, FM 51, WMO‑306 Manual on Codes) and IWXXM (XML, the ICAO Annex 3 standard) | Code groups; IWXXM elements; in BUFR, the same descriptors as SYNOP | M03/M05 group · integer °C |
| WAFS aeronautical grids (wind, temperature, icing, turbulence, CB) | GRIB2 | WMO table 4.2 codes — ICAO defines no table of its own | same as models |
| Satellite imagery | GeoTIFF / NetCDF | WMO OSCAR/Space (satellite, instrument); WMO common tables C‑5 / C‑8 | radiance, brightness temperature (K), reflectance (%) |
| All | — | CF standard names — name + canonical unit; WIGOS/WMDR observed variables | air_temperature · K |
Two observations:
- No single domain registry (GRIB2, BUFR, METAR) covers all vectors. Only the CF standard names vocabulary is common to grids and observations.
- The same quantity arrives in different units depending on the vector (K in GRIB2 and BUFR, °C in METAR, hPa for QNH, knots for aeronautical wind). The unit is part of the data.
In addition, the names of raster files and GeoServer layers must remain readable for operations (debugging on S3, layer listings), without being the source of truth.
Decision¶
1. One physical quantity = one registry entry, several codes¶
- Reference table
parameter: one row per physical quantity, readable key (param_key, e.g.air_temperature),cf_standard_name, canonical unit (UCUM notation:K,Pa,m/s,%), label. - Table
parameter_code: one row per known code of that quantity, with the scheme (grib2,eccodes,bufr,metar,iwxxm,cf), the code and the unit in that scheme. Adding a vector = adding rows, no migration. - Ingestion resolves the source code →
parameter, converts to the canonical unit and keeps the source unit alongside the value.
2. Vertical dimension coded with WMO table 4.5¶
- Table
level_type: GRIB2 table 4.5 code (1surface,100isobaric surface in Pa,103height above ground in m,101mean sea level…), name, unit. - Aeronautical flight levels (FL) have no native 4.5 code: they are stored
as an isobaric surface (standard atmosphere) with a label
FL300kept in a dedicated column. - Two different level types never share the same
ELEVATIONdimension: they are separate layers.
3. One GeoServer layer = one quantity, one unit, one style¶
- Models: one ImageMosaic per (model, quantity) pair, filtered by a SQL view
over the granule table; exposed dimensions
TIME= forecast validity time,DIM_REFERENCE_TIME= model run,ELEVATION= level. - Satellites: one ImageMosaic per (satellite, instrument, product) triple;
TIMEdimension. - The layer's unit is published in its metadata and its legend (exact mechanism to be validated in prototype 5).
4. File naming (redundant with the table, never the source of truth)¶
| Nature | Pattern | Example |
|---|---|---|
| Satellite | {satellite}_{instrument}_{product}_{YYYYMMDDThhmmZ}.tif |
meteosat-12_fci_ir105_20260901T1200Z.tif |
| Model | {model}_{param}_{levelType}{level}_{runYYYYMMDDThhZ}_{validYYYYMMDDThhZ}.tif |
arpege_t_isobaric85000_20260901T00Z_20260901T12Z.tif |
param = ecCodes shortName (readable and unambiguous for grids);
satellite/instrument identifiers follow OSCAR/Space in lowercase.
5. Observations in a time-series database¶
SYNOP / METAR / SPECI / TAF messages are decoded into an observation
hypertable (station, time, quantity, canonical value, source unit, message
type) published through SQL views. Details in sub-project 6.
Rejected alternatives¶
| Alternative | Why rejected |
|---|---|
| Pivot = GRIB2 triplet alone | Does not cover SYNOP/METAR observations or satellite products |
Pivot = ECMWF paramId alone |
Specific to ecCodes; no mapping for BUFR/METAR; numbers are hard to read |
| A single ImageMosaic per model with the quantity as a custom dimension | One layer would carry several units and several styles; ambiguous legend and GetFeatureInfo |
| Unit inferred from the file name or the style | The unit must be data, not an implicit convention |
| Keeping the source unit without conversion | Clients would need to know the originating vector to interpret the value |
Consequences¶
- The
parameter/parameter_code/level_typeregistry is a deliverable of sub-projects 1 (schema) and 5 (initial load from the WMO / ECMWF / CF registers). - The ingestion service (sub-project 5) owns the unit conversion; it rejects a file or a message whose code cannot be resolved.
- The decision will be confirmed or amended after prototypes 5 (mosaics) and 6 (observations), in particular regarding unit publication on the GeoServer side.
ADR-0004 — Database initialization by a standalone Job and one JNDI resource per data source¶
Date: 2026-09-01 · Status: accepted
Context¶
The TimescaleDB database is an existing, shared service operated by another team. GeoServer must have an isolated configuration schema there (JDBC plugins) and read data schemas owned by other services (message decoding, ingestion), potentially on other URLs and with other accounts. Credentials must come from Secrets, and the GeoServer pods must never hold an administrator account.
Decision¶
- Initialization by a standalone, replayable Kubernetes Job (
psql, idempotent SQL script), executed before GeoServer (Helm hookpre-install,pre-upgrade/ ArgoCDPreSync), sole holder of the admin Secret provided by the operator. Preconditions (PostGIS) are checked before any creation: no partial state. - One JNDI resource per (URL, account) pair:
jdbc/geoserver(configuration schema, read/write) and one resource per data schema (jdbc/meteo, …) with a read-only role created bygrant-reader.sql. GeoServer stores reference a JNDI name, never a URL. search_pathset on the role; PostGIS detected, installed only if absent and if the account is superuser, otherwise explicit failure.
Rejected alternatives¶
| Alternative | Why rejected |
|---|---|
| initContainer in the GeoServer Deployment | Admin Secret present in every replica; N concurrent executions |
| Migration tool (Flyway, Liquibase) | Oversized for one role and one schema; to reconsider for the data schemas |
| A single account for configuration and data | No read-only access possible; coupling with the producer services |
Systematic PostGIS installation in public |
Pollutes a shared database; a decision that belongs to the operator |
search_path in the JDBC URL |
Duplicated in every datasource; if forgotten = tables created in public |
Consequences¶
- Data schemas are created and owned by their services; we provide the script and the reader-role convention.
- Rotating a password = update the Secret + rerun the Job + restart GeoServer.
- Sub-project 2 declares the JNDI resources in Tomcat from the same Secrets
(keys
GEOSERVER_DB_*,READER_*).
ADR-0005 — Official image, plugins at build time, ephemeral data dir, security/ in the database¶
Date: 2026-09-01 · Status: accepted (validated by prototype 2)
Context¶
In a cluster, a GeoServer data dir shared or persisted per pod creates divergence between the disk and the database (prior operational experience: styles, GWC, master key generated per pod → undecipherable store passwords). The image must be reproducible and download nothing at startup.
Decision¶
- Base = official image
docker.osgeo.org/geoserver:<version>; stable extensions and community modules downloaded at build time (the image'sinstall-extensions.sh+ OSGeo Nexus), in-house plugin compiled in a Maven stage. Nothing at runtime. - Data dir =
emptyDirregenerated at every startup bybootstrap-datadir.shfrom environment variables; JDBCConfig and JDBCStore via JNDI;initdb/importinferred from the presence of the tables. security/is carried in the database by JDBCStore (it is not inignoreDirs): master key, keystore, users and roles are imported at first boot then re-read by all replicas. Verified: after pod replacement, the admin account from the Secret remains valid and the default password remains rejected.- JNDI resources in an overridden
server.xmlgenerated from the image's template (the image's<Context override="true">ignoresconf/context.xml).
Rejected alternatives¶
| Alternative | Why rejected |
|---|---|
| Shared PVC (RWX) for the data dir | Disk/database divergence experienced; dependency on RWX storage |
Bare tomcat + war |
Reinvents startup, probes, hardening, admin management |
| Kartoza image | Opinionated, lags behind 3.0 |
INSTALL_EXTENSIONS=true at runtime |
Downloads at every boot, non-reproducible, network-dependent |
security/ seeded from a Secret at every boot |
Unnecessary since JDBCStore carries it; would remain an option if JDBCStore were dropped |
Consequences¶
- Any configuration change goes through the database (plugin, ADR-0006) or through the image; never through a pod's disk.
- Admin password rotation:
GEOSERVER_ADMIN_PASSWORDis only applied at the first boot (import); after that, the change is made in GeoServer (UI or security API) — recorded in the risks. - The overridden
server.xmlmust be regenerated at every image version upgrade (gen-server-xml.sh).
ADR-0006 — Reproducible configuration via a Java plugin (GeoServerInitializer), not via REST¶
Date: 2026-09-01 · Status: accepted
Context¶
Prior operational experience: on a GeoServer with a database-backed catalog and several replicas, changes made through the REST API (or in the UI) turned out to be non-persisted or non-propagated (in-memory configuration of a single replica, GWC files not written, objects recreated at restart). Post-startup scripts (Jobs calling REST) are fragile: ordering, races, dependency on service availability.
Decision¶
All reproducible configuration (workspaces, stores, layers, styles, GWC
settings, services) is created by a Java plugin embedded in the image,
implementing GeoServerInitializer (called after the catalog is loaded),
idempotent (looks up before creating, updates if different), driven by
environment variables, and logging what it does. Datastores reference
JNDI names, never URLs.
REST and the UI remain reserved for one-off manual operations (diagnostics, cache purge, tests) — never as a source of truth.
Rejected alternatives¶
| Alternative | Why rejected |
|---|---|
| REST Jobs after deployment | Non-persistence and races experienced; depends on the service being up |
| Direct SQL into the JDBCConfig tables | Internal format (serialized XML), unsupported, silently ignored |
| Versioned and copied data dir | Contradicts "the database is the single source of truth"; one-time import |
Consequences¶
- Adding a layer = code + image build (or, for continuously produced data, an ingestion service using the catalog API — sub-project 5).
- The plugin is the natural home for the configuration of sub-projects 3 to 8.
- GeoServer developers must know the
Catalog/CatalogBuilderAPI.
ADR-0007 — Provisioning of plugins and artifacts (community modules, in-house plugins, images)¶
Date: 2026-09-01 · Status: proposed
Context¶
GeoServer community modules are not shipped on the download page (only
nightly builds). Verified on 2026-09-01: the OSGeo Nexus
(https://repo.osgeo.org/repository/release/org/geoserver/community/)
nevertheless publishes release JARs aligned with each tagged version
(gs-jdbcconfig, gs-jdbcstore, gs-cog*, gs-s3-geotiff, gs-hz-cluster,
gs-jms-* in 3.0.1). Stable extensions are on SourceForge. Our in-house
plugins are compiled by us. Long-term build reproducibility cannot depend on
the availability of public repositories.
Point of attention: a GeoServer plugin is generally a set of JARs
(the module plus its third-party dependencies). The OSGeo Nexus only
publishes the module's JAR; the dependencies are in the nightly zip, whose
version (3.0-SNAPSHOT) does not match the release (3.0.1). For
gs-jdbcconfig / gs-jdbcstore the JAR alone is enough (dependencies
already in GeoServer); for the cluster modules (Hazelcast, ActiveMQ) or S3
(AWS SDK) the full bundle is required.
Decision (proposed)¶
- Corporate artifact repository (Nexus or Artifactory) with: a proxy
of
repo.osgeo.org(release) and of Maven Central, a hosted repository for our plugins (org.example.meteogis:*) and, as a fallback, the community JARs fetched manually or built from source. The Dockerfile points at this repository (NEXUSargument). - Corporate image registry for the built images; base images and extensions pinned (exact version, ideally digest).
- Release-aligned community bundles resolved by Maven: starting from
the module's release POM on the OSGeo Nexus,
mvn dependency:copy-dependencies(runtime scope) excluding the artifacts already present in the image'sWEB-INF/lib, then a zip published in the hosted repository. This is the method retained for the cluster modules (sub-project 3) and S3 rasters (sub-project 4). - Building from source only for modules absent from the OSGeo Nexus or
requiring a patch: clone the GeoServer repository at the tag,
mvn -pl src/community/<module> -am install, publish into the hosted repository. - No download at pod startup (already established, ADR-0005).
Rejected alternatives¶
| Alternative | Why rejected |
|---|---|
| Depending directly on public repositories in production | Unavailability or withdrawal of artifacts = build impossible |
| Committing the JARs into the code repository | Bulky, no version management or provenance |
| Building all modules from source | Costly and unnecessary as long as the OSGeo Nexus publishes the releases |
Consequences¶
- To be worked out with the platform team: which tool already exists (Nexus, Artifactory, GitLab Package Registry…), retention policy.
- The
Dockerfileremains unchanged: only the base URLs change.
ADR-0008 — Hazelcast clustering via Kubernetes DNS discovery; UI sessions via Ingress affinity¶
Date: 2026-09-01 · Status: accepted (validated by prototype 3)
Context¶
N GeoServer replicas with a variable count, no fixed IP address, configuration in the database (JDBCConfig/JDBCStore). Configuration changes must be propagated between nodes, the UI must remain usable behind a round-robin Service, and asynchronous WPS must be served from any node. Prior operational experience: Hazelcast discovery through the Kubernetes API without RBAC blocked startups for tens of minutes.
Decision¶
- Propagation:
gs-hz-clustermodule (Hazelcast 5.3.8, version pinned by GeoServer 3.0.1),sync_method=event, bundle resolved by Maven (ADR-0007). - Kubernetes discovery in DNS mode: headless Service with
publishNotReadyAddresses: true,service-dns, fixed port 5701; no RBAC, no IPs. The bootstrap checks DNS resolution before Tomcat and fails explicitly otherwise (measured: 3 s). - UI sessions: session affinity at the Ingress by default (cookie);
HZ_SESSION_SHARING=truevalidated as an alternative when affinity is not available. - WPS: status in the database + results on a shared RWX PVC.
PodDisruptionBudget minAvailable: 1, preferred anti-affinity, node identification (GEOSERVER_NODE_OPTS).
Measurements (test bench)¶
Propagation 3 s; scale 3→5→2→3: 0 errors / 95 requests; cold boot 0→3: ~55 s, a single cluster; discovery failure: shutdown in 3 s; sessions: affinity 10/10, bare round-robin 0/10, Hazelcast sharing 10/10; WPS and admin rotation across nodes OK.
Rejected alternatives¶
| Alternative | Why rejected |
|---|---|
JMS (gs-jms-*) + ActiveMQ |
A broker to operate; primary/replica model (replicas reject modifications) |
| Discovery through the Kubernetes API | RBAC required; blocking experienced on failure |
| Multicast / tcp-ip | Unavailable in Kubernetes / fixed IPs |
| Tomcat session replication (JDBC, Redis) | Not needed: two solutions validated without any extra component |
| No propagation | Cache divergence between replicas |
Consequences¶
- The replica count is free; every new pod joins the cluster via DNS. An HPA remains possible (to be worked out with metrics).
- RWX storage becomes a platform requirement for asynchronous WPS.
- Wicket sessions are not replicated by default: losing the hosting node logs the user out (acceptable for an administration UI).
ADR-0009 — S3 object storage for the tile cache and the rasters (COG)¶
Date: 2026-09-01 · Status: accepted (validated by prototype 4)
Context¶
In a cluster with ephemeral replicas, the tile cache and the rasters cannot live on the pods' disks; a shared RWX volume is possible but is neither object storage nor cloud-native. The sub-project 5 mosaics will have to read granules on S3.
Decision¶
- GeoWebCache tile cache on S3 (
gwc-s3extension), default blobstore, created by the initialization plugin from thes3-credentialsSecret; tile layers created by code. - Rasters as COG on S3, read by ranges via the
cog-core+cog-s3modules (S3 RangeReader, path-style and endpoint via theIIO_S3_AWS_*variables). Any raster deposited into the platform is a Cloud Optimized GeoTIFF. - S3-compatible storage provided by the platform; everything parameterized (endpoint, region, path-style, buckets, keys); SeaweedFS test bench (Apache 2.0).
- Endpoint pre-check at bootstrap (exit code 5); without an S3 Secret, fallback to a local ephemeral cache.
Measurements¶
MISS → HIT on one pod, direct HIT on another pod, HIT after pod replacement and after scaling; identical COG GetMap on two pods; wrong endpoint: shutdown in 3 s.
Rejected alternatives¶
| Alternative | Why rejected |
|---|---|
| RWX PVC for the cache and the rasters | Not object storage, depends on network storage, no CDN possible |
s3-geotiff (community) |
No ImageMosaic support; COG is the standard |
| MinIO for the test bench | AGPL license |
| Blobstore created via REST/UI | Non-persistence experienced (ADR-0006) |
Consequences¶
- The ingestion service (sub-project 5) produces COGs and deposits them on S3.
- Tile retention/expiration policy and seeding remain to be defined (out of scope).
- The raster's unit and parameter travel in the COG metadata (ADR-0003).
ADR-0010 — Ingestion: COG on S3, shared PostGIS index, one view per mosaic, no notification¶
Date: 2026-09-01 · Status: accepted (validated by prototype 5)
Context¶
Continuous, multi-dimensional weather rasters, to be published on N replicas without manual action. Prior operational experience: shapefile index (truncated hours) and file-based mosaics with per-node harvest = fragile.
Decision¶
- Granule index in PostGIS (
mosaicschema, owned by the ingestion service), one view per mosaic;UseExistingSchema=true. - Granules = COG on S3, named according to ADR-0003, read by ranges.
- No notification: ImageMosaic reads the index at every request; an inserted granule is immediately visible everywhere.
- Declarative publication by the plugin (
MOSAICSConfigMap), configuration files through theResourceAPI, datastore via JNDI, title = label + canonical unit from the registry. - Units: ingestion converts to the canonical unit and keeps
source_unit; the unit is exposed in the title, the abstract and the keywords.
Rejected alternatives¶
| Alternative | Why rejected |
|---|---|
| File-based mosaics + REST harvest per node | Non-atomic, to be repeated on every replica, non-persistence experienced |
| Event queue to trigger refreshes | Unnecessary with a database index |
| One table per mosaic | Duplicates the model; the view is enough |
S3 credentials in indexer.properties (CogUser) |
Plain-text secret in a file; the environment is enough |
Consequences¶
- Adding a mosaic = one row in
MOSAICS+ one view created by the ingestion. - Granule retention (S3 deletion + index) remains to be defined (out of scope).
- Decoding of the real formats (GRIB2, NetCDF, HRIT) will replace the prototype's synthetic generation.
ADR-0011 — Observations: TimescaleDB hypertable, one view per layer, canonical values¶
Date: 2026-09-02 · Status: accepted (validated by prototype 6)
Context¶
Decoded surface observations arrive continuously from a decoding service and must be served time-qualified (WMS TIME, WFS/CQL) by every replica without restarts, with units under control (ADR-0003). Prior operational experience: raw tables exposed directly couple the publication to the producer's model.
Decision¶
- Schema
obsowned by the decoding service (every producer owns its schema):station,observationhypertable (PK station/time/param, surrogatefid), values stored in the canonical unit withsource_unitkept (conversion at decode time). - One SQL view per published layer (series view and latest-per-station
view), virtual key declared in
gt_pk_metadata; GeoServer reads through the shared read-only JNDI datasource (obs-dbdatastore,schema=obs). - Declarative publication by the init plugin (
VECTOR_LAYERS): TIME dimension (continuous interval, default latest) on the given attribute, title/abstract with the canonical unit from the reference data. - No notification mechanism: views read the live tables; new observations are visible to all replicas immediately (measured).
Rejected alternatives¶
| Alternative | Why rejected |
|---|---|
| Exposing the raw table | Couples publication to the producer's model; no per-quantity filtering |
| LIST presentation for TIME | Unbounded list of instants in the capabilities |
| GWC tile layers for observations | Requests are time-qualified; caching would need a TIME parameter filter for little gain |
| Converting units at render time | The stored value must be reliable for WFS/WPS consumers |
Consequences¶
- The real decoding service replaces the simulator without touching GeoServer.
- Retention on the hypertable (drop_chunks policy) is to be defined with the data owners (recorded in the risks).
- Thematic styles (value-dependent symbology) come with sub-project 8.
ADR-0012 — Basemaps: locally hosted reference data first, cascaded external WMS as a complement¶
Date: 2026-09-02 · Status: accepted (validated by prototype 7)
Context¶
Maps need a background layer. It can be produced from downloaded reference data hosted in the platform, or proxied from an external provider (cascade). The platform must control licences, availability and load put on third parties.
Decision¶
- Hosted reference data is the default: public-domain datasets (Natural
Earth for the prototype) loaded into the
basemapschema by anogr2ogrJob, published with code-managed SLD styles, cached on S3. Source archives come from the internal artifact repository in production (ADR-0007). - Cascaded external WMS is an optional complement (
BASEMAP_WMS_*), served and tile-cached by GeoServer — subject to the provider's terms of use and availability; disabled by emptying the URL. - Both patterns are configured by the init plugin — nothing manual.
Rejected alternatives¶
| Alternative | Why rejected |
|---|---|
| External tiles fetched directly by clients | No caching control, mixed origins, licence checks pushed to every client |
| Cascade as the only basemap | Availability and licence dependency for a core visual function |
| Raster basemap generated offline (pre-rendered tiles) | Heavier pipeline; can be added later without changing the architecture |
Consequences¶
- Basemap updates are data loads (Job), not configuration changes.
- The cascaded provider can be swapped or disabled by ConfigMap.
- Vector-tile (MVT) or relief basemaps can be added later under the same declarative mechanism.
ADR-0013 — Sparse upper-air interpolation: model first guess + observed corrections¶
Date: 2026-09-02 · Status: accepted (backed by the prototype-8 study)
Context¶
Upper-air observations (TEMP soundings, AMDAR) are few and badly distributed; maps are displayed one vertical level at a time. Pure spatial interpolation must invent values over huge empty areas. The platform already ingests the corresponding model fields (sub-project 5).
Decision¶
For sparse networks, the analysis of choice is model first guess + corrections (successive-correction / optimal-interpolation family): take the model field at the requested level and validity, compute the innovations (observation − first guess at the stations), spread them spatially (IDW or Barnes) and add them to the field. Measured on a known truth with 5 clustered stations: RMSE 1.36 K versus 4.40 (IDW), 4.93 (Barnes), 6.77 (RBF) — and 2.37 for the raw model field.
For dense networks (SYNOP, METAR), plain IDW is adequate and is provided
as the in-house meteo:IDWInterpolation process / rendering transformation.
Rejected alternatives¶
| Alternative | Why rejected |
|---|---|
| RBF thin-plate as the sparse default | Worst extrapolation of the panel with clustered stations (RMSE 6.77 K) despite its reputation on sparse data |
| Barnes with a wide radius | Honest but flat far from data (4.93 K); no use of known physics |
| Ordinary kriging | A variogram cannot be estimated from a handful of stations |
| Observations-only display (no interpolation) | Remains available (point styles), but does not answer the "field" need |
Consequences¶
- Production implementation = a process combining a coverage (first guess from
the mosaics) and the observation features — natural follow-up of the
meteo-wpsmodule; recorded in the building blocks as future work. - The study script and metrics are versioned (
poc/08-processing/study/) and rerunnable to re-evaluate with other station layouts.
ADR-NNNN — Decision title¶
Date: YYYY-MM-DD · Status: proposed | accepted | superseded by ADR-NNNN
Context¶
What problem or constraint forces a decision?
Decision¶
What is retained, in one or two sentences.
Rejected alternatives¶
| Alternative | Why rejected |
|---|---|
| … | … |
Consequences¶
What the decision entails (positive and negative), what it enables or forbids.








































































