Skip to content

S3 storage — tile cache and rasters

Goal: move everything bulky or shared off the pods' disks — the GeoWebCache tile cache and the rasters — to S3 (or compatible) object storage, configured by code and shared by all replicas.

flowchart LR
    C[WMTS / WMS client] --> P1[GeoServer pod A]
    C --> P2[GeoServer pod B]
    P1 -- "GWC: tile? MISS → render → PUT" --> S3T[(gwc-tiles bucket)]
    P2 -- "GWC: GET → HIT" --> S3T
    P1 -- "COG: range reads (Range)" --> S3R[(rasters bucket<br/>demo/demo-temperature.tif)]
    P2 -- "COG" --> S3R
    SEC[(s3-credentials Secret)] -.-> P1 & P2
    DB[(Database: S3 blobstore, tile layers,<br/>COG store in JDBCStore/JDBCConfig)] -.-> P1 & P2
Hold "Alt" / "Option" to enable pan & zoom

Decision: ADR-0009. Artifacts: poc/04-s3/ plus evolutions of poc/02-geoserver-image/ (COG bundle, plugin, bootstrap).

Prerequisites

  • Sub-projects 1 to 3 deployed.
  • An S3-compatible object store reachable from the pods, two buckets (tiles, rasters) and one key pair. Test bench: SeaweedFS (Apache 2.0), k8s/test-seaweedfs.yaml — not a reference deployment.
  • s3-credentials Secret (template k8s/secret-s3.example.yaml):
Key Role
S3_ENDPOINT, S3_REGION Endpoint (http/https) and region
S3_ACCESS_KEY, S3_SECRET_KEY Credentials
S3_PATH_STYLE true for most S3-compatible stores (SeaweedFS, MinIO, Ceph)
S3_GWC_BUCKET Tile cache bucket
S3_RASTER_BUCKET, COG_DEMO_KEY Rasters bucket and key of the demonstration COG

The Deployment references this Secret with optional: true: without S3, GeoServer runs with a local, ephemeral tile cache.

Tile cache: default S3 blobstore

  • gwc-s3 extension (in the image since sub-project 2).
  • The geoserver-init plugin creates or updates at every startup an S3BlobStoreInfo (s3-tiles): bucket, endpoint, useHTTPS derived from the URL, credentials, maxConnections, default=true — through BlobStoreAggregator (org.geowebcache.storage), never through REST/UI (ADR-0006). With a custom endpoint, the client automatically switches to path-style.
  • The GWC configuration (gwc/blobstores.xml, gwc-layers/*.xml) lives in the database via JDBCStore: it survives pod replacement and is common to all replicas.
  • The tile layers are created explicitly by the plugin (GWC.add(GeoServerTileLayer)) in a ContextLoadedEvent listener — GeoWebCache is not yet initialized when the GeoServerInitializers run, and auto-creation upon layer addition did not happen at the prototype stage.

Rasters: COGs read directly from S3

  • Community modules gs-cog-core + gs-cog-s3 (bundle resolved by Maven, ADR-0007): HTTP range requests against a Cloud Optimized GeoTIFF — no local copy, no volume.
  • GeoTIFF store created by the plugin: URL cog://s3://<bucket>/<key>, metadata CogSettings.Key → CogSettingsStore (S3 RangeReader, credentials encrypted in the database).
  • Endpoint, region and path-style of the S3 RangeReader through environment variables exported by the bootstrap: IIO_S3_AWS_ENDPOINT, IIO_S3_AWS_REGION, IIO_S3_AWS_FORCE_PATH_STYLE (without it, the client tries bucket.endpoint: "Unable to execute HTTP request" — hit at the prototype stage), IIO_S3_AWS_USER / IIO_S3_AWS_PASSWORD as a fallback.
  • This is the mode required for the ImageMosaic on S3 of sub-project 5 (Cog=true, CogRangeReader=…S3RangeReader in indexer.properties).

The demonstration COG (data/make-cog.sh, GDAL in a container) is a synthetic temperature in kelvin (north-south gradient), 256×256, EPSG:4326, tiled 128, DEFLATE — metadata units=K, param_key=air_temperature (ADR-0003).

Pre-check and failure modes

The bootstrap requires S3_ENDPOINT to answer over HTTP (any status code) before starting Tomcat; otherwise explicit stop with exit code 5 (measured: 3 s). A missing bucket or wrong credentials show up in the GeoServer log (blobstore or COG store as a warning, service intact).

Steps

cd poc/04-s3
kubectl apply -f k8s/test-seaweedfs.yaml            # test bench only
./data/make-cog.sh && kubectl -n meteo-gis create configmap demo-cog --from-file=demo-temperature.tif=data/demo-temperature.tif
kubectl apply -f k8s/secret-s3.example.yaml -f k8s/job-seed-rasters.yaml   # buckets + COG
IMAGE=<registry>/meteo-gis/geoserver:3.0.1-poc4 ./deploy.sh

Verification (tests in poc/04-s3/tests/)

Buckets and COG seeded (seed-s3 Job); startup: S3 blobstore verified, COG store and layer, tile layers:

S3 seed

Startup with S3

01 First tile MISS (rendered then written to S3), second HIT; objects in the bucket:

MISS then HIT

02 Same tile on another pod: HIT (no rendering):

Cross-pod HIT

03 Pod replaced: blobstore still present and still the default, tile HIT:

Persistence after replacement

04 COG layer on S3, GetMap from two pods:

COG GetMap

Rendering of the demonstration COG

05 Scale 3 → 2 → 3: all three pods serve the tile as a HIT:

Scale and cache

06 S3 endpoint unreachable: explicit stop (exit code 5):

S3 outage

UI: blobstores and cached layers:

Blobstores

Cached layers

Deviations and pitfalls observed

  • BlobStoreAggregator lives in org.geowebcache.storage (GWC 2.0.1) and the S3BlobStoreInfo setters take Strings.
  • GWC.get() is null while the GeoServerInitializers run → tile layers created on ContextLoadedEvent; do not look up Catalog.class by type (several beans) but keep the GeoServer reference from initialize().
  • A listener that throws an exception prevents the webapp from starting ("One or more listeners failed to start", details in the container's logs/localhost.*.log, not on stdout).
  • kubectl apply rejects a ConfigMap with more than 256 KB of annotation: use kubectl create for the COG.
  • Same image tag republished: imagePullPolicy: Always + rollout restart.