Skip to content
Configuration

Configuration

Puddle is configured with a single YAML file passed via -c:

puddle -c /path/to/puddle.yaml

puddle recover -c /path/to/puddle.yaml reads the same file to rebuild a lost metastore from the warehouses’ catalog sidecars (see Recovering without a metastore backup).

A config file is required — puddle won’t boot without one, and it must declare at least one warehouse. ${VAR} references in string values are expanded from the process environment; any unset variable is a hard error at boot (see Environment variable expansion).

This document covers every top-level block. A minimal config that boots is just:

puddle.yaml
warehouses:
  default:
    location: file:///tmp/puddle-warehouse

Top-level blocks:

  • server — listen address.
  • logging — log level and format.
  • telemetry — OpenTelemetry and the management listener (off by default).
  • metastore — where catalog metadata is stored.
  • warehouses — one or more named warehouses.
  • external-warehouses — warehouses managed by other metastores, available for registerTable.
  • authn — bearer-token validators (off by default).
  • authz — policy backend (allow-authenticated, OPA, CEL).

server

puddle.yaml
server:
  addr: ":8089"               # default
  read-header-timeout: 10s    # default
  read-timeout: 60s           # default
  write-timeout: 60s          # default
  idle-timeout: 120s          # default
  shutdown-timeout: 25s       # default
  purge-timeout: 10m          # default
  max-request-body: 8MiB      # default

The Iceberg REST API is mounted at /api/catalog/, so clients point their catalog uri at http://<host>/api/catalog.

HTTP timeouts

The four *-timeout knobs are forwarded to Go’s http.Server and bound how long the process will wait on a misbehaving (or merely slow) client:

  • read-header-timeout — reading request headers.
  • read-timeout — reading the full request, headers + body. Caps slow-body attacks where a client drips body bytes indefinitely.
  • write-timeout — writing the response. It also sets the deadline of every catalog request (see below); long-running operations (commit retry, STS round-trip during credential vending) live under this budget, so tune it up if your vending path is slow.
  • idle-timeout — bound how long an idle keep-alive connection stays open.

A value of 0s is accepted literally as “no limit” — useful for debugging, dangerous in production. Negative values are rejected at config-load.

Request deadline

Go’s write timeout doesn’t stop a handler; it only makes its late answer fail. So each catalog request also gets a deadline of write-timeout minus a margin (10%, at most 5s: 55s with the defaults), leaving time to send an error. Metastore calls, storage I/O and commit retries all stop at it:

  • An operation that runs out of time answers 503 ServiceUnavailableException with the request id; the cause is logged under the same id. Java clients retry a 503 on idempotent requests.
  • A commit that hits the deadline while writing the table pointer may have landed. Puddle re-reads the pointer; if it can’t confirm the commit, it keeps the new metadata file and answers 500 CommitStateUnknownException, which tells the client not to clean up either.
  • Commit retries after a concurrent update (up to 4, backing off from 100ms, as in Java’s commit.retry.* defaults) stop before the deadline, answering the last 409 CommitFailedException. Java’s 30-minute total retry budget doesn’t apply.

With write-timeout: 0s there is no request deadline.

Request body size

max-request-body caps the size of incoming request bodies; the limit is installed as a http.MaxBytesReader wrap before route dispatch, so every JSON-bodied endpoint inherits it without per-handler wiring. Over-limit requests get a 400 BadRequestException with a request body exceeds maximum size of N bytes message.

The default of 8 MiB is comfortably above any realistic metadata.json payload — typical commits are tens of KB, and the largest pathological cases observed in the wild are still well under a megabyte. Bump it for unusual workloads; the trade-off is that a higher cap allows larger memory spikes per request.

Accepts a bare integer (bytes) or a value with a unit suffix: B, KB/KiB, MB/MiB, GB/GiB. Matching is case-insensitive. Zero, negative, and unknown-unit values are rejected at config-load — the field exists to impose a limit, so disabling it via config is misconfiguration.

Purge

Dropping a table with purgeRequested=true drops its pointer, then deletes its files (only those under the table’s location). Once the pointer is gone, the files are unreachable from the catalog, so the deletion is not tied to the request: a client disconnect or the request deadline doesn’t stop it halfway. Puddle answers when the deletion is done, as Java’s catalog does, or when the request deadline arrives, whichever comes first; in the second case the purge continues in the background. Either way the drop succeeded and the answer is 204.

purge-timeout bounds each purge’s deletion (0s means no limit). A purge that runs out of time stops, and the files not yet deleted stay in place. Each purge logs one line when it ends (purge: finished, or purge: stopped before finishing with the reason), with the table, its metadata location, the number of files deleted, and the request id of the drop. Timeouts and purge covers what a purge deletes and cleaning up after one that stopped.

Graceful shutdown

On SIGTERM, SIGINT, or SIGHUP, puddle stops accepting new connections and waits up to shutdown-timeout for in-flight requests, and then for purges still running in the background, to finish before exiting. If the deadline elapses with requests or purges still in flight, puddle exits non-zero so the platform sees the unclean shutdown. Each abandoned purge is logged (purge: abandoned at shutdown) with its table, metadata location and the number of files deleted so far; the rest of its files stay behind.

SIGHUP is treated as “shut down cleanly” rather than “reload config” — config changes are restart events, not in-process refreshes.

Tune shutdown-timeout to fit comfortably inside your platform’s termination grace window. On Kubernetes, that’s terminationGracePeriodSeconds (default 30s) — the 25s default above leaves a few seconds of headroom before the kubelet sends SIGKILL. If you raise the K8s grace period, raise this in step.

Probes

Two endpoints sit on the top-level mux for orchestrators:

  • GET /healthz — liveness. Returns 200 once the process is past startup. The contract is “this process is responsive enough to write a 200”; nothing more. Any failure here is a reason to restart the pod.
  • GET /readyz — readiness. Returns 200 with {"status":"ready"} when the metastore pings successfully; returns 503 with {"status":"not ready"} otherwise. The reason (which can name the database host and user) is not in the response: it is logged at error level as readiness check failed, with a request_id. The metastore ping has a 2 s context deadline so a wedged backend never stalls the probe.

Both are unauthenticated by design — kubelets don’t carry credentials, and the responses leak only “process up” / “DB reachable”, no PII or table contents. Lock the surface down via NetworkPolicy if your environment requires it. With telemetry enabled, the management listener serves the same two probes, so the kubelet can probe that port instead of the catalog’s.

A minimal Kubernetes probe block:

livenessProbe:
  httpGet: { path: /healthz, port: http }
  periodSeconds: 10
readinessProbe:
  httpGet: { path: /readyz, port: http }
  periodSeconds: 5
  failureThreshold: 3

/readyz only checks the metastore, not the warehouse storage — S3 latency / throttling is a write-path concern that is better surfaced on the actual catalog operation than as flapping pod readiness.

logging

puddle.yaml
logging:
  level: info     # debug | info | warn | error
  format: text    # text | json

Both fields default to the values shown above. Logs go to stderr.

Request ids and errors

Every response carries an X-Request-Id header. An inbound X-Request-Id (printable ASCII, at most 128 bytes) is kept, so a proxy’s or client’s id threads through; otherwise puddle generates a UUIDv7. Every log line about a request carries it as request_id.

Error responses use the Iceberg error envelope, and its error object carries the same request_id:

{"error": {"message": "Internal server error (request id: 0199…)",
           "type": "ServiceFailureException", "code": 500,
           "request_id": "0199…"}}
  • 4xx errors are the client’s own: the message says what was wrong with the request (the wording Iceberg clients expect).
  • 5xx errors never describe the failure to the client: storage, database, STS, and policy-engine errors can name hosts, buckets, roles, and users. The message only names the request id; the full error is logged at error level as request failed, with the same request_id plus op, method, path, warehouse, and principal (when authenticated). To investigate a client’s 500, search the logs for the id it reports. A CommitStateUnknownException keeps Iceberg’s guidance text (check whether the commit landed before retrying) after the generic message. A request that ran out of time is a 503 ServiceUnavailableException, Request deadline exceeded (request id: …) (see Request deadline).
  • A stored metadata file puddle cannot parse is reported as a 500, not as the client’s mistake. Registering a table or view from a file that is not valid metadata is a 400.
  • A request body that is not valid JSON, or has a field of the wrong JSON type, is a 400 BadRequestException whose message names the problem (Malformed JSON at offset 17: …, Invalid JSON: expected string for "location", got number). The JSON decoder’s own error is logged at info level as bad request with the request_id.
  • A request that matches no route gets a 400 BadRequestException, No route for request: <METHOD> <path>, as Iceberg’s own REST server answers. <path> is the path as sent, such as /api/catalog/v1/nope.

telemetry

puddle.yaml
telemetry:
  enabled: false   # true turns on OpenTelemetry and the management listener
  addr: ":9090"    # the management listener

Off by default: no metrics, no traces, no second listener. With enabled: true, puddle starts the OpenTelemetry SDK and a management listener on addr (a host:port, not server.addr) that serves:

  • GET /metrics — Prometheus metrics: requests, authentication and authorization, commits, the metastore, vending, and the Go runtime and process, with the resource (service.name, service.version, the commit) on target_info; the metric catalog lists them. Unauthenticated by design: keep the port off the network the catalog is exposed on.
  • GET /healthz and GET /readyz — the same probes as the catalog listener.

It uses the server block’s timeouts and stops after the catalog listener has drained at shutdown, within shutdown-timeout.

Everything else comes from the standard OpenTelemetry environment variables:

  • Traces are exported over OTLP only when asked for: OTEL_EXPORTER_OTLP_ENDPOINT (or OTEL_EXPORTER_OTLP_TRACES_ENDPOINT) set, or OTEL_TRACES_EXPORTER=otlp; OTEL_TRACES_EXPORTER=none turns them off. OTEL_EXPORTER_OTLP_PROTOCOL is http/protobuf (the default) or grpc; headers, TLS and timeouts use the exporter’s usual OTEL_EXPORTER_OTLP_* variables.
  • Sampling is parent-based: a request whose traceparent says sampled is recorded, and 10% of the others (parentbased_traceidratio). OTEL_TRACES_SAMPLER and OTEL_TRACES_SAMPLER_ARG change it.
  • OTEL_SERVICE_NAME and OTEL_RESOURCE_ATTRIBUTES add to or override the resource.
  • OTEL_SDK_DISABLED=true turns the SDK off; the management listener still answers the probes, with an empty /metrics.

A malformed value (an unknown sampler or exporter, an endpoint that isn’t an http:// or https:// URL) fails at boot. Puddle never waits for a collector: exports that fail at runtime are logged as opentelemetry error, and spans still queued at shutdown are dropped once shutdown-timeout runs out.

Log lines written in the context of a trace carry its trace_id and span_id, next to request_id.

Observability has the rest: every variable puddle honours, sampling advice, the metrics, the trace structure, Kubernetes snippets and suggested alerts.

metastore

Where puddle keeps the catalog’s own state (which tables exist, what their current metadata locations are). Table data and the metadata files themselves live in the warehouse, not here.

puddle.yaml
metastore:
  type: memory    # memory | sqlite | postgresql

type selects the backend; the matching sub-block (if any) carries its settings.

memory

Non-persistent. State is lost on shutdown. Useful for tests, demos, and the RCK harness.

puddle.yaml
metastore:
  type: memory

sqlite

File-backed. Suitable for single-node deployments.

puddle.yaml
metastore:
  type: sqlite
  sqlite:
    path: ./tmp/puddle.db   # required; ":memory:" for ephemeral

path is required. Use :memory: for an ephemeral in-process DB (tests).

postgresql

Shared Postgres. Suitable for multi-replica deployments where several puddle instances point at one database.

puddle.yaml
metastore:
  type: postgresql
  postgresql:
    url: "postgres://puddle:${PUDDLE_PG_PASSWORD}@db:5432/puddle?sslmode=require"
    schema: puddle              # default: "puddle"
    max-open-conns: 10          # default: 10
    max-idle-conns: 5           # default: 5
    conn-max-lifetime: 1h       # default: 1h

url must use the postgres:// (or postgresql://) URL form. Use ${VAR} env expansion for the password to keep secrets out of YAML.

schema is auto-created on first start. Must match [a-zA-Z_][a-zA-Z0-9_]*.

warehouses

warehouses: is a map from warehouse name to its configuration. At least one entry is required. Clients address a specific warehouse by name (e.g. via the warehouse=<name> query parameter).

Names must match [a-zA-Z0-9_-]+. config, oauth, and tokens are reserved and rejected at boot.

puddle.yaml
warehouses:
  default:
    location: file:///tmp/puddle-warehouse
  prod:
    location: s3://my-warehouse
    s3: { ... }      # see below

location is a URL whose scheme picks the storage backend.

Supported schemes:

  • file:// — local filesystem. No sub-block. Suitable for dev, CI, and single-node deployments where data lives on local disk.

    puddle.yaml
    warehouses:
  default:
    location: file:///tmp/puddle-warehouse

  • s3:// (also s3a://, s3n://) — S3 or any S3-compatible store: AWS S3, RustFS, Ceph RGW, Cloudflare R2, Tigris. Requires an s3: sub-block.

    puddle.yaml
    warehouses:
  prod:
    location: s3://my-warehouse
    s3:
      region: us-east-1                       # required
      endpoint: https://s3.amazonaws.com      # omit for AWS, set for RustFS/Ceph
      path-style-access: false                # true for RustFS/Ceph

      # Catalog's own S3 credentials. Leave empty to defer to the
      # AWS default credential chain (env vars, profile, IRSA,
      # instance metadata).
      access-key-id: ${AWS_ACCESS_KEY_ID}
      secret-access-key: ${AWS_SECRET_ACCESS_KEY}
      session-token: ""

      # Optional toggles, all default off / empty.
      use-arn-region: false
      checksum-enabled: false
      acl: ""                                 # e.g. bucket-owner-full-control
      write-storage-class: ""                 # STANDARD | STANDARD_IA | ...

    region, endpoint, and path-style-access are forwarded to clients automatically — PyIceberg / Spark / Trino pick them up without further configuration.

Vended credentials

When enabled, puddle mints short-lived, prefix-scoped session tokens via STS AssumeRole and hands them to clients on every table load. The catalog’s own credentials never leave the server. A caller the authorizer allows table.commit on the table gets a read-write token (engines write data files with the token from the table load before they commit); anyone else gets a read-only one (see Credential vending). Default policy templates scope every token to the table’s own prefix, so a token for db.events cannot read or write db.orders. Clients refresh tokens before they expire through the table’s credentials endpoint, which vends what a table load would (see Refreshing credentials).

puddle.yaml
    s3:
      ...
      vended-credentials:
        enabled: true
        role-arn: arn:aws:iam::123456789012:role/IcebergTableAccess
        duration-seconds: 3600              # default; AWS allows [900, 43200]
        sts-endpoint: ""                    # defaults to s3.endpoint
        sts-region: ""                      # defaults to s3.region
        external-id: ""                     # cross-account
        read-policy-template: ""            # text/template for read-only tokens
        write-policy-template: ""           # text/template; default scopes to table prefix

Vending requires explicit access-key-id + secret-access-key on the warehouse (the catalog uses them to call STS). Works the same against AWS, RustFS, and Ceph (RGW); only role-arn and the optional sts-endpoint differ. See Credential vending for the per-target operator runbook (trust policies, role creation, template overrides, troubleshooting).

Locations

Every table and view gets its own directory, named after its UUID:

  • A table or view created without a location lives at <warehouse location>/<uuid>, where <uuid> is its Iceberg table (or view) UUID, so analytics.events might live at s3://my-warehouse/4f1c3e0a-9d2b-4c55-8a51-0e6f3b7d9a12. Names are not part of the path: renaming a table, or moving it to another namespace, doesn’t make its directory misleading.
  • Namespaces have no directory. A namespace gets a location property only if a client sets one (CREATE SCHEMA ... WITH (location = ...), CREATE NAMESPACE ... LOCATION ...), for example to give a team its own prefix for S3 lifecycle rules, IAM policies or cost reports. Tables and views created in that namespace without a location then default to <namespace location>/<uuid>. The location applies to that namespace only, not to namespaces nested in it. A nested namespace can only be created inside an existing parent (creating analytics.daily before analytics fails with NoSuchNamespaceException), and a namespace with children can’t be dropped.

A client may also choose a location itself. Since puddle authorizes by name but vends credentials for, and deletes files under, locations, it only accepts locations that no other table can claim:

  • A table’s or view’s location must be strictly under the warehouse location (not the warehouse root itself), and not at or inside <warehouse location>/_puddle, which puddle keeps for its catalog sidecars.
  • The path properties write.metadata.path, write.data.path, write.object-storage.path and write.folder-storage.path must be at or under the table’s (or view’s) own location, so the location covers everything written for it.
  • No two tables or views may overlap: a location equal to, inside, or containing another table’s or view’s is rejected, whichever namespaces they are in.
  • A namespace’s location must be strictly under the warehouse location (and outside _puddle), and must not overlap another namespace’s location or a table’s or view’s, except that it may contain what is inside the namespace (its tables and views, and nested namespaces’ locations). A nested namespace’s location doesn’t have to be under its parent’s. Likewise a table or view can’t be placed inside the location of a namespace it isn’t in.

These apply on namespace create and property updates, table create, stage-create and create-by-commit, set-location and property changes in commits, register-table, and view create, register and commits. Locations are compared after normalisation (. segments and trailing slashes are ignored, a .. segment, query or fragment is rejected, and the scheme and bucket/host must match exactly), on / boundaries: a/t1 does not contain a/t10. A location that breaks a rule, or that no configured storage owns, is rejected with 400 BadRequestException.

Paths are taken literally, as Iceberg’s own FileIOs do: percent escapes are not decoded. Iceberg writes partition directories URL-encoded (name=a%20b), and that is the name of the S3 object key or local directory puddle reads and deletes. %2F is part of a name, not a separator. A percent-encoded .. (%2e%2e) is still rejected, since a client that decodes paths would resolve it.

The metastore records each table’s and view’s location, and each namespace location that is set, and checks overlaps against those records, so it needs no metadata reads. The check is atomic with the write: of two concurrent creates at overlapping locations, at most one succeeds (with PostgreSQL, location-changing writes in a warehouse take a transaction lock and run one at a time; SQLite and the in-memory metastore serialize writes anyway). Stage-create is checked too, before credentials are vended, but reserves nothing until the table is committed. Dropping a table or view (with or without purge) frees its location; removing a namespace’s location property frees that one, and its new tables go under the warehouse root again. Registering the metadata of a table that is still in the catalog, under another name, overlaps it and is rejected; drop it first.

The recorded location is what a table owns. Puddle vends credentials for it and purges under it, and nothing else. Renaming a table (also into another namespace) and changing a namespace’s location don’t move any files, so they don’t change what existing tables can reach: a table keeps its directory and its credentials. A table whose metadata names a different location than the recorded one (a pointer changed behind puddle’s back) gets no credentials, and puddle logs a warning.

Moving a table

A table’s location can change (set-location in a commit, which is what Spark’s ALTER TABLE ... SET LOCATION sends) only while the table has no snapshots. Once it has any, current or not, a commit that changes its location is rejected with 400 BadRequestException:

Cannot change the location of table analytics.events: it has snapshots; create a new table at the new location instead

The files those snapshots reference were written under the old location, which the table would no longer own, so puddle could no longer vend credentials for them or purge them. The same goes for a commit that adds a snapshot and then changes the location, one that removes the snapshots while moving (their files are still there), and a replace (CREATE OR REPLACE TABLE ... LOCATION ...) of a table with snapshots at another location, since a replaced table keeps its old snapshots. A snapshot added after the location change in the same commit, as a replace transaction of an empty table sends it, was written for the new location and is accepted. This holds with unrestricted-locations too, and in multi-table transactions, which fail as a whole. To move a populated table, create a new one at the new location and copy the data (CREATE TABLE ... AS SELECT). This is stricter than the Iceberg Java catalogs, which accept any set-location; it keeps one table to one directory and may be relaxed later.

An empty table can move, subject to the rules above, and its recorded location moves with it: the old directory is free for other tables, and credentials and purge follow the new one. Once the move has committed, puddle deletes the metadata files written before it from the old directory, best effort (the metadata log still lists them, as it does files write.metadata.delete-after-commit.enabled removes), so a table later given that directory doesn’t find them there. Engines that cache a table (Spark) keep the credentials vended for the old location until they reload it, and fetch credentials for the new one through the refresh endpoint meanwhile. Trino commits an empty snapshot when it creates a table, so a Trino-created table always has one; Trino can’t change a table’s location anyway.

Views have no data files: a view’s location can change at any time, subject to the rules above, and its recorded location moves with it. Once the move has committed, puddle deletes the view’s metadata files written before it from the old directory, best effort. View metadata doesn’t list its earlier files, so puddle lists the old metadata directory and deletes the metadata files there that belong to this view (by UUID); anything else in it stays. On storage puddle can’t list, only the last metadata file before the move is deleted.

Commits only check locations they change. A table registered from an external warehouse keeps its external location and stays committable, and may set write.metadata.path under its own location.

register-table and register-view accept a metadata file from this warehouse’s storage when it is strictly under the location of the table (or view) it describes, where puddle would have written it; the table’s location and path properties must then follow the rules above. Metadata in another routed warehouse is rejected with 400 BadRequestException: register it through that warehouse’s prefix instead. Registering from an external warehouse is covered below.

Dropping a table with purgeRequested=true deletes only files strictly under the table’s recorded location. Manifests and metadata are written by clients and can name any path; anything else they name (another table’s files, another warehouse, an external warehouse, a path with ..) is left in place, and puddle logs a warning with the number of skipped paths and an example.

Namespace levels and table and view names may not be empty, . or .., or contain / (400 BadRequestException), on create, register and rename. Puddle’s own locations don’t contain names, but engines build paths from them (Trino proposes <namespace location>/<name>-<random> in a namespace with a location). Dots elsewhere in a name (new.db) are fine.

Engines and locations

Spark, PyIceberg and the Iceberg Java client send no location when creating a table unless you give one (LOCATION in Spark), so puddle’s default applies. Trino (iceberg.catalog.type=rest) proposes a location only when the schema has a location property: with iceberg.unique-table-location=true (the default) that is <schema location>/<table name>-<random>, which puddle accepts as it is. In schemas without a location, which is the default, Trino leaves the choice to puddle and its tables get <warehouse>/<uuid> like any other. No Trino setting is needed for that; setting a schema location is what brings table names back into paths, as a prefix under that location.

unrestricted-locations: true on a warehouse turns all of this off, including the overlap check, and accepts any location its FileIO can write, even outside the warehouse. Nothing is recorded, so vending covers whatever location a table’s metadata names. The name rule stays on, and so does the refusal to move a table that has snapshots. Purge stays confined, to the table’s location and its write.metadata.path, and deletes nothing for a table located at a warehouse root or outside every configured warehouse. It exists for conformance suites (the Iceberg REST Compatibility Kit creates tables under /tmp); don’t use it where principals don’t trust each other.

puddle.yaml
warehouses:
  rck:
    location: file:///tmp/puddle-rck
    unrestricted-locations: true

external-warehouses

external-warehouses: declares storage locations belonging to warehouses managed by other metastores, that you want to register tables from. Each entry has the same shape as a routed warehouse — a location URL and the matching per-scheme sub-block — and the same naming and validation rules apply across the union of warehouses: and external-warehouses:.

puddle.yaml
warehouses:
  default:
    location: file:///tmp/puddle-warehouse

external-warehouses:
  partner:
    location: s3://partner-bucket/x
    s3:
      region: us-east-1
      access-key-id: ${PARTNER_AWS_ACCESS_KEY_ID}
      secret-access-key: ${PARTNER_AWS_SECRET_ACCESS_KEY}

External warehouses are register-only: creating a table or view in one is rejected. Tables and views registered from one follow these rules:

  • The registered table’s location, and any write.*.path property it carries, must be strictly inside the same external warehouse (not at its root), or registration fails with 400 BadRequestException.
  • Commits keep writing new metadata files into the external warehouse, under the table’s own location (or its write.metadata.path), as the source catalog would.
  • Purge never deletes anything in an external warehouse. Dropping such a table with purgeRequested=true removes only puddle’s pointer and succeeds; the skipped purge is logged at warn level. The files stay for the catalog that owns them.

authentication

authn: is opt-in. Omit it entirely (and leave out authz: too) and every request runs anonymously with full access — fine for local dev. Puddle logs a warning at startup when it runs this way. Configure it and every request must present a valid bearer token in the Authorization header or get a 401.

Puddle refuses to start rather than fall back to an open catalog by accident:

  • an authn: block that configures no validator (for example static-tokens: [], or jwt: with no issuers) is an error;
  • any authz: setting (mode: allow-authenticated, mode: opa, mode: cel, or an opa: or cel: block) without an authn: block is an error.

Both kinds of token can be configured at once: tokens that look like a JWT are sent to the JWT validator, everything else to the static-token list.

puddle.yaml
authn:
  static-tokens: [ ... ]    # see below
  jwt: { ... }              # see below

Either validator yields a principal (the caller’s subject) and its claims, which is what authorization policy sees of the caller: the claims you configure (a static token’s claims:, or the ones a JWT issuer’s claims: mapping selects, such as groups), plus sub and iss, which puddle always sets itself. Configuration that names sub or iss is refused at boot. Authorization describes the model.

static-tokens

A list of bearer tokens, each tied to a principal name and claims.

puddle.yaml
authn:
  static-tokens:
    - token: ${PUDDLE_ADMIN_TOKEN}
      principal: admin
      claims:
        groups: [admins]
        team: platform
    - token: ${PUDDLE_CI_TOKEN}
      principal: ci-runner
      claims:
        groups: [ci]

token: should always use ${VAR} env-var expansion so the literal secret stays out of source control. claims: is passed to authorization policy as the caller’s claims; the keys are operator-chosen, except sub and iss: puddle sets sub to the token’s principal and iss to static. The values take JSON’s types, as a JWT’s claims do, so numbers are doubles: level: 3 reaches policy as 3.0, the same as a JWT’s "level": 3.

Configuration errors hard-fail at boot: empty token or principal, duplicate tokens, a sub or iss claim, and tokens whose value would be misrouted to the JWT validator.

jwt

OIDC-style validation against one or more issuers. Each request’s JWT is routed to its issuer by exact match on the iss claim.

puddle.yaml
authn:
  jwt:
    issuers:
      - issuer: https://login.example.com/
        audience: puddle                       # string or list of strings
        algorithms: [RS256]                    # allowed algs (default: [RS256, ES256])
        # JWKS source — pick one (default: OIDC discovery)
        jwks-uri: https://login.example.com/.well-known/jwks.json
        # jwks: |                              # inline alternative
        #   { "keys": [ ... ] }
        claims:                                # name → dotted JWT claim path
          email: email
          groups: realm_access.roles

If neither jwks-uri nor jwks is given, puddle does OIDC discovery (GET <issuer>/.well-known/openid-configuration) and follows the returned jwks_uri.

Keys are only fetched over TLS: issuer, jwks-uri, and the jwks_uri a discovery document advertises must all be https:// URLs, or puddle refuses to start. Keys served over plain HTTP could be swapped in transit and used to forge tokens. The one exception is an issuer with inline jwks:, where nothing is fetched and issuer is only compared against the token’s iss claim.

Fetched JWKS are cached and refreshed in the background every 15 minutes. Token verification never waits for a refresh: the previous key set stays in use until the new one has been fetched, and a failed refresh keeps it (logged at warn level). Discovery documents and JWKS responses are capped at 1 MiB. A token whose kid isn’t in the cached set (the IdP rotated its keys) triggers an immediate refetch, at most once every 30 seconds per issuer; other unknown-kid tokens in that window are rejected without contacting the IdP. A token whose kid is known but whose signature doesn’t verify is rejected without a refetch. IdPs normally publish a new key well before signing with it, so rotation needs no operator action.

algorithms: lists the JWS algorithms (the token header’s alg) this issuer may use; it defaults to [RS256, ES256]. none is refused, even if listed. Tokens must carry a kid header, and are verified only with the JWKS key of that kid:

  • A key that declares alg verifies only tokens with exactly that alg (Google, for one, publishes alg).
  • A key without alg (Microsoft Entra ID publishes none) verifies a token whose alg is in algorithms: and fits the key: RS* or PS* for an RSA key, ES256/ES384/ES512 for an EC key on P-256/P-384/P-521, EdDSA or Ed25519 for an Ed25519 key. A symmetric (oct) key without alg is never used, so a public key can’t be turned into an HMAC secret.
  • A key marked use other than sig, or with key_ops that doesn’t include verify, is never used.

Entra ID signs with RS256, which the default allows. Add PS256, RS512, etc. only if your IdP actually uses them.

claims: is an allowlist: only the listed JWT claims are passed through to authorization policy, each under its name here, so policies stay the same across IdPs (cognito:groups or realm_access.roles both become groups). The path is dotted (a.b.c descends into nested objects; a claim whose own name contains a dot can’t be reached), and a claim the token doesn’t carry is left out. Anything not in the map is dropped, apart from sub and iss, which puddle always passes on from the verified token; they can’t be used as names here. Values keep their JSON types: numbers are doubles (see Numbers for what that means in CEL). A non-empty sub is required.

With several issuers, two of them can issue the same sub. A policy that grants by subject should check iss as well.

Debugging: who am I?

GET /api/catalog/v1/whoami echoes back the identity of the calling token: the principal and the claims, exactly as authorization policy sees them. Useful for confirming a static token’s claims: are wired the way you expect, or seeing which JWT claims actually surface under your claims: allowlist. With authn: unconfigured the response is {"principal": "", "claims": {}, "anonymous": true}.

$ curl -s -H "Authorization: Bearer $TOKEN" \
    http://localhost:8089/api/catalog/v1/whoami | jq
{
  "principal": "admin",
  "claims": {
    "groups": ["admins"],
    "iss": "static",
    "sub": "admin",
    "team": "platform"
  }
}

authorization

authz: selects the policy that gates every request after authentication. What a policy sees (the caller, the actions and the resources), how each mode decides, and which to choose are on Authorization.

puddle.yaml
authz:
  mode: allow-authenticated   # allow-authenticated (default) | opa | cel
  opa: { ... }                # required when mode is opa; see below
  cel: { ... }                # required when mode is cel; see below

An opa: or cel: block under another mode is refused at boot, as is any authz: setting without authn:. Decisions are logged with event="authz.decision" in every mode.

allow-authenticated

Default mode. Permit any authenticated request. When neither authn: nor authz: is configured, puddle runs as an open catalog and allows everything (see authentication); setting mode: allow-authenticated explicitly requires authn:.

puddle.yaml
authz:
  mode: allow-authenticated

opa

Delegate every decision to Open Policy Agent, running as a separate process (sidecar, daemonset, or shared service), which puddle calls on every check.

puddle.yaml
authz:
  mode: opa
  opa:
    url: http://localhost:8181/v1/data/puddle/rbac/decision

url is the OPA data API endpoint of the rule that decides: it returns a bare boolean, or {"allow": ..., "reason": "..."}, whose reason becomes the 403’s message. The input puddle posts, and how OPA’s answers and failures are handled, are in Authorization: OPA; a starter policy is in examples/opa/.

cel

Decide every request with one CEL expression, compiled at boot and evaluated in-process: policy without running OPA.

puddle.yaml
authz:
  mode: cel
  cel:
    expression: |
      "admin" in claims.groups
      || (action.startsWith("table.") && "writer" in claims.groups)

expression is required. It sees principal, claims, action and resource, and allows a request only if it returns true for every resource; an expression that doesn’t compile or type-check stops puddle from starting. The variables, the rules and examples are in Authorization: CEL; a starter expression equivalent to the OPA policy is in examples/cel/.

Environment variable expansion

The YAML is parsed first; then ${VAR} references inside string values are replaced with the value of VAR from the process environment. Mapping keys are never expanded, and an expanded value is always taken verbatim as that one value: an environment variable holding a newline or key: value text cannot change the structure of the config.

SyntaxResult
${VAR}the value of VAR; an unset VAR is an error
$$a literal $
any other $kept as-is, so ab$cd stays ab$cd

Only the braced form is expanded: $VAR is literal text, which keeps a $ inside a secret safe. Write $${VAR} for the literal text ${VAR}. VAR must match [A-Za-z_][A-Za-z0-9_]*; anything else after ${ (including shell-style defaults such as ${VAR:-x}) is an error at boot rather than being kept as text.

An unset variable is a hard error — config-load fails listing every missing name with where it was used, e.g. undefined environment variable(s): AWS_ACCESS_KEY_ID (warehouses.prod.s3.access-key-id, line 10). A set but empty variable expands to the empty string.

An unquoted reference can feed a non-string field: max-open-conns: ${PG_MAX_CONNS} decodes as an integer and path-style-access: ${S3_PATH_STYLE} as a boolean. A quoted reference ("${PG_MAX_CONNS}") is always a string.

puddle.yaml
authn:
  static-tokens:
    - token: ${PUDDLE_ADMIN_TOKEN}
      principal: admin
warehouses:
  prod:
    location: s3://my-warehouse
    s3:
      region: us-east-1
      access-key-id: ${AWS_ACCESS_KEY_ID}
      secret-access-key: ${AWS_SECRET_ACCESS_KEY}

Use this for every secret. Don’t paste literal tokens into config files.