API keys

Layer mints its own API keys. An ApiKey is a minted credential as a resource. Layer owns the credential lifecycle — mint, verify, revoke, expire — and what the key opens is declared per resource: each entitlement names a VectorStore, a Warehouse, an Agent, or Layer itself, and carries the scopes and claims for that target. Claims are opaque to Layer — an external system can use Layer as its key store and keep authorization decisions to itself.

Keys have two authoring surfaces that round-trip through one schema: kubectl get apikey -o yaml and GET /v2/keys/{keyId} are two spellings of the same object.

apiVersion: hevlayer.com/v1alpha1
kind: ApiKey
metadata:
  name: cohort-reader
  namespace: layer
spec:
  owner: acme
  description: cohort read access
  entitlements:
    vectorstore.prod-turbopuffer:
      scopes: [read]
      namespaces: ["cohort-*"]
    warehouse.prod-snowflake:
      claims:
        - "notes:cohort:*:read"
  expiresAfter: 365d
status:
  keyId: 0a1b2c3d-…
  phase: Active
  lookupHash: sha256:…
  createdAt: "2026-06-10T00:00:00Z"
  expiresAt: "2027-06-10T00:00:00Z"
  secretRef:
    name: apikey-cohort-reader

You don’t have to mint anything. The default VectorStore credential (the store key you already own) is accepted as an admin bearer. Minting starts from it: admins can also delegate the prefix mint authority below.

Key model

• Tokens look like hvl_iqGFsDD2PNkyhCqr59jjvKuKL47vqXMz. The raw token is returned once, in the mint response. Layer stores only one-way hashes; a lost token is revoked and re-minted, never recovered.
• Every key is an ApiKey resource in the cluster — kubectl get apikeys is the audit trail, and kubectl apply is an equal authoring surface (the operator mints and delivers the token via Secret).
• Revoke is the default end-of-life action. It disables the credential while keeping its owner, entitlements, and lifecycle record available for audit. Propagation takes seconds, not milliseconds.
• Hard delete is a rare, deliberate cleanup action. A key must be Revoked first; deletion then permanently removes its audit record.

Spec

Entitlements

Scope meanings match inbound auth: read covers query, fetch, scans, and metrics; write covers namespace writes and worker routes.

claims is a list of opaque strings, allowed on any entitlement and the only field on a warehouse entitlement. Layer stores them, returns them from list, get, and authenticate, and never interprets them — an existing permission grammar (service:resource_type:resource_id:action strings, a legacy entitlement vocabulary) drops in verbatim, and the consuming application maps them to its own authorization.

An entitlement whose target does not exist grants nothing and surfaces as a status condition (EntitlementTargetMissing) — not an admission error, so keys and their targets can be applied in either order. Check the condition after applying: a typo in a target name looks the same as a missing target.

A key whose entitlements carry only claims — no scopes — is a pure external-store key: it authenticates, but opens no Layer route.

Routes

Admin here means a key with the layer entitlement at admin scope, or the bootstrap gateway key. Prefix mint authority permits the constrained operations described below. authenticate is unauthenticated by construction — the token is the credential.

Prefix mint authority

An admin can give a service a narrow credential for issuing per-machine keys:

{
  "vectorstore.prod-turbopuffer": {
    "scopes": ["mint"],
    "namespaces": ["kit-*"]
  }
}

mint requires a nonempty literal prefix followed by * on a vectorstore entitlement. It permits minting read and/or write grants on that same store, with every namespace an exact name or trailing-star prefix contained in the issuer’s prefix. For example, kit-alice-traces and kit-alice-* fit kit-*. Empty namespace lists, *, other stores, admin or mint grants, warehouse claims, and agent/MCP invocation grants are refused with HTTP 403.

The issuer can list only keys it minted. Getting, revoking, or hard-deleting a key additionally requires that its current entitlements still fit the issuer’s authority. Ownership uses the issuer’s immutable key ID, persisted in the ApiKey annotation hevlayer.com/issuer-key-id; the request’s owner field is descriptive and grants no authority. Older keys without that annotation remain manageable by admins, but cannot be claimed by a minter. Recreating an issuer with the same resource name does not inherit ownership.

The same credential can DELETE /v2/namespaces/kit-alice-traces when that namespace resolves to the granted store. It cannot delete outside its prefix. Mint alone grants no namespace reads or writes and no Pipeline/Function control. Admin credentials keep their existing access.

Provision and use a kit minter

Use the actual VectorStore resource name in the entitlement, rather than its backend type or hostname. An admin provisions the service credential:

curl -X POST "$LAYER_GATEWAY_URL/v2/keys" \
  -H "Authorization: Bearer $ADMIN_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name":"kit-invite-minter","entitlements":{
    "vectorstore.prod-turbopuffer":{"scopes":["mint"],"namespaces":["kit-*"]}
  },"expiresAfter":"90d"}'

Save the returned token as the service’s KIT_MINTER_KEY. The service can issue a machine key for an exact namespace or a narrower prefix:

curl -X POST "$LAYER_GATEWAY_URL/v2/keys" \
  -H "Authorization: Bearer $KIT_MINTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name":"kit-alice-laptop","owner":"alice","entitlements":{
    "vectorstore.prod-turbopuffer":{"scopes":["read","write"],"namespaces":["kit-alice-*"]}
  },"expiresAfter":"30d"}'

curl "$LAYER_GATEWAY_URL/v2/keys?includeRevoked=true" \
  -H "Authorization: Bearer $KIT_MINTER_KEY"
curl -X POST "$LAYER_GATEWAY_URL/v2/keys/$MACHINE_KEY_ID/revoke" \
  -H "Authorization: Bearer $KIT_MINTER_KEY"
curl -X DELETE "$LAYER_GATEWAY_URL/v2/namespaces/kit-alice-traces" \
  -H "Authorization: Bearer $KIT_MINTER_KEY"

MACHINE_KEY_ID is the child’s returned keyId. Use ["read"] or ["write"] for a single-scope child. A request containing both kit-alice-traces and orders is refused in full, as are orders alone, kit*, and a child with admin or mint scope. An attempt to revoke a key minted by another issuer returns 403 with {"error":"key was not minted by this issuer"}, even if its namespace is inside kit-* and its owner is alice. Deleting orders also returns 403. Revoking a machine key does not delete its archive; namespace deletion is a separate request.

Mint authority is shared across the granted prefix: this credential can issue keys for every kit-* tenant and delete their namespaces. Keep it in the invite service; give each machine only its child token. If the service needs namespace discovery, an admin must explicitly add read to the minter’s store grant. That also grants data reads within the prefix.

Mint

key = await client.mint_key({
    "name": "cohort-reader",
    "owner": "acme",
    "entitlements": {
        "vectorstore.prod-turbopuffer": {
            "scopes": ["read"],
            "namespaces": ["cohort-*"],
        },
        "warehouse.prod-snowflake": {
            "claims": ["notes:cohort:*:read"],
        },
    },
    "expiresAfter": "365d",
})
print(key.token)  # shown once, never again

key, err := client.MintKey(ctx, &hevlayer.MintKeyRequest{
	Name:  "cohort-reader",
	Owner: "acme",
	Entitlements: hevlayer.ApiKeyEntitlements{
		"vectorstore.prod-turbopuffer": {
			Scopes:     []string{"read"},
			Namespaces: []string{"cohort-*"},
		},
		"warehouse.prod-snowflake": {
			Claims: []string{"notes:cohort:*:read"},
		},
	},
	ExpiresAfter: "365d",
})
fmt.Println(key.Token) // shown once, never again

const key = await client.mintKey({
  name: "cohort-reader",
  owner: "acme",
  entitlements: {
    "vectorstore.prod-turbopuffer": {
      scopes: ["read"],
      namespaces: ["cohort-*"],
    },
    "warehouse.prod-snowflake": {
      claims: ["notes:cohort:*:read"],
    },
  },
  expiresAfter: "365d",
});
console.log(key.token); // shown once, never again

curl -X POST "$LAYER_GATEWAY_URL/v2/keys" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "cohort-reader",
    "owner": "acme",
    "entitlements": {
      "vectorstore.prod-turbopuffer": {"scopes": ["read"], "namespaces": ["cohort-*"]},
      "warehouse.prod-snowflake": {"claims": ["notes:cohort:*:read"]}
    },
    "expiresAfter": "365d"
  }'

The response is 201 Created:

{
  "keyId": "0a1b2c3d-4e5f-6071-8293-a4b5c6d7e8f9",
  "name": "cohort-reader",
  "owner": "acme",
  "entitlements": {
    "vectorstore.prod-turbopuffer": {"scopes": ["read"], "namespaces": ["cohort-*"]},
    "warehouse.prod-snowflake": {"claims": ["notes:cohort:*:read"]}
  },
  "phase": "Active",
  "createdAt": "2026-06-10T00:00:00Z",
  "expiresAt": "2027-06-10T00:00:00Z",
  "token": "hvl_iqGFsDD2PNkyhCqr59jjvKuKL47vqXMz"
}

An agent invocation entitlement is "agent.<name>": {}. It does not accept scopes or namespace restrictions.

Validation failures are 400s with field-level messages; nothing is created on a failed mint. An entitlement whose target does not exist is accepted — it grants nothing and surfaces as an EntitlementTargetMissing condition on the resource, so keys and targets can be applied in either order.

Mint an admin key the same way — "entitlements": {"layer": {"scopes": ["admin"]}}. An admin key can mint further keys; key custody is the only hierarchy.

List and get

keys = await client.list_keys()
key = await client.get_key("0a1b2c3d-4e5f-6071-8293-a4b5c6d7e8f9")

keys, err := client.ListKeys(ctx, nil)
key, err := client.GetKey(ctx, "0a1b2c3d-4e5f-6071-8293-a4b5c6d7e8f9")

const keys = await client.listKeys();
const key = await client.getKey("0a1b2c3d-4e5f-6071-8293-a4b5c6d7e8f9");

curl "$LAYER_GATEWAY_URL/v2/keys" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY"

{
  "keys": [
    {
      "keyId": "0a1b2c3d-4e5f-6071-8293-a4b5c6d7e8f9",
      "name": "cohort-reader",
      "owner": "acme",
      "entitlements": {
        "vectorstore.prod-turbopuffer": {"scopes": ["read"], "namespaces": ["cohort-*"]},
        "warehouse.prod-snowflake": {"claims": ["notes:cohort:*:read"]}
      },
      "phase": "Active",
      "createdAt": "2026-06-10T00:00:00Z",
      "expiresAt": "2027-06-10T00:00:00Z",
      "lastSeenAt": "2026-06-10T12:00:00Z"
    }
  ]
}

Listings carry metadata only — never tokens or recoverable hashes. lastSeenAt advances at most once per five minutes per key.

Authenticate

External systems present a raw token and get back keyId — a stable actor id — plus the full entitlements map, then make their own authorization decisions from the claims. This is the verb that makes Layer a key store for applications that keep authorization to themselves: Layer never interprets a claim string.

identity = await client.authenticate_key({"token": presented})
claims = identity.entitlements["warehouse.prod-snowflake"].claims

identity, err := client.AuthenticateKey(ctx, &hevlayer.AuthenticateKeyRequest{
	Token: presented,
})
claims := identity.Entitlements["warehouse.prod-snowflake"].Claims

const identity = await client.authenticateKey({ token: presented });
const claims = identity.entitlements["warehouse.prod-snowflake"].claims;

curl -X POST "$LAYER_GATEWAY_URL/v2/keys/authenticate" \
  -H "Content-Type: application/json" \
  -d '{"token": "hvl_iqGFsDD2PNkyhCqr59jjvKuKL47vqXMz"}'

200 returns {keyId, name, owner, entitlements, expiresAt}. Invalid, revoked, and expired tokens all answer 401 indistinguishably, in constant time on a miss. The route is rate-limited.

Verification

External systems present the raw token to POST /v2/keys/authenticate and get back keyId (a stable actor id) plus the full entitlements map, then make their own authorization decisions from the claims. The gateway also accepts any Active key’s token as a bearer on its own routes, enforcing the entitlement for the store or control-plane surface the route resolves to.

Verification is one indexed lookup plus one hash check against a watch-fed in-memory map — the hot path never reads the control plane per request. status.lastSeenAt advances at most once per five minutes per key.

Revoke is the default lifecycle end-state. Revoked ApiKey resources are retained indefinitely for audit and are not automatically garbage-collected. Operators should set a retention policy appropriate to their obligations. Hard delete is rare and deliberate: DELETE /v2/keys/{keyId} accepts only a Revoked key and permanently removes the resource, its lifecycle record, and any owned token Secret. Other phases return 409 Conflict; revoke first, then delete only after the retention period has passed or policy requires the record itself to be forgotten.

Deleting a VectorStore or Warehouse inerts every entitlement that names it: the keys stay Active for their other entitlements, and the deletion is finalizer-guarded on the target’s side while keys still reference it.

Revoke by default

await client.revoke_key("0a1b2c3d-4e5f-6071-8293-a4b5c6d7e8f9")

_, err := client.RevokeKey(ctx, "0a1b2c3d-4e5f-6071-8293-a4b5c6d7e8f9")

await client.revokeKey("0a1b2c3d-4e5f-6071-8293-a4b5c6d7e8f9");

curl -X POST "$LAYER_GATEWAY_URL/v2/keys/$KEY_ID/revoke" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY"

The gateway stops accepting a revoked key within seconds. Rotation is mint-new, deploy, revoke-old — there is no in-place rotation. Revoked keys remain in ?includeRevoked listings indefinitely: Layer does not automatically garbage-collect them. Set an external retention policy that meets your audit requirements.

Hard delete is the explicit exception for records that must be forgotten. It is accepted only after the key is Revoked; deleting an Active, Pending, or Expired key returns 409 Conflict:

curl -X DELETE "$LAYER_GATEWAY_URL/v2/keys/$KEY_ID" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY"

Deleting removes the ApiKey resource and its audit history permanently. For CRD-authored keys, Kubernetes also garbage-collects the owned token Secret. Use deletion only after the applicable retention period has passed or when policy requires the record itself to be removed.

Using a minted key

A minted key works anywhere its entitlements reach. A vectorstore.<name> entitlement opens data-plane routes against that store, inside its namespace globs:

curl "$LAYER_GATEWAY_URL/v2/namespaces/cohort-7/query" \
  -X POST \
  -H "Authorization: Bearer hvl_iqGFsDD2PNkyhCqr59jjvKuKL47vqXMz" \
  -H "Content-Type: application/json" \
  -d '{"rank_by": ["text", "BM25", "acme"], "top_k": 10}'

Outside the grant, the gateway answers 403:

{"error": "namespace not in key grant", "namespace": "orders"}

A request that needs a higher scope answers 403 with the scope named:

{"error": "insufficient API key scope", "required_scope": "admin"}

A key whose entitlements carry only claims — no scopes — is a pure external-store key: it authenticates, but opens no Layer route.

On a deriveFromStore gateway, minted keys are verified by Layer with the same entitlements, scopes, and namespace globs as in keys mode. Requests use the selected store’s configured credential upstream. Invalid, expired, or revoked minted tokens are rejected with 403; they never fall back to upstream pass-through. Other bearers retain their existing pass-through behavior.

For /mcp/demo, grant "mcp.demo": {} together with a read entitlement for the server’s namespaces, for example "vectorstore.default": {"scopes": ["read"], "namespaces": ["docs"]}. The MCP entitlement opens the server; the store grant controls which tools and data the caller can access.

A minted key is gateway-only and cannot authenticate directly to the upstream store. SDKs return connection errors when the gateway is unreachable; see client failure behavior.

kubectl

The resource is the other authoring surface — apply an ApiKey with no credential and the operator mints the token into a Secret named by status.secretRef:

kubectl apply -f key.yaml
kubectl get apikeys -n layer
kubectl get secret apikey-cohort-reader -n layer -o jsonpath='{.data.token}' | base64 -d

The operator writes the token under the Secret’s token key and moves phase from Pending to Active. The Secret is the token delivery; it is owned by the ApiKey and garbage-collected with it. Rotate by minting a replacement, deploying it, and revoking the old key.

Kubernetes RBAC

CRD authoring makes kubectl a minting surface, so the chart ships roles to delegate key administration without cluster-admin:

Neither role aggregates into the built-in view/edit/admin ClusterRoles: namespace viewer never silently means key viewer. Bindings are the cluster operator’s explicit act; set rbac.keyRoleBindings in Helm values to render them for the single-team case.

Bootstrapping

LAYER_GATEWAY_API_KEY is the bootstrap credential: it mints the first admin key —

spec:
  entitlements:
    layer:
      scopes: [admin]

— after which routine minting uses minted admin keys. Cluster operators can equally bootstrap by applying an ApiKey resource, since CRD authoring needs only kubectl access.

Backup and migration

Use a full namespace Velero backup, or an equivalent etcd snapshot, for portable ApiKey migration. Do not export and apply YAML: the verifier lives in status, and apply paths do not preserve that subresource.

1. Back up these Layer namespace objects:

   • ApiKey resources with status: status.keyId, phase, lookupHash, and tokenHash are the credential record; the raw token is not recoverable.
   • The chart’s *-keys Secret: it holds LAYER_KEY_PEPPER, which must match every restored verifier.
   • Token-delivery Secrets referenced by CRD-authored keys through status.secretRef.

2. Restore ApiKey status explicitly. Velero restores specs by default and drops status unless the restore uses --status-include-resources apikeys.hevlayer.com, or the Restore sets spec.restoreStatus.includedResources: ["apikeys.hevlayer.com"].

3. Restore the namespace, including the pepper Secret, before running Helm on the target release. If Helm touches an empty namespace first, it can generate a new random pepper, making restored argon2id verifiers unusable.

4. If source and target Velero servers share object storage, wait until the target server lists the completed source backup before creating the restore; otherwise its local Backup object may not have synced.

5. Refresh source Velero discovery before backup. Install Layer CRDs before Velero discovers resources, or restart the source Velero deployment after CRD installation and wait for readiness; stale discovery can produce a Completed backup that omits ApiKey.

6. Quiesce the Layer operator before backup and keep it quiesced through initial restore. Velero restores ApiKey spec and status separately; an operator that starts in that window can mark a transient spec-only key MintBlocked or Pending.

7. Inspect the completed backup from inside the Velero server’s cluster. Its item list must include apikeys.hevlayer.com, expected ApiKey objects, the chart’s *-keys pepper Secret, and token-delivery Secrets referenced by status.secretRef.

8. Restart gateway/operator discovery after restore so the watch-fed key map rebuilds from restored ApiKey status and Secrets before traffic depends on the migrated tokens.

9. Treat MintBlocked as a failed restore, not a key rotation. The operator refuses to mint a replacement when status.keyId is empty but the deterministic delivery Secret exists; it writes a MintBlocked condition and warning Event instead. A new CRD-authored ApiKey with no delivery Secret still mints normally.

Validate the key store before accepting traffic:

SOURCE_CONTEXT=layer-a \
TARGET_CONTEXT=layer-b \
LAYER_NAMESPACE=layer \
SOURCE_BASE_URL=https://source.example.com \
TARGET_BASE_URL=https://target.example.com \
SOURCE_ADMIN_KEY=hvl_... \
scripts/apikey-velero-migration.sh

The script mints REST and CRD-authored keys, backs up the namespace, waits for the target Velero server to sync the completed backup, restores with --status-include-resources apikeys.hevlayer.com, and asserts:

• original active tokens still authenticate
• revoked and expired tokens remain refused
• the completed backup contains expected ApiKey resources and Secrets
• the source operator was quiesced and the expired control key reached Expired before backup
• the restored pepper Secret matches the source
• restored key status fields are present, so spec-only, incomplete, racing, or misordered restores fail loudly

Local reproduction currently needs the temporary test-license setup from the /v2/keys test kid keygen work in PR #199.

Velero backups are credential-bearing: they contain one-way verifiers, the pepper, and any remaining CRD-authored delivery Secrets. Store and encrypt them like other credential material.

CLI

The same operations from the CLI:

layer keys mint cohort-reader --owner acme \
  --entitle vectorstore.prod-turbopuffer=read \
  --namespaces "cohort-*" \
  --claim warehouse.prod-snowflake="notes:cohort:*:read"
layer keys ls
layer keys revoke cohort-reader
# Rare cleanup after the retention period:
layer keys rm cohort-reader

layer keys mint prints the token once, alone on stdout for piping; the metadata table goes to stderr.