CLI retrieval

JSON contract

Retrieval commands emit JSON by default. -o json makes that intent explicit; -o names is rejected. Supply a JSON request object through -f request.json or -f - for stdin. The request uses the existing gateway grammar; the CLI preserves numeric IDs, filters, ranking expressions, and unknown fields for gateway validation. It does not rewrite a query into another retrieval mode.

layer query articles -f query.json -o json
layer search articles -f search.json -o json
layer read articles -f read.json -o json
layer capabilities articles -o json
layer capabilities primary --store -o json

Query, search, read, and scan replies have this envelope:

{
  "contract": "layer.retrieval.v1",
  "data": { "rows": [], "next_cursor": "opaque" },
  "headers": { "x-layer-next-cursor": "opaque" }
}

data is the complete gateway JSON body, including actual hybrid legs, routing, performance, rerank degradation, and future response extensions. headers includes any returned x-layer-next-cursor, x-layer-stable-as-of, x-layer-warning, x-layer-cache, and traceparent. Missing fields stay missing. A successful command prints one JSON object. Failures print no result, return exit 1, and report the HTTP status and gateway error code on stderr. Malformed local input returns exit 2. Error messages and credentials are not copied from gateway bodies. Requests are not retried or redirected.

--timeout 30s bounds each command (including both capability reads). SIGINT and SIGTERM cancel in-flight requests. Cancellation does not delete a server scan job: use scan delete explicitly. Request files are limited to 8 MiB and responses to 32 MiB; reduce page size if a response exceeds the limit.

Supported retrieval APIs

Command	Gateway API and semantics
query NS -f FILE	POST /v2/namespaces/NS/query; vector, stored-vector seeds, BM25, HybridText, Embed, ordered/filter-only queries, batch and supported native fusion/aggregation bodies
query-all -f FILE	POST /v2/query; federated retrieval across body-specified namespaces
explain NS -f FILE	Native explain_query plan where the store supports it
search NS -f FILE	POST /v2/namespaces/NS/search; schema-based query embedding, actual ANN/BM25/fuzzy leg reports and optional reranking; no pagination
read NS -f FILE	POST /v2/namespaces/NS/documents; ids plus optional include_attributes, returning documents and missing IDs
capabilities NS	Namespace store report plus metadata/schema; fails if either read fails
capabilities STORE --store	GET /v2/vectorstores/STORE/capabilities; store report only
scan create NS -f FILE	Create count, IDs or values selection; filters, full text, HybridText or ANN radius where supported
scan list NS / scan get NS ID	Discover jobs and read status, failure and completeness
scan results NS ID --limit 1000 --offset 0	Read a bounded IDs/values page; limit 1..10000, offset nonnegative
scan delete NS ID	Delete server scan state

This is a consumer of the existing query, search, and embedding contracts. Unsupported store operations retain their gateway error; the CLI implements no local fuzzy, regex, dense, or hybrid search engine.

Capabilities and actual legs

Namespace capability output is {contract, capabilities, metadata}. capabilities identifies the selected store and its declared features and limits. metadata.schema identifies namespace indexing, including full_text_search, fuzzy, vector definitions, and any embed model declarations. Store support alone does not prove an attribute is indexed, nor that a particular query ran every possible leg.

Preflight must inspect both the feature report and the schema. Preserve the receipt in the private integration, then inspect the query response’s actual hybrid echo. HybridText can execute a single BM25 leg when fuzzy indexing is absent; hybrid.legs: 1 alone does not prove dense or fuzzy retrieval. Query routing may add ANN only where configured and supported. /search reports individual effective leg kinds and dropped legs; use those reports to label runs. Never infer executed legs from a command name.

BM25 uses rank_by: ["text", "BM25", "query"]. HybridText uses rank_by: ["text", "HybridText", "query", {"fuzziness":"auto"}]. Fuzzy and regex filters are sent using the gateway grammar only where the store and attribute support them. There is no separate CLI regex contract: check the effective store report and a successful gateway response for the exact requested semantics. Missing support is a gap, not a fallback engine.

Dense embedding compatibility

For schema-managed embeddings, query with rank_by: ["text", "ANN", ["Embed", "query text"]] and inspect the namespace’s embed declaration. The gateway resolves that declaration for query embedding. A source declaration alone does not prove historical vectors were produced by it: verify the existing corpus embedding manifest before claiming compatibility. search also uses the namespace’s schema embedding declaration; set "rerank": false explicitly when reranking is unwanted.

For numeric ANN, including batch, federated and radius-scan selectors, query, query-all, and scan create require --embedding-space space.json. The receipt must contain matching query and document identities:

{
  "query": {
    "model": "model-name", "revision": "immutable-model-revision",
    "dims": 2, "normalization": "l2", "distance_metric": "cosine_distance"
  },
  "document": {
    "model": "model-name", "revision": "immutable-model-revision",
    "dims": 2, "normalization": "l2", "distance_metric": "cosine_distance"
  },
  "document_vectors_verified": true,
  "document_manifest_sha256": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
}

Numeric ANN accepts one vector per leg; multivector bags are outside this verified-space CLI contract. The CLI checks identity equality, complete fields, finite vector components, and dimension agreement before querying. This is an assertion from a verified private corpus receipt; the CLI cannot authenticate the provenance of stored vectors or the manifest. Pin the manifest and verify its contents separately. Do not fill it with guessed identities. nearest_to_id reuses the stored vectors without creating query embeddings. Compatibility verification requires no row writes or bulk reembedding.

Pagination and exhaustive selection

Pass a returned data.next_cursor (or the corresponding response header) as cursor in the next query body. The CLI fetches one page and does not claim that a top-k result exhausts a namespace. Some stores use ordered queries plus an advancing ID filter instead of cursors; unsupported cursor modes fail.

For exhaustive selection, create a job with the required selector, for example:

{"mode":"ids","source":"origin","exhaustive":true,"filters":["year","Gte",2020]}

Poll scan get until the gateway reports completion, then page results by offset and limit. Check failures, totals, and truncated where returned; never treat an unfinished, truncated, capped or failed scan as exhaustive. Fetch selected IDs with read as needed. Job creation and result pagination are separate operations so cancellation does not lose the continuation ID.

Pinning an executable

Build from a committed source revision in the private monorepo:

cd apps/layer-cli
revision=$(git rev-parse HEAD)
go build -trimpath -ldflags "-X main.version=cli-retrieval-v1.$revision" -o /tmp/layer-retrieval .
/tmp/layer-retrieval version
shasum -a 256 /tmp/layer-retrieval

Record source revision, executable hash, Go version, contract version, namespace/store capability receipt, corpus manifest, embedding identities, and actual query legs in the private integration. Pass gateway credentials through the existing environment configuration, outside agent tool payloads. Gold relevance labels belong outside retrieval tools and receipts.

The namespace /v1/.../query compatibility alias is covered by the v2 query command. Agent-specific retrieval at /v2/agents/NAME/query is outside these row-retrieval commands; its embedding-profile and tool-loop contract remains in the agent SDK. The CLI does not expose telemetry metric queries as document retrieval.