bastion-agent/docs/manifest.md

bastion agent job manifest: schema

The manifest is the contract between the agent and whoever directs it: what to read, what to do with it, and who may query the result. Every part of the agent validates against it. The loader is bastion_agent/manifest.py; the signature is checked by the runtime (runtime.md) with bastion_agent/x509gate.py.

What a bastion agent is

A bastion agent is a program that runs on the owner's own machine and does the data work that must stay there: walk a folder, extract text, sterilize personal data, chunk, embed with a local model, catalog, and seal. A model on the machine may serve as a sensor (for recall of personal data, embeddings and summaries), and that is all: the same rule as the PII sterilizer (pii-sterilization.md), a deterministic program owns the control flow and every change, and a model proposes, never acts.

The owner's host writes a job manifest, and it is signed (by the bastion, in the managed install). The agent executes exactly that manifest and nothing else. The output is a catalog (small, structured cards per file and per directory) plus local vectors. The files themselves never move; a host reasons over them by querying the agent. Sharing is a grant that lets another host query this catalog: access moves, data does not.

Rules the schema enforces

1. Plaintext never leaves. sink.plaintext is the constant never. Any other value is a validation error, not a setting.
2. Nothing runs that was not granted. mask is the ceiling; every stage must be covered by an op in the mask, or the manifest is refused before it runs.
3. Nothing leaves unsealed. A catalog sent to the host, seal_to recipients, or any grant requires the seal stage.
4. No addresses. The sanitizer and the embedder are found from the machine's environment when the agent runs. The manifest never carries a host:port.
5. Signed by a CA-issued identity. The runtime refuses a manifest whose signing certificate does not chain to the root in FREEAGENT_CA_ROOT, and refuses to judge at all without that root, because "signed by the key in the certificate next to the manifest" proves nothing. The signing certificate must carry exactly one CN. The loader checks the shape and computes the signing payload; the runtime checks the signature and the chain in the standard library (ECDSA P-256 / SHA-256), so the agent needs neither openssl nor a third-party package on the machine.

File layout

freeagent: 1
id: <slug>
owner:
  host: <slug>                 # the authoring host's cert CN

scope:
  roots: [<abs-or-~path>, ...]
  exclude: [<glob>, ...]        # optional
  follow_symlinks: false        # optional
  max_file_bytes: 268435456     # optional

mask: [read, search, index, summarize]   # subset; 'read' is mandatory

stages: [walk, extract, sanitize, chunk, embed, catalog, seal]  # ordered subset

sanitize:                       # required iff 'sanitize' in stages
  mode: sterilize | flag
  scope: <token scope>          # optional, default = id
  verify: false                 # optional

embed:                          # required iff 'embed' in stages
  model: <name>
  dim: <int>
  chunk:
    size_tokens: <int>
    overlap_tokens: <int>

summarize:                      # optional; needs 'summarize' in mask
  model: <name>
  per: [file, directory]

sweep: 15m | hourly | daily | manual

sink:
  catalog: local | host
  vectors: local
  plaintext: never
  seal_to: [<key-id>, ...]      # optional

grants:                         # optional, default []
  - host: <slug>
    ops: [search, read_range]
    until: <YYYY-MM-DD>

signature:                      # optional in the file; runtime requires it
  alg: ecdsa-p256-sha256
  cert: <path>
  value: <base64>

The first key, freeagent: 1, keeps the program's first name on purpose: the bastion writes it into every manifest it generates, and renaming it would make the agent refuse them.

Fields

Top level

Field	Required	Rules
freeagent	yes	Schema version. Integer 1.
id	yes	Lowercase slug [a-z0-9-]+, 1–63 characters. Names the catalog and is the default token scope for personal data.
owner.host	yes	Slug, 1–63 characters. The authoring host's certificate CN; the runtime checks the signing certificate's CN against it.

Unknown keys anywhere are a validation error, so a misspelled excludes: fails loudly instead of silently indexing everything.

scope

Field	Required	Rules
roots	yes	Non-empty list of paths. Each must be absolute (POSIX /srv/a, a Windows drive C:\Users\me or C:/Users/me, UNC \\server\share) or start with ~/ (or be exactly ~), which expands to the running user's home on every OS. Relative paths are refused, so a manifest means the same thing wherever it lands. No duplicates.
exclude	no	List of glob strings, matched against the /-separated path relative to each root (** spans directories; a pattern with no / also matches any single path segment, so node_modules prunes it at any depth). Case-insensitive on Windows only. Default []. See walk.md.
follow_symlinks	no	Boolean, default false.
max_file_bytes	no	Positive integer, default 268435456 (256 MiB). Larger files are catalogued by metadata only, never read.

mask: the ceiling

A non-empty list from read, search, index, summarize, with no duplicates.

• read: mandatory; every stage reads.
• search: the agent may answer search queries over the vectors.
• index: the agent may write vectors and catalog entries. Required by embed and catalog.
• summarize: a local model may write summaries. Required by the summarize section.

There is no write and no delete: an agent never changes the files it reads. Any other value is an error.

stages: what runs, in order

A non-empty list, a subset of the canonical order, in canonical order:

walk → extract → sanitize → chunk → embed → catalog → seal

Stage	Needs	Does
walk	(always first, always present)	The file system and metadata map: path, size, mtime, type, hash. The cheapest thing a host can plan from.
extract	walk	Text out of documents, code and (with a local OCR tool) images.
sanitize	extract	Personal data replaced by tokens in the text every later stage reads; never in the source files.
chunk	sanitize	Splits the sanitized text for embedding.
embed	chunk; mask index	Vectors from a local model, stored locally.
catalog	sanitize; mask index	Per-file and per-directory cards: what it is, when, entity counts, summary. This is the curated dataset.
seal	catalog	Encrypts the catalog to the owner's key and sink.seal_to.

Out-of-order, duplicate or unknown stage names are errors, as is a stage whose prerequisite stage is absent.

sanitize

Present if and only if sanitize is in stages.

Field	Required	Rules
mode	yes	sterilize: the full pipeline (rules, local-model recall, vault). flag: the rules pass only; deterministic, no model, and not "no personal data" (see sanitize.md).
scope	no	Token scope, 1–63 characters. Default: the manifest id. The same (scope, type, value) gets the same token; a different scope gets a different token, so two catalogs cannot be linked through placeholders.
verify	no	Boolean, default false. A second, adversarial model pass (sterilize only; an error with flag).

Where the sterilizer runs is never in the manifest; the agent reads it from its environment and fails closed if it is absent.

embed

Present if and only if embed is in stages.

Field	Required	Rules
model	yes	Model name, 1–128 characters.
dim	yes	Integer 1–8192. The stage refuses to run if the local endpoint's vectors have another dimension. Vectors are stored per model, so a later model change never discards existing vectors.
chunk.size_tokens	yes	Integer 1–8192.
chunk.overlap_tokens	yes	Integer, 0 ≤ overlap < size.

There is no endpoint field. Embedding is local, always.

summarize

Optional. Requires summarize in mask and catalog in stages.

Field	Required	Rules
model	yes	Local model name, 1–128 characters.
per	yes	Non-empty subset of file, directory.

sweep

One of 15m, hourly, daily, manual.

sink

Field	Required	Rules
catalog	yes	local: the sealed catalog stays here, and the host queries it. host: the sealed catalog is sent to the owner's host, so it can plan while this machine is asleep. Requires the seal stage.
vectors	yes	local, the only value.
plaintext	yes	never, the only value, ever. It is there so the rule is written down in every manifest and can be searched for, not remembered.
seal_to	no	List of recipient keys, 1–128 characters each, no duplicates. Requires the seal stage. The owner's key is always added by the agent and need not be listed.

grants

An optional list, default []. Each entry lets another host query this catalog. Requires the seal stage.

Field	Required	Rules
host	yes	Slug, 1–63 characters: the grantee host's certificate CN. Must not equal owner.host. Unique across grants.
ops	yes	Non-empty subset of search, read_range. search requires search in the mask; read_range requires read (always present).
until	yes	YYYY-MM-DD expiry, a real calendar date.

Revoking is removing the entry and re-signing; serve stops serving that host on its next request. The grantee's client certificate must also chain to FREEAGENT_CA_ROOT (serve.md).

signature

Optional in the file, so a manifest can be written, validated and reviewed before it is signed. The runtime refuses an unsigned manifest. python3 -m bastion_agent validate --require-signed enforces that here.

Field	Required	Rules
alg	yes	ecdsa-p256-sha256.
cert	yes	Path to the signing host's PEM certificate, 1–256 characters, read relative to the manifest.
value	yes	Base64 signature over the signing payload.

Signing payload = the manifest with signature removed, serialized as canonical JSON (sorted keys, no whitespace, UTF-8, dates as ISO strings). bastion_agent.signing_payload(doc) produces it and signing_digest(doc) its SHA-256 hex, so the signer and the runtime agree byte for byte. Any ECDSA P-256 / SHA-256 signer will do; the managed install signs in the bastion and records the result with sign --value. By hand, with openssl:

python3 -m bastion_agent payload manifests/home-documents.yaml > payload.json
openssl dgst -sha256 -sign <host key> payload.json | base64 -w0
python3 -m bastion_agent sign manifests/home-documents.yaml --value <that base64> --cert <cert path>

(sign --key <host key> --cert <cert path> does the same in one step.)

The gate, in the standard library: exactly one CN, equal to owner.host; the certificate inside its validity window and either identical to a certificate in FREEAGENT_CA_ROOT (the managed install: its certificate is its own root, and its self-signature must verify) or signed directly by one that is a CA (basicConstraints, and keyCertSign when the root has a keyUsage extension); and the signature over this payload. It is a one-level PKI only: longer chains are not built.

Loader

bastion_agent/manifest.py (Python 3, PyYAML). Library API:

from bastion_agent import load_manifest, signing_payload, signing_digest
m = load_manifest("manifests/home-documents.yaml")   # raises ManifestError listing every problem
m.id, m.owner_host, m.scope.roots, m.mask, m.stages   # typed, frozen
m.sanitize, m.embed, m.summarize                      # None when the section is absent
m.sink.catalog, m.grants, m.signed                    # grants: tuple of Grant
signing_payload(m.raw)                                # bytes the signature covers

CLI (from the repository root):

python3 -m bastion_agent validate [--require-signed] <manifest.yaml> [...]
python3 -m bastion_agent show <manifest.yaml>          # stages, mask, grants, signed?
python3 -m bastion_agent payload <manifest.yaml>       # canonical signing payload to stdout

Validation collects all errors before failing, so one pass shows everything wrong with the file. python3 -m bastion_agent validate manifests/*.yaml validates every example. python3 -m unittest bastion_agent.test_manifest runs the loader's tests, which also load manifests/home-documents.yaml, so the example can never drift from the schema.

Deliberately absent

• A write op. An agent reads; adding a way to change files is a decision of its own, not a field.
• An endpoint or address field anywhere, for the reason in rule 4.