privatae/bastion-agent
2
0
Fork 0
Code Issues Actions Packages Projects Releases Wiki Activity

bastion agent 0.1.16
ci / test (push) Successful in 1m1s
Details
ci / personal-data (push) Successful in 3s
Details
ci / reproducible (push) Successful in 4s
Details

Browse Source
This commit is contained in:
Privatae
2026-10-25 23:16:50 +00:00
parent 53094a6940
commit e3074f50b0
34 changed files with 1345 additions and 156 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+55 -23
View File
@@ -1,16 +1,18 @@
# bastion agent
[![ci](https://git.privatae.ai/privatae/bastion-agent/actions/workflows/ci.yml/badge.svg?branch=main)](https://git.privatae.ai/privatae/bastion-agent/actions)
[![ci](https://github.com/Privatae/bastion-agent/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/Privatae/bastion-agent/actions/workflows/ci.yml)
The small program the bastion runs on your own machine, for one folder.
It is deployed **from a grant, never installed by a person**: the bastion fetches
It is deployed **from a grant**, and you do not install it yourself: the bastion fetches
`privatae.ai/bastion-agent/version.json`, checks its ed25519 release signature
(`version.json.<first 16 hex of its sha256>.sig`) against the keys compiled into the bastion, downloads the `.pyz`
and checks it against the sha256 in that signed manifest,
mints a P-256 identity per store, writes and signs a manifest from the grant you
approved, and supervises `sweep` and `serve`. Your files never leave the machine;
what crosses the tunnel is the index and what the grant's mask allows.
approved, and supervises `sweep` and `serve`. Your files stay on the machine;
what crosses the tunnel is what the grant's mask allows: catalog cards, and
the sanitized excerpts that `search` and `read_range` return, each reply naming
the sanitize mode that made it (`sanitize_mode`, [serve.md](docs/serve.md)).
The published version is whatever `privatae.ai/bastion-agent/version.json`
names. `bastion_agent/VERSION` on `main` is the next release, which may not be
@@ -18,7 +20,7 @@ published yet. Release manifests are published signed from 0.1.13 on, and the
`.pyz` can be rebuilt from this source and compared
([Verify a release](#verify-a-release)).
This repository is free software, licensed under the GNU General Public License version 3 or (at your option) any later version (see `LICENSE`). Copyright Privatae LLC. You may run it, read it, change it and redistribute it under that license. The Privatae platform the bastion connects to is separate, proprietary software and is not covered by this license. A member runs their own build on their own machine — nothing here depends on a Privatae-held credential or a Privatae-side build.
This repository is free software, licensed under the GNU General Public License version 3 or (at your option) any later version (see `LICENSE`). Copyright Privatae LLC. You may run it, read it, change it and redistribute it under that license. The Host the bastion connects to is separate software and is not covered by this license: a C-Series Host is [Privatae/cseries](https://github.com/Privatae/cseries), under AGPL-3.0-only, and a Host on Privatae's own platform is proprietary software. A member runs their own build on their own machine — nothing here depends on a Privatae-held credential or a Privatae-side build.
## What it does
@@ -30,13 +32,13 @@ failure stops the run:
|---|---|---|
| walk | the map of the folder: files, sizes, types, what the grant excludes | [walk.md](docs/walk.md) |
| extract | text out of each file, with built-in extractors or local tools | [extract.md](docs/extract.md) |
| sanitize | finds personal data and replaces it with tokens; the originals stay in a vault on this machine | [sanitize.md](docs/sanitize.md), [pii-sterilization.md](docs/pii-sterilization.md) |
| sanitize | replaces the personal data its mode finds with tokens; the originals stay in a vault on this machine. In the mode the bastion deploys it does not find names, street addresses or dates of birth ([below](#how-the-bastion-runs-it)) | [sanitize.md](docs/sanitize.md), [pii-sterilization.md](docs/pii-sterilization.md) |
| chunk, embed | splits the text and embeds it with a model on this machine | [embed.md](docs/embed.md) |
| catalog | the curated dataset built from the stages above | [catalog.md](docs/catalog.md) |
| seal | packs an allowlisted bundle of the catalog and encrypts it with age to the owner and the manifest's named recipients; the one way a dataset leaves the machine | [seal.md](docs/seal.md) |
`serve` answers queries over the index with mutual TLS, returning sanitized
chunks and catalog cards ([serve.md](docs/serve.md)). The gate refuses an
`serve` answers queries over the index with mutual TLS, returning chunks and
catalog cards from the sanitized text ([serve.md](docs/serve.md)). The gate refuses an
unsigned, edited or foreign manifest, and the bastion never starts the agent
without a valid one. (`--allow-unsigned` exists for development only; the
bastion never passes it.) See [runtime.md](docs/runtime.md) and
@@ -45,12 +47,20 @@ bastion never passes it.) See [runtime.md](docs/runtime.md) and
## How the bastion runs it
The bastion installs and runs the agent itself from a folder grant you
approved; nobody installs it by hand. For each shared folder the bastion:
approved; you do not install it by hand. (The `.pyz` can also be installed and
run by hand, to try it outside the bastion: [packaging.md](docs/packaging.md).)
For each shared folder the bastion:
- downloads the `.pyz` only through the signed manifest, and never replaces an
installed agent with an older version;
- mints the agent its own identity and writes and signs its manifest from the
grant, so the agent works only on what you granted;
- sets the sanitize stage to `flag` mode: rules only, no model. It replaces
e-mail addresses, phone numbers, card, bank and social security numbers, IP
and MAC addresses and keys. **It does not find names, street addresses or
dates of birth**, so text a Host receives from the agent still contains them.
`sterilize` mode adds a model on this machine that looks for those, and the
bastion does not set it ([sanitize.md](docs/sanitize.md#modes-manifest-sanitizemode));
- points the embed stage at the model server on this machine (a local ollama)
and at nothing else;
- runs `serve` on the loopback address only, with mutual TLS, so the bastion is
@@ -81,12 +91,16 @@ package, which is what the built artifact has always looked like inside.
## Build and test
```sh
pip install pyyaml==6.0.3 # the PyYAML build.py vendors; the manifest loader imports it
python3 -m unittest discover . # every module's tests, plus the 3.8 floor
python3 -m bastion_agent.build --out dist # produces dist/bastion-agent-<ver>.pyz + .sha256
python3 dist/bastion-agent-<ver>.pyz --version
```
Every push runs the same checks in CI (`.gitea/workflows/ci.yml`).
A test that needs `age`, `openssl` or `git` is skipped where that tool is not
installed. Every push runs the same checks in CI (`.gitea/workflows/ci.yml` on
git.privatae.ai, `.github/workflows/ci.yml` on GitHub), with the tools
installed, so there no test is skipped, plus `pyflakes`.
The build is reproducible: the same commit gives the same bytes on any machine
with PyYAML at the pinned version (`PYYAML_VERSION` in `build.py`).
@@ -113,8 +127,9 @@ it); no key or account of ours. It:
`bastion_agent/verify_release.py`, copied from `client-go/release_sign.go`
in the bastion's repository;
2. checks out the source of that version in a separate worktree: the tag
`v<version>`, or else the commit the manifest names as `built_from` if the
clone has it;
`v<version>`, which must be annotated with the line
`built from <built_from>`, or else the commit the manifest names as
`built_from` if the clone has it;
3. rebuilds the `.pyz` with that version's own `build.py`, and compares its
sha256 with every `.pyz` the manifest lists.
@@ -122,21 +137,38 @@ A match means the file every bastion downloads is exactly what this source
produces, and the signature means the release key holder published it. The
exit code is 0 only when both hold.
The public repository is a copy of each release's source, not the development
history, so `built_from` names a commit that exists only in the development
repository. From this release on, each public snapshot carries the tag
`v<version>`, which marks the source the release was built from. A snapshot
published before that has no tag, and can be checked from `built_from` only
where that commit is available. A manifest from before signing has no
From 0.1.16 on, each GitHub release carries the `.pyz`, its `.sha256`, the
signed manifest `version.json` and its signature `version.json.<h>.sig`. The
release workflow (`.github/workflows/release.yml`) attaches them when the tag
is pushed, after running this same check against the published manifest and
rebuilding the `.pyz` from the tag; it attaches nothing if a digest differs.
To check downloaded files yourself, `sha256sum -c bastion-agent-<ver>.pyz.sha256`
checks the `.pyz` against its `.sha256`, the manifest lists the sha256 of both,
and `check_signature` in `bastion_agent/verify_release.py` checks the manifest
against the pinned release keys:
```sh
python3 -c 'import sys; from bastion_agent import verify_release as v; print(v.check_signature(open(sys.argv[1], "rb").read(), open(sys.argv[2]).read())[0]["files"])' version.json version.json.*.sig
```
The public repository does not carry the development history, so
`built_from` names a commit that exists only in the development repository.
Each published release from 0.1.14 on carries the tag `v<version>`, which
marks the source the release was built from. Earlier releases have no tag
here, and can be checked from `built_from` only where that commit is
available. A manifest from before signing has no
signature file beside it, and the check refuses it.
## How this repo is published
Development happens in a private repository. This repository receives the
source of each published release as one snapshot commit, tagged
`v<version>`, whose tree is the source the signed manifest says the release
was built from; that is why its history is short. The same snapshot is on
GitHub at [github.com/Privatae/bastion-agent](https://github.com/Privatae/bastion-agent).
Development happens in a private repository, and its history is not
published. When a release is published, each change merged since the last one
arrives on GitHub, at
[github.com/Privatae/bastion-agent](https://github.com/Privatae/bastion-agent),
as a commit with the same files, titled as the change was and dated when it
was merged; the copy at git.privatae.ai receives the release as one commit.
In both, the release is tagged `v<version>` on a commit whose tree is the
source the signed manifest says the release was built from.
Every published `.pyz` can be rebuilt from its tag and compared
([Verify a release](#verify-a-release)).