bastion agent packaging: one file for three operating systems

The artifact

python3 -m bastion_agent.build --out dist → dist/bastion-agent-<version>.pyz and .sha256.

A zipapp, the standard library's own single-file format: one file of about 150 KB that runs on any Python 3.8 or later on Mac, Windows or Linux:

python3 bastion-agent-<version>.pyz doctor
python3 bastion-agent-<version>.pyz sweep manifest.yaml --once
python3 bastion-agent-<version>.pyz serve manifest.yaml

Inside: the bastion_agent package (every stage, the scheduler, the endpoint; no tests, and neither build.py nor verify_release.py, which are tools for making and checking a release), the pii package (the sterilizer; no tests, no Dockerfile), a vendored pure-Python PyYAML (MIT, its notice included; the C accelerator is left out so one file runs everywhere), which is the one import the manifest loader needs beyond the standard library, and the repository's LICENSE, so the license travels with the program. The version comes from bastion_agent/VERSION, read with pkgutil so it works from inside the zip.

bastion_agent/VERSION on main is the next release, not necessarily the one bastions run. A merged change waits there until it is published, and the published version is whatever https://privatae.ai/bastion-agent/version.json names at the moment: a bastion installs that version and no other. Between a merge and a publish, main is ahead of the manifest; when a second release is merged before the first is published, only the later one is published.

Reproducible. Fixed member timestamps, sorted members, no bytecode, and the vendored PyYAML pinned to one release (PYYAML_VERSION; the build refuses any other): two builds of the same tree are byte-identical on any machine, so the .sha256 is a real statement about the file, and anyone can rebuild a release and compare (python3 -m bastion_agent.verify_release, in the README). test_build asserts identical bytes across two builds.

age stays what it is, a single external binary, because the standard library cannot do X25519 or ChaCha20. The manifest gate does not need openssl: it reads the certificate and verifies the ECDSA P-256 signature in the standard library (x509gate.py), so the agent runs on a machine without it (Windows ships none). openssl is needed only to sign with sign --key and for SEAL_SIGN_KEY seal signatures. doctor reports both.

doctor

bastion-agent doctor [--json] is the preflight; it changes nothing:

Check	Used by	Required?
Python ≥ 3.8, PyYAML	everything	yes
`age`	seal	yes
`openssl`	`sign --key`, seal signatures	no: the gate verifies in the standard library
`pdftotext`, `tesseract`	extract	no: without them the gaps are recorded
`gliner` package	sanitize	no: the laptop recall backend
`EMBED_URL` (default `http://127.0.0.1:11434`), and `PII_ADDR` / `SUMMARIZE_URL` when set	embed / sanitize / catalog	no: reachable? Any HTTP answer counts, so a proxy that allows only the embedding path (and answers 403 on `/`) is reachable
`FREEAGENT_OWNER_RECIPIENT` set; `FREEAGENT_CA_ROOT`, `FREEAGENT_TLS_CERT`, `FREEAGENT_TLS_KEY` set and the files present	seal / sweep / serve	no, for the preflight: the stages that need them refuse without them

The exit code is 1 only when a required item is missing; optional gaps are printed as opt.

Installers (a hand install)

The bastion installs the agent itself from a grant; these are for running it by hand.

install.sh <pyz>: Linux and macOS, POSIX sh, no root, no downloads. It checks for python3 3.8 or later, verifies the .sha256 when it sits next to the .pyz (sha256sum or shasum), copies to ~/.local/bin/bastion-agent.pyz, writes a bastion-agent wrapper (and repoints an existing freeagent wrapper, so an earlier hand install keeps working), runs doctor, and prints the next step. BASTION_AGENT_BIN_DIR, or else FREEAGENT_BIN_DIR, overrides the directory.
install.ps1 <pyz>: Windows PowerShell, no administrator. The same checks with Get-FileHash; installs to %LOCALAPPDATA%\bastion-agent, writes bastion-agent.cmd, adds the directory to the user's PATH, and runs --version and doctor.

Neither installer fetches anything. Get the .pyz and its .sha256 from the download address, then run one command (Linux or macOS):

V=<version from https://privatae.ai/bastion-agent/version.json>
curl -fsSLO https://privatae.ai/bastion-agent/dl/bastion-agent-$V.pyz && \
curl -fsSLO https://privatae.ai/bastion-agent/dl/bastion-agent-$V.pyz.sha256 && \
curl -fsSL https://privatae.ai/bastion-agent/install.sh | sh -s -- bastion-agent-$V.pyz

The download address

bastion_agent/publish.sh publishes to every edge server named in its environment, each serving from its own disk, at /bastion-agent/*, with backups and dotfiles refused. The legacy /freeagent/* path is retired (see "The legacy name" below):

https://privatae.ai/bastion-agent/version.json          version, built_from, python_floor, files (url, sha256, bytes), installers, notes
https://privatae.ai/bastion-agent/version.json.<h>.sig  ed25519 signature; <h> = first 16 hex of version.json's sha256
https://privatae.ai/bastion-agent/dl/bastion-agent-<v>.pyz  the artifact
https://privatae.ai/bastion-agent/dl/bastion-agent-<v>.pyz.sha256
https://privatae.ai/bastion-agent/install.sh            and install.ps1

The manifest is signed with the bastion's release key, which stays on the key holder's own computer. A publish therefore runs twice: the first run builds and prints the release-sign sign bastion-agent <version> <sha256> command to run there, and the second, with SIGNATURE=<the line it printed> and BASTION_TREE pointing at a privatae-bastion checkout, checks the signature against the keys that checkout pins and the keys the published bastion pins, and only then publishes. What is signed is a short statement naming the kind of release, its version and the manifest's sha256. A bastion from 0.41.0 on fetches version.json.<h>.sig, refuses a manifest that does not verify, and never replaces an installed agent with an older version.

The script builds reproducibly, runs the built .pyz once (--version), refuses to publish a version that is not newer than the one already served (ALLOW_REPUBLISH=1 overrides it deliberately), uploads to every edge, then checks each edge's bytes against the manifest it serves. Where it publishes is not written into the script: the edges, the ssh key, the web root and the backup directory come from its environment, and it stops if one is missing.

The manifest's notes link to the tag v<version> on the public repository, which marks the source of that release. publish.sh does not make the tag; it is made when that release's source is published to the public repository.

The legacy name

The agent was first published as freeagent, at privatae.ai/freeagent/, and a bastion older than 0.20.0 knows only that address and the freeagent-<v>.pyz file name. Bastions from 0.20.0 read the new address first and fall back to the legacy one; from 0.42.0 they read only the new one.

The legacy address is retired: no edge has the /freeagent/* route, and https://privatae.ai/freeagent/version.json answers 404. A bastion that cannot fetch the manifest keeps the agent it has installed, so a bastion older than 0.20.0 keeps the agent it has and gets no newer one. What would strand a machine is the other case, a legacy manifest that answers but does not list the legacy file name, and a retired address cannot produce it.

publish.sh still checks on every run, on every edge (the route in the Caddy configuration each edge has loaded) and at the public address. With no route anywhere and a 404 in public it prints, per edge, "no /freeagent route" and publishes the new file name only. It publishes both names again, the same bytes under each, if any edge still has the route, if any edge cannot be asked (ssh fails or times out, or the configuration Caddy has loaded cannot be read), or if the public address answers anything but 404 (no answer included), so that address could not serve a manifest without the name an old bastion asks for.

The names inside the agent that keep the word freeagent (the FREEAGENT_* environment variables and the manifest's freeagent: 1 field) are stable names, not part of this retirement: the bastion sets and writes them, so renaming one would have to change the bastion and the agent in step, for no change in behaviour.

What this is not

A native binary. PyInstaller or Nuitka make one per OS, but each has to be built on that OS, and none is produced; it would be built from the same members() list. Until then the .pyz and a system Python are the artifact. Python 3.8 or later is on every Linux and current Mac (with the Xcode command line tools or Homebrew) and one Store install away on Windows, and the floor test keeps the code there.

Tests

python3 -m unittest bastion_agent.test_build covers byte-identical builds; the contents (the entry point, both packages, VERSION, the vendored YAML and its notice; no tests, no .so or .pyc, no build.py, verify_release.py or Dockerfile; every member at the fixed time; under 1 MB); the .pyz answering --version and doctor --json and validating the example manifest; a full sweep from the .pyz in a scratch directory with PYTHONPATH="" (nothing from the repository on the path) producing a catalog with sanitized heads; the .agent default output directory; and the doctor's checks against unreachable and missing inputs.

This repository has no CI. CONTRIBUTING.md lists the checks to run before proposing a change, and publish.sh builds the release and runs it before anything is uploaded. The installers are not exercised by the test suite.

9.8 KiB Raw Blame History