bastion 0.55.3

CHANGELOG.md

@@ -23,6 +23,338 @@ From 0.39.1 on, a heading marks a version *not published* or *held* once
 that is settled. The newest entries may not be published yet; the manifest
 says which version is. Older entries are not marked.
 
+## 0.55.3
+
+- **A pairing change reaches every Host on the machine at once.** Each pairing
+  beats its Host every 60 seconds, and the beat is how every Host on the
+  machine learns that a pairing went active or was revoked. Approving or
+  revoking a pairing changed it on disk immediately but triggered no beat, so
+  every Host kept the old state for up to a minute: a connection request to a
+  Host the anchor had just approved was refused as not on a shared machine. Now
+  a grant that makes a pairing active, and a revoke, make every running
+  pairing beat right away. A refused grant, or a re-sync of a pairing that is
+  already active, beats nothing.
+
+## 0.55.2
+
+- **A search reply says how its text was sanitized.** Every successful reply
+  from the bastion agent (search, read range, catalog) was stamped
+  `sanitized: true`, while the agent the bastion deploys runs in `flag` mode,
+  which replaces e-mail addresses, phone numbers, card, bank and ID numbers,
+  IP and MAC addresses and keys, and leaves names, street addresses and
+  dates of birth in. A reply now carries `sanitize_mode` (the mode this
+  bastion wrote into the agent's manifest, or for an agent run by hand the
+  mode it reports, else `unknown`), `sanitized` true only for `sterilize`,
+  and `sanitize_note` saying what the mode leaves in. A reply cannot raise
+  the mode this bastion wrote.
+- **The README says what the bastion needs.** It said "no Docker, no Python".
+  The bastion itself needs neither, and the README now names the optional
+  features that do: coder runs, the service slot and the memory mirror need a
+  container runtime, folder search needs Python 3.8 or later, Python ops need
+  `/usr/bin/python3`, and the Linux accessibility reader needs python3 with
+  AT-SPI. It also said Go 1.25; `client-go/go.mod` needs 1.26.
+- **GitHub releases carry their files.** A new workflow,
+  `.github/workflows/release.yml`, runs on each release tag (and by hand for
+  an earlier one): `deploy/release-assets.sh` rebuilds the release from its
+  tag, writes the release manifest from the rebuilt binaries with
+  `deploy/manifest.py` (which `publish.sh` now uses too, so the two cannot
+  differ), and fetches the signature published for that manifest. Only when
+  the signature verifies are the six binaries, `version.json`, its signature
+  and `SHA256SUMS` attached to the release. Releases 0.53.2, 0.54.0 and
+  0.54.1 were rebuilt this way and each matched its signed manifest.
+- **The lab harness is out of the release build.** `mode=lab` drove one test
+  lab's own cluster and database, and its layout was in the published source.
+  It is now built only with `-tags lab` and is not in the published source; a
+  release bastion answers a lab run with exit 126, as a bastion without a lab
+  always did.
+- **staticcheck is clean** on linux, darwin and windows, for both Go modules,
+  and CI runs it (`deploy/staticcheck.sh`, a pinned staticcheck release, with
+  the per-OS runs merged so code used on one OS only is not reported unused).
+  Unused code is removed and error strings no longer end in punctuation; no
+  behaviour changes.
+- **The claim tests a container cannot run now run on GitHub.** 25 tests the
+  claims table cites (the op and action-op sandboxes, the egress tests in
+  private namespaces, the root command watch) skipped in CI, because a CI
+  container has no systemd, no namespaces of its own and no nftables. A new
+  job runs every cited test as root on GitHub's discarded virtual machine and
+  fails if any skips; `deploy/check-claims.sh --skipped` names them.
+
+## 0.55.1
+
+- **A root op's tasks are capped.** An op's tasks were capped at 64 with
+  `RLIMIT_NPROC`, which the kernel never applies to root, so a root op could
+  start as many tasks as it liked until its time limit. A root op now runs
+  in a systemd scope of its own in `privatae_ops.slice` with `TasksMax=96`,
+  which counts every task in it, root's included: the op's 64 and the
+  bastion's two processes that start it. Those two run with one CPU's worth
+  of Go runtime threads (`GOMAXPROCS=1`), so they take the same share of the
+  cap on a machine of any size: with the runtime set for 64 CPUs they took
+  44 of the 96, and set for 256 no root op could start. The scope's process
+  reads its own cgroup's `pids.max` before the op starts, and the op's
+  report carries `tasks_capped` only when that is the cap. Where no scope
+  can be made (no systemd, no pids controller, or `systemd-run` failing),
+  the op runs as before and its report's `tasks_note` says its tasks were
+  not capped, and why. A user op's tasks are held by `RLIMIT_NPROC` as
+  before, and its report says so. A scope is killed with the run at the
+  time limit, when the bastion stops, and by the startup sweep. Two ops can
+  run at once, so an action op's memory watch now reads each other op's
+  unit or scope in the slice as another program: it yields when a root op
+  beside it makes the machine wait on memory, as it did when root ops ran
+  in the bastion's own cgroup, and its own unit still never counts.
+
+## 0.55.0
+
+- **A local machine's egress lane never reaches the machine itself.** Before,
+  on a machine with no anchor the lane vetted no destination, so a host could
+  dial this machine's own services through it: `POST
+  http://127.0.0.1:11434/api/pull` reached ollama whatever the `models` switch
+  said. It now refuses `localhost` names, loopback and unspecified addresses
+  (`0.0.0.0/8` and `::` included), link-local addresses (the cloud metadata
+  address included), the rest of `64:ff9b::/32`, the deprecated IPv6 forms
+  below, and any address this machine takes as its own, as the anchored lane
+  checks them, also inside an IPv4-mapped, NAT64 or 6to4 address. The LAN
+  stays reachable: a private, CGNAT or reserved address that is not this
+  machine's is dialled. Every address a name resolves to is checked, and only
+  those addresses are dialled, never the name again.
+- **A container or VM on this machine is this machine, not the LAN.** On
+  Linux, an address the kernel routes straight onto a container's veth, or
+  onto a bridge whose every port is a veth, a tap or a dummy (the bridge
+  networks Docker, Podman, LXD and libvirt make for their containers and
+  VMs), is refused as this machine. A route through a gateway on such a link
+  leaves, and a link an IPv4 or IPv6 default route of the main table goes
+  out of (any nexthop's, for a multipath default) is never inside: a bastion
+  that is itself in a container reaches the internet through the machine it
+  runs on, and dials the network on its own veth, its LAN included, as
+  before. Before, the local egress lane dialled such an address as LAN,
+  so a host could reach a service in a container here, `172.17.0.2:11434`
+  for one, though the same service on loopback was refused. A bridge with a
+  network card, a bond or a VLAN among its ports is still the LAN. Every lane
+  reads the same check, so a coder run's proxy, the anchored lane and the
+  backup lanes now also refuse a public address on such a network; their
+  private addresses were refused already. macOS and Windows have no route
+  lookup here, so there such an address is still dialled.
+- **Every lane refuses the deprecated IPv6 forms.** IPv4-compatible `::/96`,
+  IPv4-translated `::ffff:0:0/96` and site-local `fec0::/10` are no
+  destination of their own, and where each is delivered depends on the
+  platform. A coder run's proxy and the backup lanes already refused them; the
+  egress lane now does too, anchored or local, by address and in any answer a
+  name resolves to. The anchored lane refuses nothing else it did not refuse
+  before. All the lanes now read one address classifier.
+- **Both egress lanes try each vetted address, as Happy Eyeballs does.** A
+  name with more than one address is dialled at each in order, the next
+  starting as soon as an attempt fails or when none has answered within
+  300 ms, and the first to answer is used; the others are cancelled, and one
+  that connects anyway is closed. Before, the anchored lane dialled only the
+  first address, and a coder run's proxy gave each address an even share of
+  the dial budget, so an address whose packets go nowhere held up the next
+  one for seconds. The next one now starts within 300 ms, about what the OS
+  dialler took when it was given the name.
+- **`pair` says to enrol a second factor after every pairing that has none.**
+  It said so only after it anchored a networked machine, so a local pairing,
+  one over TCP or a unix socket, and a later host on an anchored machine were
+  told nothing, and on a local machine a folder grant then installs on the
+  host's signed grant alone. The line names `--host` when more than one host
+  is paired, and `sudo -H` only when `pair` itself ran with machine authority,
+  so a local install is told to enrol as its own user. A pairing that keeps
+  its factor (a `--replace`), or a machine with an approval device pinned, is
+  not told.
+- **A root-command grant lasts at most 30 days.** It was a year. `root-exec
+  grant --for 31d` is refused and says the longest term; grant again before
+  it ends. A grant given under the longer term ends 30 days after it was
+  given, and `root-exec` shows that end. A grant file with no grant time is
+  refused, since it cannot be held to the term.
+- **SECURITY.md, the README's license section and `docs/host-on-this-machine.md`
+  name each kind of Host.** They described only a Host on Privatae's platform.
+  They now say that a C-Series Host runs on your own machine in its own Linux
+  virtual machine, is open source (Privatae/cseries, AGPL-3.0-only), needs no
+  Privatae account, and that `npx cseries` sets the bastion up networked,
+  paired, anchored and with a second factor on every pairing (the commands are
+  shown); that C-Series Cloud and a network membership are optional and paid;
+  and that a Host on Privatae's own platform is the case where the Host's
+  software is not open source.
+- **Security reports go to security@privatae.ai,** the address
+  privatae.ai/.well-known/security.txt names. SECURITY.md said info@.
+- **`docs/models.md` says what stops a host changing your models on a local
+  machine.** It told you to scope the pairing on your host, which a local
+  machine never reads. There, switching `models` off for the host is what
+  filters the ollama forward, and the egress lane is no way around it. On an
+  anchored machine the forward filters for every pairing, which the page now
+  says too.
+- **A tunnel endpoint the host sends must be one host and port.** The
+  pairing response's `endpoint` and `endpoints`, and the endpoints in a
+  heartbeat's rotate directive, were written into the WireGuard
+  configuration unchecked. That configuration is one setting per line, so an
+  endpoint with a newline in it added settings of its own:
+  `203.0.113.7:51820` followed by a newline and `allowed_ip=::/0` let the
+  tunnel carry every IPv6 address, not only the host's. An endpoint must now
+  be an IP address or a host name with a port from 1 to 65535, and nothing
+  else, no space or control character. A pairing response with any other is
+  refused and nothing is saved, and a rotate directive with one is not acted
+  on. A WireGuard key must be 32 bytes. The tunnel's peer is now configured in
+  one place, which checks every value it writes.
+- **A pairing file that holds such an endpoint is listed but never served.**
+  An older build may have saved one. Its tunnel and lanes never come up, and
+  `run` names the file and says why. Every other pairing still loads and
+  serves, `status` shows the pairing as `invalid` with the same reason, and
+  `unpair` removes it, as does pairing that host again with `pair --replace`.
+  One such file no longer stops `run`, `status` and `unpair` for the whole
+  machine.
+- **Such a pairing cannot hold the anchor or be approved, and no host is told
+  of it.** `claim-anchor` marked it active and anchored the machine to it; it
+  now refuses, says why, and changes nothing. An approval from the anchor
+  host marked it active and started it with the endpoint its file holds; it
+  is now refused. The heartbeat and the anchor host's pairing list leave it
+  out: a host knows only the states active, pending and revoked, and could
+  show a pairing in any other state as active.
+- **A harness run's agent credential is readable only by the harness.** The
+  credential file was readable by every user on the machine (0644), in a
+  folder anyone could open and write (0777), for the length of the run. A
+  bastion running as root now hands the folder (0700) and the file (0600) to
+  the user the harness image runs its agent as (uid 10005). One that is not
+  root cannot hand them over, so it leaves them open for the agent, and the
+  folder sits inside one that only the bastion's own user can enter.
+- **A harness run's tree is out of other users' reach.** In a worktree run the
+  tree mounted at `/work` was opened to every user (0777), its top folder
+  included, in the shared temporary folder. Any local user could read or
+  change the Host's tree during the run. It now sits in a folder only the
+  bastion's own user can enter, as a git run's tree already did, and a run
+  refuses a tree whose folder other users can enter. Opening the tree for the
+  agent no longer follows a link in it. Before, on a bastion running as root,
+  a link in the repository, to `/etc/shadow` for example, made the linked
+  file world-writable. A file is opened as 0666, or 0777 if it was
+  executable, so a git run no longer commits every file as executable.
+- **A worktree run returns only what is in its tree.** A file swapped for a
+  link while the tree was being packed, or a folder swapped for a link, was
+  read through the link, so a bastion running as root could return any file
+  on the machine to the Host. Each name is now opened inside the folder
+  already open above it, never through a link, and goes in only if it is the
+  plain file or folder that was looked at. On Linux a device or pipe swapped
+  in for a file in that gap is not opened either: opening some devices does
+  something by itself. Links, pipes, devices and sockets are left out
+  (before, each came back as an empty entry of its kind), and the run's
+  result names them in `worktree_skipped`, with `worktree_skipped_count`.
+- **A coding agent can no longer run code as the bastion through the
+  repository's git settings.** In a git run the bastion committed and pushed
+  in the agent's own folder, `.git` included, so a hook, a file system
+  monitor, a filter, a signing program or a changed remote the agent set ran
+  as the bastion's own user, which can be root, or sent the push, with its
+  token, to another server. The bastion now makes its own fresh clone, in a
+  folder the agent cannot reach, copies in the agent's plain files and
+  folders (never a link, never anything of its `.git`), and commits and
+  pushes only there. The branch gets one commit with the agent's final
+  files; the agent's own commits are not pushed one by one. A link the
+  agent made is left out and named in `worktree_skipped`; a link the
+  repository already holds stays as it is. Every git the bastion runs for a
+  harness or a mirror runs with no hooks, no file system monitor, no `ext::`
+  transport, and no system or user git settings of the machine.
+- **The agent in a git run no longer holds the push token, and no process
+  argument shows it.** The agent's clone named the repository with the
+  run's token in it, so the agent could push straight to the repository
+  during the run, past the bastion's copy, and every git the bastion ran
+  for the run had the token in its arguments, which any local user can
+  read. Every git call now names the repository without the token and gets
+  it from its environment, as an `Authorization` header sent to that
+  repository's address only; no clone's config holds it, and a run is
+  refused if the token is in any file of the agent's clone's `.git`.
+- **A user-privilege op is no longer refused because other programs run as
+  its user.** An op's tasks are capped at 64, and the cap counted every task
+  the machine ran as the op's user (`privatae-session`, or `nobody`, which
+  system daemons share), so on a busy machine a user op could not start or
+  could not fork at all. The op now runs in a user namespace of its own,
+  as the same user and seeing every file's real owner, where the cap counts
+  only its own tasks. Where the machine has no user namespaces, the op runs
+  as before.
+- **A harness run's git calls carry the harness identity.** git made one
+  from this machine's user and host name instead, which cost a host name
+  lookup per call (seconds where there is no resolver) and wrote this
+  machine's name into the clone handed to the agent.
+- **A run whose agent container cannot be removed uses nothing of its
+  tree.** A failed removal was ignored, so an agent still running could go
+  on writing the tree while the bastion read it. The run now fails, says
+  so, and copies, returns and pushes nothing.
+- **A git run whose commit fails pushes nothing and fails.** A failed add or
+  commit was ignored and the branch pushed anyway, and the run read as
+  successful. It now ends the run with an error and no push. A run in which
+  the agent changed no file pushes no branch and reports `changed: false`.
+- **Repositories that keep files in Git LFS work in git runs again.** No
+  user git settings are read any more, so the LFS filter `git lfs install`
+  sets there was lost. When `git-lfs` is installed, the bastion now supplies
+  the standard LFS filter itself on its own git commands, sends the large
+  files before the push, and sends them only to the repository's own LFS
+  address, whatever a `.lfsconfig` the agent wrote names. A filter of any
+  other name in a `.gitattributes` still runs nothing. Tested with git-lfs
+  3.7.1.
+- **Names that open as `.git` on Windows or macOS are left out of a git
+  run's copy.** Besides `.git` in any letter case, the copy now leaves out
+  `.git` followed by dots or spaces or a `:`, the short name `GIT~1`, and
+  `.git` with the characters HFS+ ignores, the names git itself refuses, so
+  no file the agent made can be written into the clean clone's own `.git`.
+- **A run's tree has a size limit, and a file is never read into memory
+  whole.** A file of any size in the tree, even one that takes no disk
+  space, was read whole into memory, so an agent could make the bastion run
+  out of memory. A worktree run now returns at most 9 MiB of tree, which is
+  what fits in one message to the Host beside the transcript; a larger tree
+  is refused with `worktree_too_large` and none of it is returned. A git run
+  copies at most 2 GiB, and refuses a larger tree with `tree_too_large`
+  before it pushes anything; each file and folder counts 512 bytes and its
+  name besides its content, so a tree of millions of empty files is refused
+  too, and the copy keeps nothing per file in memory. Of the entries left
+  out, the count is kept in full and the first 100 names. A folder is read
+  512 names at a time, each counted as it is read, so a folder of millions
+  of names is refused without being read whole first. A result too large for one message to the Host
+  is refused with `result_too_large` instead of being lost on the way.
+- **The local console shows a pairing that is not served as "invalid".** A
+  pairing whose file holds an endpoint this build refuses showed as active
+  on the console when its stored state was empty, as it is in older files.
+  It now reads as the setup page and `status` read it.
+- **Built with Go 1.26.10, golang.org/x/net 0.61.0, x/crypto 0.58.0 and x/sys
+  0.49.0.** `govulncheck` finds 13 known vulnerabilities this code reaches in
+  Go 1.25.14 and x/net 0.57.0, which 0.54.1 was built with: in crypto/x509
+  (intermediate certificates since found compromised were still trusted),
+  crypto/tls, net/http and HTTP/2, net/textproto, os and html/template. With
+  these versions it finds none. The
+  build image is still pinned by digest, and CI runs in the same one. The
+  module now needs Go 1.26, as x/net's fixed release does. A published release
+  is still rebuilt by `deploy/verify-release.sh` with the toolchain its own
+  tag pins.
+- **`go test ./...` passes for a contributor who is not root.** A root
+  command now starts in `/` when the process cannot enter root's home, as it
+  already did when that home is missing; the bastion runs these commands only
+  as root, so on a machine nothing changes, and the root-command tests now run
+  as any user. `TestTreeCheckoutMovesOnlyForward`, which needs a tree root
+  owns, skips as another user, and `TestClaimAnchorWithNoToken` checks the
+  refusal without machine authority and then skips the rest, where it passed
+  having checked nothing more.
+- **The tests name no folder of our own build machine.** The sandbox tests'
+  fixture folder is `/srv/privatae-optest` (`/tmp` cannot hold an op's
+  effects), and the action tests' run note asks for any folder outside
+  `/tmp`.
+- **How this repository is published (docs only).** From this release on,
+  GitHub receives each change merged since the last release as its own commit,
+  titled as the change was, instead of one commit per release. The README,
+  CONTRIBUTING.md and SECURITY.md say so, and say that releases from 0.53.2 on
+  can be verified from a clone, since the earlier tags are not public.
+- **The personal-data check reads what it used to miss.** A listed term
+  written right after a backslash escape (`\n`, `\t`, `\u00e9`) is found, and
+  so is any value inside a run of base64 or base64url that decodes to text.
+  The reserved entity ids are read from `/etc/privatae/reserved-entity-ids`
+  (one per line, `RESERVED_ENTITY_IDS_FILE` to point elsewhere) instead of
+  being written in the script; where that file is absent, as in public CI,
+  the entity-id check is skipped and the run prints a NOTE, as the term check
+  already does without its key. A file that holds no id, or a line that is
+  not one, fails the run.
+- **The former-name check reads comments and Markdown.** A kept identifier
+  is set aside only in Go code; in a comment, this changelog or any other
+  file it is a hit like the bare word, and the two entries that spelled the
+  device-fingerprint prefix out now name `fpSANPrefix` instead.
+- **CI on GitHub.** `.github/workflows/ci.yml` runs the jobs of
+  `.gitea/workflows/ci.yml` on GitHub Actions in the same Go image, pinned by
+  the same digest, with no action from the marketplace and a token that can
+  only read. A step fails if its digest stops matching `deploy/build.sh`.
+- Tests use documentation addresses (`2001:db8::/32`, `203.0.113.0/24`) and
+  neutral paths, and comments state their reason instead of citing a pull
+  request by number; a test keeps published source free of such numbers.
+
 ## 0.54.1
 
 - **No change to what the bastion does.** The README is now a short front
@@ -225,13 +557,13 @@ says which version is. Older entries are not marked.
 - **The device fingerprint is sent under a second name to a Host with no
   platform.** A pairing with such a Host (`--redeem-url unix://…` or
   `https://…`) sends an operator certificate request that carries the
-  fingerprint as two URI SANs: first `urn:skema:hwfp:<fp>` (`fpSANPrefix`),
-  then `urn:privatae:hwfp:<fp>`, which that Host reads. A platform pairing's
-  request names it once, under `urn:skema:hwfp:`, as before: the platform
-  signs a request with that one name and refuses one with any other. A
-  signed accounting verifies with the fingerprint under either name, and a
-  signer certificate that names two different fingerprints is refused rather
-  than resolved.
+  fingerprint as two URI SANs: first under the original device-fingerprint
+  URN prefix (`fpSANPrefix` in pair.go), then `urn:privatae:hwfp:<fp>`, which
+  that Host reads. A platform pairing's request names it once, under
+  `fpSANPrefix`, as before: the platform signs a request with that one name
+  and refuses one with any other. A signed accounting verifies with the
+  fingerprint under either name, and a signer certificate that names two
+  different fingerprints is refused rather than resolved.
 
 ## 0.52.0