From 1441dabcca1e6b371172bf875b3b6b59d8aeaf69 Mon Sep 17 00:00:00 2001 From: Stephen Waits Date: Mon, 26 Oct 2026 19:32:07 -0600 Subject: [PATCH] feat(deploy): self-host recipe, the enforced isolation flag, and the tunnel's door (P9a) The books can live on a machine of the owner's own (plan 3.8, 8.2 P9a, D-51). Three things make that honest instead of nominal: - `--isolation same_user|enforced` sets `server::Isolation`, carried through `Served` and `BooksServer` to `books.start`. Locally the books stay `same_user (advisory)`; under the self-host unit the server is told `enforced` and reports `service_user (enforced)`. The server never guesses, and the literal it used to report is gone. - `--http [ADDR]` binds the same MCP router on loopback (default 127.0.0.1:7411) beside the Unix sockets, before the ready line, for the TLS proxy or tunnel in front of the server. It is the same bearer door and the same graceful close; a failed bind is reported and is not fatal (3.9). `serve` carries both flags through the host, so the system unit can ask for them once. `deploy/wanderbooks-server.service` is the unit: a dedicated account, a 0700 data root, `ProtectSystem=strict`, no seccomp or Landlock (D-32), and loopback-only networking. `deploy/README.md` is the recipe: install, setup, move the books by `books adopt` with its head check (D-31: a copied book is the transport), mint a bearer token, put Caddy or Tailscale Funnel in front of the loopback door, and point Claude Code and Codex at it. The two-account CI job (`.github/workflows/two-user.yml`) makes the deployment and runs `crates/ledger-cli/tests/two_user.rs` as another account: `agent_uid_cannot_read_tenant` proves the tenant root and its journal are refused, then reads, writes, uploads, files, exports and verifies over HTTP with the bearer (in a second book, so the moved books' head is not advanced); `self_host_adopt_same_head` compares the moved books' head and event count with the source clone's. `isolation_label_is_enforced_under_the_unit` runs everywhere, including the unit-file read that keeps the recipe from drifting. Accounting reason for the shape: nothing here changes an entry, a report or a figure. It changes who can reach them: the move is lossless and verifiable at the head, and the boundary is the operating system's, not a promise in our text. No journal code, kernel function, Kani harness or Lean step is touched, so `just prove` is not required. No go-ahead needed: no new journal feature or event; the live books are unaffected. --- .github/workflows/two-user.yml | 111 ++++++ crates/ledger-cli/tests/two_user.rs | 464 ++++++++++++++++++++++++ crates/ledger-host/src/commands.rs | 35 +- crates/ledger-host/src/host.rs | 29 +- crates/ledger-host/src/opener.rs | 24 +- crates/ledger-host/src/start.rs | 11 +- crates/ledger-mcp/handbook/self-host.md | 29 ++ crates/ledger-mcp/src/handbook.rs | 1 + crates/ledger-mcp/src/local.rs | 78 +++- crates/ledger-mcp/src/server.rs | 54 ++- crates/ledger-mcp/src/tools/kept.rs | 10 +- crates/ledger-mcp/src/types.rs | 2 +- deploy/README.md | 258 +++++++++++++ deploy/wanderbooks-server.service | 60 +++ docs/architecture.md | 32 ++ docs/mcp-interface.md | 13 +- docs/parity.md | 13 + docs/testing.md | 72 ++++ 18 files changed, 1269 insertions(+), 27 deletions(-) create mode 100644 .github/workflows/two-user.yml create mode 100644 crates/ledger-cli/tests/two_user.rs create mode 100644 crates/ledger-mcp/handbook/self-host.md create mode 100644 deploy/README.md create mode 100644 deploy/wanderbooks-server.service diff --git a/.github/workflows/two-user.yml b/.github/workflows/two-user.yml new file mode 100644 index 0000000..4620e80 --- /dev/null +++ b/.github/workflows/two-user.yml @@ -0,0 +1,111 @@ +# The two-account, self-host deployment (plan 8.2 P9a, D-51): the books +# server runs as one account, an agent as another, and the agent reaches +# the books only through the server's HTTP door with a static bearer +# token. `crates/ledger-cli/tests/two_user.rs` is the check; this file +# makes the deployment it needs and runs it against real accounts. A +# laptop's `cargo test` skips those two tests (WB_TWO_USER unset); this +# job sets it, so a skip here is a failure's cousin: the job only goes +# green when both tests ran. +# +# The token is minted with the test knob `file_keyring=1` (a file under +# the root's run/, 0600, ledger-host tokens.rs INVARIANT 3), because a +# runner has no Secret Service; deploy/README.md says what a real server +# does instead. +name: two-user +on: + push: + pull_request: + workflow_dispatch: +# A newer push makes an older run pointless: it is cancelled. +concurrency: + group: two-user-${{ github.ref }} + cancel-in-progress: true +jobs: + two-user: + runs-on: ubuntu-24.04 + timeout-minutes: 45 + env: + WB_PORT: "17411" + WB: /usr/local/bin/wanderbooks + steps: + - uses: actions/checkout@v4 + - uses: jdx/mise-action@v2 + - name: Build the server-only executable and the test + run: | + cargo build --release -p ledger-cli --no-default-features + cargo test -p ledger-cli --test two_user --no-run + - name: Two accounts, one data root only one can read + run: | + set -euo pipefail + sudo useradd --system --create-home --home-dir /var/lib/wanderbooks \ + --shell /usr/sbin/nologin wanderbooks + sudo install -d -o wanderbooks -g wanderbooks -m 0700 /srv/wanderbooks + sudo install -d -o wanderbooks -g wanderbooks -m 0700 /srv/desktop + sudo install -d -o wanderbooks -g wanderbooks -m 0700 /srv/desktop-home + sudo install -m 0755 target/release/wanderbooks "$WB" + - name: The deployment, the move, and the check + run: | + set -euo pipefail + as_books() { sudo -u wanderbooks -H env HOME=/var/lib/wanderbooks "$@"; } + wait_open() { + for _ in $(seq 1 240); do + grep -q "The books are open" "$1" && return 0 + sleep 0.5 + done + echo "the server did not open:" >&2 + cat "$1" >&2 + return 1 + } + stop_servers() { + as_books "$WB" close --root /srv/desktop >/dev/null 2>&1 || true + as_books "$WB" close --root /srv/wanderbooks >/dev/null 2>&1 || true + } + trap stop_servers EXIT + + # The machine the books come from: books of this version, made by + # adopting the repository's fixture (so the move changes no event). + # Its own home, so the two roots are not "somewhere else" to one + # another (root::setup records one root per account home). + as_books "$WB" setup --root /srv/desktop --home /srv/desktop-home + as_books "$WB" serve --root /srv/desktop > /tmp/desktop.log 2>&1 & + wait_open /tmp/desktop.log + sudo cp -a crates/ledger-cli/tests/fixtures/p2-served /tmp/fixture + as_books "$WB" books adopt /tmp/fixture --root /srv/desktop --id acme --yes + as_books "$WB" close --root /srv/desktop + + # The backup clone that is the transport (plan D-31): readable by + # the agent account, because the test reads its journal to know + # the head the move must keep. + sudo cp -a /srv/desktop/books/acme /tmp/source + sudo chown -R "$(id -u):$(id -g)" /tmp/source + sudo chmod -R a+rX /tmp/source + + # The server's machine: the same books, moved, behind `enforced`. + as_books "$WB" setup --root /srv/wanderbooks + as_books "$WB" serve --root /srv/wanderbooks --isolation enforced \ + --http "127.0.0.1:$WB_PORT" > /tmp/tenant.log 2>&1 & + wait_open /tmp/tenant.log + as_books "$WB" books adopt /tmp/source --root /srv/wanderbooks --id acme --yes + as_books "$WB" books create tunnel --root /srv/wanderbooks + + # Tokens for the agent account (the test), kept in the root's + # run/ by the test knob; a real server uses its OS keyring. + as_books "$WB" token create --root /srv/wanderbooks --book acme --name ci \ + --test-knobs file_keyring=1 >/dev/null + TOKEN=$(as_books "$WB" token print --root /srv/wanderbooks --book acme \ + --name ci --test-knobs file_keyring=1) + as_books "$WB" token create --root /srv/wanderbooks --book tunnel --name ci \ + --test-knobs file_keyring=1 >/dev/null + WRITE_TOKEN=$(as_books "$WB" token print --root /srv/wanderbooks --book tunnel \ + --name ci --test-knobs file_keyring=1) + + # The check, as the agent account, over the loopback door. + WB_TWO_USER=1 \ + WB_TENANT_ROOT=/srv/wanderbooks \ + WB_SERVER_URL="http://127.0.0.1:$WB_PORT" \ + WB_BOOK=acme \ + WB_TOKEN="$TOKEN" \ + WB_WRITE_BOOK=tunnel \ + WB_WRITE_TOKEN="$WRITE_TOKEN" \ + WB_SOURCE_DIR=/tmp/source \ + cargo test -p ledger-cli --test two_user -- --nocapture --test-threads=1 diff --git a/crates/ledger-cli/tests/two_user.rs b/crates/ledger-cli/tests/two_user.rs new file mode 100644 index 0000000..115ae96 --- /dev/null +++ b/crates/ledger-cli/tests/two_user.rs @@ -0,0 +1,464 @@ +//! The two-account self-host deployment of the books (plan 8.2 P9a, +//! D-51): the server runs as one account on the owner's own machine and +//! an agent runs as another, reaching the books over HTTP with a static +//! bearer token. +//! +//! ```text +//! GOAL: two things the plan promises are checked, not assumed: an agent +//! with its own account cannot read the data root, and it reaches +//! every tool through the server's HTTP door all the same; and the +//! books moved to that machine verify to the head they had. +//! SOURCE: plan 8.2 P9a (Build items 1 and 2; Done when 1), 2 I-1 +//! (isolation: advisory locally, enforced on a machine of the +//! owner's), 3.3 (the static bearer), 3.8 (local vs self-host), +//! 3.9 (the server's sockets, and `--http`), D-51 (P9 split), +//! D-31 (a backup clone is the move's transport). +//! INVARIANTS: +//! 1. The two-account tests run only against a deployment the caller +//! made (the job in `.github/workflows/two-user.yml`, or the recipe +//! in `deploy/README.md`); with no deployment they say they skipped, +//! so a laptop's `cargo test` stays green. `WB_TWO_USER=1` turns +//! them on, and then every other setting is required. +//! 2. The test never reads the books' files in the tenant: it proves it +//! cannot (INVARIANT 3), then asks the server for everything. +//! 3. The tenant root is unreadable to this account: `opendir` and a +//! journal `open` are refused. +//! 4. The books say `isolation: "service_user (enforced)"`: the server +//! was told `--isolation enforced` by the unit or the job, never +//! guessed. +//! 5. A read, a write, an upload and an export go over the tunnel's +//! HTTP door with the bearer; the local Unix socket is not used. +//! They go to a second book (`WB_WRITE_BOOK`), so the moved books' +//! head is never advanced by a check. +//! 6. The moved books' head (seq and hash) and every event are the +//! source's: the move loses nothing. +//! ``` + +use common::{BOOK, Client, Home, Wire}; +use serde_json::{Value, json}; +use std::path::{Path, PathBuf}; +use std::process::{Child, Command, Stdio}; +use std::time::{Duration, Instant}; + +mod common; + +// ---------------------------------------------------------------- harness + +/// The deployment the caller made, from the environment +/// (INVARIANTS 1 to 3). `None` when `WB_TWO_USER` is unset. +struct Deployment { + /// The data root this account must not be able to read. + root: PathBuf, + /// `http://host:port`, the tunnel's door's base. + server: String, + book: String, + token: String, + /// A second book the write checks go to, so the moved books' head is + /// never advanced by a check (defaults to `book`). + write_book: String, + write_token: String, + /// The backup clone the books were moved from, readable here. + source: Option, +} + +fn deployment() -> Option { + std::env::var_os("WB_TWO_USER")?; + let need = |name: &str| -> String { + std::env::var(name).unwrap_or_else(|_| panic!("WB_TWO_USER is set, so {name} must be too")) + }; + let book = need("WB_BOOK"); + let token = need("WB_TOKEN"); + let write_book = std::env::var("WB_WRITE_BOOK").unwrap_or_else(|_| book.clone()); + let write_token = std::env::var("WB_WRITE_TOKEN").unwrap_or_else(|_| token.clone()); + Some(Deployment { + root: PathBuf::from(need("WB_TENANT_ROOT")), + server: need("WB_SERVER_URL"), + book, + token, + write_book, + write_token, + source: std::env::var_os("WB_SOURCE_DIR").map(PathBuf::from), + }) +} + +impl Deployment { + /// The MCP URL of `book`. + fn mcp_url(&self, book: &str) -> String { + format!("{}/b/{}/mcp", self.server, book) + } + + /// A URL under `/b/{book}/`. + fn book_url(&self, book: &str, rest: &str) -> String { + format!("{}/b/{}/{}", self.server, book, rest) + } +} + +/// A client of the books over the tunnel's HTTP door (INVARIANT 5), with +/// the static bearer token. +async fn connect_over_http(d: &Deployment, book: &str, token: &str) -> Client { + use rmcp::ServiceExt; + use rmcp::transport::StreamableHttpClientTransport; + use rmcp::transport::streamable_http_client::StreamableHttpClientTransportConfig; + let config = StreamableHttpClientTransportConfig::with_uri(d.mcp_url(book)) + .auth_header(token.to_string()); + let http = reqwest::Client::new(); + let transport = StreamableHttpClientTransport::with_client(http, config); + rmcp::model::ClientConfig::default() + .serve(transport) + .await + .unwrap_or_else(|e| panic!("the tunnel's door refused the session: {e}")) +} + +/// GET `url` with the bearer: the status and the bytes. +async fn get_with_token(url: &str, token: &str) -> (u16, Vec) { + let r = reqwest::Client::new() + .get(url) + .bearer_auth(token) + .send() + .await + .unwrap(); + let status = r.status().as_u16(); + (status, r.bytes().await.unwrap_or_default().to_vec()) +} + +/// A running `wanderbooks server` on `home`'s root with `args` beside the +/// root, ended (by its own PID) when dropped. The harness's `Server` is +/// the same without extra flags; this test needs `--isolation` and +/// `--http`. +struct FlaggedServer { + child: Child, +} + +impl FlaggedServer { + fn start(home: &Home, args: &[String]) -> Self { + let log = std::fs::OpenOptions::new() + .create(true) + .append(true) + .open(home.dir.path().join("server.log")) + .unwrap(); + let mut child = Command::new(env!("CARGO_BIN_EXE_wanderbooks")) + .arg("server") + .arg("--root") + .arg(home.root()) + .args(args) + .env_clear() + .env("PATH", "/usr/bin:/bin") + .env("HOME", home.dir.path()) + .env("XDG_CONFIG_HOME", home.dir.path().join(".config")) + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .stderr(log) + .spawn() + .unwrap(); + let mut line = String::new(); + std::io::BufRead::read_line( + &mut std::io::BufReader::new(child.stdout.as_mut().unwrap()), + &mut line, + ) + .unwrap(); + assert_eq!(line, "ready\n", "the server didn't say ready"); + Self { child } + } + + /// Close it as the host does (stdin ends) and wait for it to exit. + fn close(mut self) -> std::process::ExitStatus { + drop(self.child.stdin.take()); + let deadline = Instant::now() + Duration::from_secs(60); + loop { + if let Some(status) = self.child.try_wait().unwrap() { + return status; + } + assert!(Instant::now() < deadline, "the server didn't close"); + std::thread::sleep(Duration::from_millis(20)); + } + } +} + +impl Drop for FlaggedServer { + fn drop(&mut self) { + let _ = self.child.kill(); + let _ = self.child.wait(); + } +} + +/// A port nothing is listening on now, for the tunnel's door. +fn free_port() -> u16 { + let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap(); + listener.local_addr().unwrap().port() +} + +/// `books.start` over a wire, as the tools' answer. +fn start_of(wire: &mut Wire) -> Value { + let r = wire.call("books.start", json!({})); + common::answer_of(&r).unwrap_or_else(|e| panic!("books.start: {e}")) +} + +// ---------------------------------------------------------------- the tests + +/// The server told `--isolation enforced` says so (`books.start`), over +/// its Unix socket and over the HTTP door, and a server not told says the +/// local truth: the server never guesses (plan 3.8, P9a). +#[test] +fn isolation_label_is_enforced_under_the_unit() { + // The local default first: no flag, the advisory truth (D-32). + let local = Home::new("isolation-local"); + let server = FlaggedServer::start(&local, &[]); + local.create(BOOK, "USD"); + { + let mut wire = Wire::new(&local, BOOK); + wire.initialize("self-host-check"); + assert_eq!(start_of(&mut wire)["isolation"], "same_user (advisory)"); + } + assert!(server.close().success()); + + // Told enforced: `books.start` says so, on both doors (INVARIANT 4). + let home = Home::new("isolation-enforced"); + let port = free_port(); + let server = FlaggedServer::start( + &home, + &[ + "--isolation".into(), + "enforced".into(), + "--http".into(), + format!("127.0.0.1:{port}"), + ], + ); + home.create(BOOK, "USD"); + let over_socket = { + let mut wire = Wire::new(&home, BOOK); + wire.initialize("self-host-check"); + start_of(&mut wire) + }; + assert_eq!(over_socket["isolation"], "service_user (enforced)"); + // The HTTP door is the same service with the same bearer token. + let d = Deployment { + root: home.root(), + server: format!("http://127.0.0.1:{port}"), + book: BOOK.into(), + token: home.token(BOOK), + write_book: BOOK.into(), + write_token: home.token(BOOK), + source: None, + }; + let rt = tokio::runtime::Runtime::new().unwrap(); + let over_http = rt.block_on(async { + let client = connect_over_http(&d, &d.book, &d.token).await; + let answer = common::call(&client, "books.start", json!({})).await; + let _ = client.cancel().await; + answer + }); + let over_http = over_http.expect("books.start over HTTP"); + assert_eq!(over_http["isolation"], "service_user (enforced)"); + assert_eq!(over_http["book"]["name"], over_socket["book"]["name"]); + assert_eq!(over_http["head"], over_socket["head"]); + assert!(server.close().success()); + + // The unit's own line is what starts a server that way (P9a step 1): + // it passes the flag this test exercises, and the tunnel's door. + let unit = std::fs::read_to_string(concat!( + env!("CARGO_MANIFEST_DIR"), + "/../../deploy/wanderbooks-server.service" + )) + .expect("deploy/wanderbooks-server.service"); + for line in [ + "User=wanderbooks", + "--isolation enforced", + "--http 127.0.0.1:7411", + "IPAddressDeny=any", + "IPAddressAllow=localhost", + ] { + assert!(unit.contains(line), "the unit lacks {line:?}:\n{unit}"); + } +} + +/// An agent with its own account cannot read the books' data root, and +/// every tool it needs still answers over the tunnel's HTTP door with the +/// bearer (INVARIANTS 2, 3, 4 and 5). +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn agent_uid_cannot_read_tenant() { + let Some(d) = deployment() else { + eprintln!( + "SKIPPED agent_uid_cannot_read_tenant: no two-account deployment (WB_TWO_USER unset; `.github/workflows/two-user.yml` or deploy/README.md makes one)" + ); + return; + }; + // INVARIANT 3: the root, and a journal inside it, are unreadable. + match std::fs::read_dir(&d.root) { + Err(e) => assert_eq!( + e.kind(), + std::io::ErrorKind::PermissionDenied, + "the tenant root is not refused for the wrong reason: {e}" + ), + Ok(_) => panic!( + "this account can read the tenant root ({}): the deployment did not make it private", + d.root.display() + ), + } + let journal = d.root.join("books").join(&d.book).join("journal.jsonl"); + match std::fs::File::open(&journal) { + Err(e) => assert_eq!( + e.kind(), + std::io::ErrorKind::PermissionDenied, + "the journal is not refused for the wrong reason: {e}" + ), + Ok(_) => panic!("this account can read the books' journal"), + } + + // INVARIANT 4: the books were told enforced, and say so. + let books = connect_over_http(&d, &d.book, &d.token).await; + let start = common::call(&books, "books.start", json!({})) + .await + .expect("books.start over the tunnel"); + assert_eq!(start["isolation"], "service_user (enforced)"); + assert!( + !start.to_string().contains(&d.root.display().to_string()), + "the answer names the data root" + ); + + // The write checks go to a second book, so the moved books' head is + // never advanced by a check (`self_host_adopt_same_head` reads it). + let client = connect_over_http(&d, &d.write_book, &d.write_token).await; + let write_start = common::call(&client, "books.start", json!({})) + .await + .expect("books.start over the tunnel"); + assert_eq!(write_start["isolation"], "service_user (enforced)"); + + // A read. + let chart = common::call(&client, "books.chart", json!({})) + .await + .expect("books.chart over the tunnel"); + assert!(chart["accounts"].is_array(), "{chart}"); + + // A write: an account added over the tunnel, then read back. + let code = "1999"; + let added = common::call( + &client, + "books.add_account", + json!({"code": code, "name": "Over the tunnel", "account_type": "asset"}), + ) + .await + .expect("books.add_account over the tunnel"); + assert!(added["account_id"].is_number(), "{added}"); + let chart = common::call(&client, "books.chart", json!({})) + .await + .expect("books.chart over the tunnel"); + assert!( + chart["accounts"] + .as_array() + .unwrap() + .iter() + .any(|a| a["code"] == code), + "the account written over the tunnel is not in the chart" + ); + + // A file in: an upload through the tunnel's own route. + let bytes = b"a receipt filed over the tunnel"; + let begun = common::call( + &client, + "books.begin_upload", + common::upload_args(bytes, "tunnel.txt", "text/plain", "receipt"), + ) + .await + .expect("books.begin_upload over the tunnel"); + let upload_id = begun["upload_id"] + .as_str() + .unwrap_or_else(|| panic!("no upload begun: {begun}")); + let sent = reqwest::Client::new() + .put(d.book_url(&d.write_book, &format!("u/{upload_id}"))) + .bearer_auth(&d.write_token) + .body(bytes.to_vec()) + .send() + .await + .unwrap(); + assert_eq!(sent.status().as_u16(), 200, "the upload route refused it"); + // And file it, so it is a stored, named attachment the books verify. + let filed = common::call( + &client, + "books.file_document", + json!({"category": "other", "upload_id": upload_id, + "file_name": "tunnel.txt", "mime_type": "text/plain", + "description": "filed over the tunnel"}), + ) + .await + .expect("books.file_document over the tunnel"); + assert!(filed["attachment_id"].is_number(), "{filed}"); + + // A file out: an export handed over, fetched through the tunnel. + let exported = common::call( + &client, + "books.export", + json!({"format": "csv", "report": "trial_balance"}), + ) + .await + .expect("books.export over the tunnel"); + let file = &exported["file"]; + let command = file["download"].as_str().expect("a download command"); + let id = command + .split(' ') + .find(|w| w.starts_with("ex_")) + .unwrap_or_else(|| panic!("no export id in {command}")); + let (status, fetched) = get_with_token( + &d.book_url(&d.write_book, &format!("x/{id}")), + &d.write_token, + ) + .await; + assert_eq!(status, 200, "the export route refused it"); + assert_eq!(file["size_bytes"], json!(fetched.len())); + assert_eq!( + file["sha256"], + json!(ledger_store::journal::sha256_hex(&fetched)) + ); + + // And the books verify, stored files included: the upload landed. + let verified = common::call(&client, "books.verify", json!({"blobs": true})) + .await + .expect("books.verify over the tunnel"); + assert_eq!(verified["status"], "ok"); + assert!(verified["blobs_verified"].as_u64().unwrap_or(0) >= 1); + + let _ = client.cancel().await; + let _ = books.cancel().await; +} + +/// The books moved to the server verify to the head the source clone had: +/// the same events, the same seq and hash (INVARIANTS 2 and 6). +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn self_host_adopt_same_head() { + let Some(d) = deployment() else { + eprintln!( + "SKIPPED self_host_adopt_same_head: no two-account deployment (WB_TWO_USER unset)" + ); + return; + }; + let Some(source) = d.source.as_deref() else { + eprintln!("SKIPPED self_host_adopt_same_head: WB_SOURCE_DIR is not set"); + return; + }; + let (seq, hash, events) = source_head(source); + let client = connect_over_http(&d, &d.book, &d.token).await; + let start = common::call(&client, "books.start", json!({})) + .await + .expect("books.start over the tunnel"); + assert_eq!(start["head"]["seq"], seq, "the moved books' seq"); + assert_eq!(start["head"]["hash"], hash, "the moved books' head hash"); + let verified = common::call(&client, "books.verify", json!({})) + .await + .expect("books.verify over the tunnel"); + assert_eq!(verified["status"], "ok"); + assert_eq!(verified["events"], events, "every event moved"); + assert_eq!(verified["head"], hash, "the verified head"); + let _ = client.cancel().await; +} + +/// The source clone's head and event count, read as the move's transport +/// is read (plan D-31): the journal's last line's seq and canonical hash. +fn source_head(source: &Path) -> (u64, String, u64) { + let lines = ledger_store::journal::read_and_verify_journal(&source.join("journal.jsonl")) + .unwrap_or_else(|e| panic!("the source clone doesn't verify: {e}")); + let seq = lines.last().map_or(0, |l| l.seq); + let hash = lines.last().map_or_else( + || ledger_store::journal::GENESIS_PREV.to_string(), + |l| ledger_store::journal::sha256_hex(l.to_canonical().as_bytes()), + ); + (seq, hash, lines.len() as u64) +} diff --git a/crates/ledger-host/src/commands.rs b/crates/ledger-host/src/commands.rs index 466daab..5f2e155 100644 --- a/crates/ledger-host/src/commands.rs +++ b/crates/ledger-host/src/commands.rs @@ -231,16 +231,44 @@ pub fn serve( args: impl IntoIterator, ) -> Result<(), Box> { let mut common = Common::default(); - let mut args = args.into_iter(); + let mut server_flags: Vec = Vec::new(); + let mut args = args.into_iter().peekable(); while let Some(arg) = args.next() { if common.take(&arg, &mut args)? { continue; } match arg.as_str() { + // What this server is told (P9a): passed through the host. + "--isolation" => { + let level = args + .next() + .ok_or("--isolation needs same_user or enforced")?; + if !matches!( + level.as_str(), + "same_user" | "advisory" | "service_user" | "enforced" + ) { + eprintln!("error: --isolation takes same_user or enforced, not {level:?}"); + return Err(exit(2)); + } + server_flags.push(arg); + server_flags.push(level); + } + "--http" => { + server_flags.push(arg); + if let Some(v) = args.peek() + && !v.starts_with('-') + { + server_flags.push(args.next().unwrap_or_default()); + } + } "--help" | "-h" => { - eprintln!("usage: {prog}"); + eprintln!("usage: {prog} [--root DIR] [--isolation LEVEL] [--http [ADDR]]"); eprintln!(" Opens the books and holds them open until Ctrl-C. They close 10 s"); eprintln!(" after nothing else (the app, an agent session) holds them."); + eprintln!(" --isolation same_user | enforced: how books.start says these books"); + eprintln!(" are kept from agents (enforced: the owner's own machine, P9a)."); + eprintln!(" --http [ADDR]: also serve the MCP door on 127.0.0.1:7411 (or ADDR),"); + eprintln!(" for a TLS proxy or a tunnel in front of it."); return Ok(()); } other => { @@ -259,7 +287,8 @@ pub fn serve( common.knobs, Kind::Serve, None, - )?; + )? + .starting_with(server_flags); // Ctrl-C, SIGTERM and SIGHUP are heard from the start, whatever // serve is doing (opening, waiting to try again): none is missed. let stop = std::sync::Arc::new(tokio::sync::Notify::new()); diff --git a/crates/ledger-host/src/host.rs b/crates/ledger-host/src/host.rs index 132b29e..7921311 100644 --- a/crates/ledger-host/src/host.rs +++ b/crates/ledger-host/src/host.rs @@ -244,16 +244,32 @@ struct Args { /// server finds the root itself, and no command line names it. root: Option, knobs: Knobs, + /// Flags the server is started with (P9a): `--isolation`, `--http`. + server_flags: Vec, } fn parse_args(prog: &str, args: impl IntoIterator) -> Result, String> { let mut root = None; let mut knobs = Knobs::default(); - let mut args = args.into_iter(); + let mut server_flags: Vec = Vec::new(); + let mut args = args.into_iter().peekable(); while let Some(arg) = args.next() { match arg.as_str() { "--root" => root = Some(PathBuf::from(args.next().ok_or("--root needs a folder")?)), FLAG => knobs = Knobs::parse(&args.next().ok_or("--test-knobs needs a value")?)?, + // Passed through to the server (P9a). + "--isolation" => { + server_flags.push(arg); + server_flags.push(args.next().ok_or("--isolation needs a level")?); + } + "--http" => { + server_flags.push(arg); + if let Some(v) = args.peek() + && !v.starts_with('-') + { + server_flags.push(args.next().unwrap_or_default()); + } + } "--help" | "-h" => { eprintln!("usage: {prog} [--root DIR]"); eprintln!(" Holds the books open while anything holds a lease on it; started by"); @@ -263,7 +279,11 @@ fn parse_args(prog: &str, args: impl IntoIterator) -> Result return Err(format!("unknown argument: {other}")), } } - Ok(Some(Args { root, knobs })) + Ok(Some(Args { + root, + knobs, + server_flags, + })) } /// `wanderbooks host`. @@ -395,6 +415,8 @@ struct Host { root: Root, run: PathBuf, knobs: Knobs, + /// Flags the server is started with (P9a). + server_flags: Vec, leases: BTreeMap, next_lease: u64, ready: bool, @@ -417,6 +439,7 @@ impl Host { root, run, knobs: args.knobs, + server_flags: args.server_flags, leases: BTreeMap::new(), next_lease: 1, ready: false, @@ -655,6 +678,8 @@ impl Host { if self.explicit_root { command.arg("--root").arg(self.root.path()); } + // What `serve` was told, for the server (P9a). + command.args(&self.server_flags); let d = Knobs::default(); for (flag, now, default) in [ ("--backup-quiet", self.knobs.backup_quiet, d.backup_quiet), diff --git a/crates/ledger-host/src/opener.rs b/crates/ledger-host/src/opener.rs index 65b872e..53eba6e 100644 --- a/crates/ledger-host/src/opener.rs +++ b/crates/ledger-host/src/opener.rs @@ -100,6 +100,9 @@ pub struct Opener { opened_at: Option, /// Whether this holder has held a lease (INVARIANT 5). has_held: bool, + /// Flags the server this opener starts is started with (P9a): + /// `--isolation` and `--http`. Empty for every ordinary holder. + server_flags: Vec, } /// A held lease: the books stay open while it lives. @@ -146,9 +149,20 @@ impl Opener { failures: 0, opened_at: None, has_held: false, + server_flags: Vec::new(), }) } + /// Start the server this opener brings up with these flags (P9a: + /// `--isolation`, `--http`). Only `serve` uses it: on the owner's own + /// machine it is the one holder, so the host it starts is the server's + /// only one. A host already running was started with other flags, and + /// this one has no way to change it. + pub fn starting_with(mut self, flags: Vec) -> Self { + self.server_flags = flags; + self + } + /// The timings in force. pub fn knobs(&self) -> Knobs { self.knobs @@ -256,9 +270,12 @@ impl Opener { async fn start(&self) -> Result<(), String> { let root = self.root.clone(); let knobs = self.knobs; - tokio::task::spawn_blocking(move || crate::start::start_host(root.as_deref(), &knobs)) - .await - .map_err(|e| e.to_string())? + let flags = self.server_flags.clone(); + tokio::task::spawn_blocking(move || { + crate::start::start_host(root.as_deref(), &knobs, &flags) + }) + .await + .map_err(|e| e.to_string())? } /// Say hello on a connection and read the host's first word. @@ -457,6 +474,7 @@ mod tests { failures: 0, opened_at: None, has_held: false, + server_flags: Vec::new(), }; for _ in 0..MAX_TRIES { o.failed().await.unwrap(); diff --git a/crates/ledger-host/src/start.rs b/crates/ledger-host/src/start.rs index 855dc32..47f7bae 100644 --- a/crates/ledger-host/src/start.rs +++ b/crates/ledger-host/src/start.rs @@ -174,8 +174,12 @@ fn systemd_run() -> Option<&'static str> { } /// Start a host for the root (`root`, else the one the host finds -/// itself), with these knobs (INVARIANTS 1-5). -pub fn start_host(root: Option<&Path>, knobs: &Knobs) -> Result<(), String> { +/// itself), with these knobs and these server flags (INVARIANTS 1-5). +pub fn start_host( + root: Option<&Path>, + knobs: &Knobs, + server_flags: &[String], +) -> Result<(), String> { let found = ledger_store::root::Root::find(root).map_err(|e| e.to_string())?; let run = found.run_dir().map_err(|e| e.to_string())?; let home = ledger_store::root::account_home().map_err(|e| e.to_string())?; @@ -187,6 +191,9 @@ pub fn start_host(root: Option<&Path>, knobs: &Knobs) -> Result<(), String> { args.push(found.path().as_os_str().to_os_string()); } args.extend(knobs.args().into_iter().map(Into::into)); + // What the server this host starts is told (P9a): the same flags + // `serve` was given. + args.extend(server_flags.iter().map(Into::into)); let log = || { let path = run.join(HOST_LOG); // A large log is moved aside first (to `.1`), never from under a diff --git a/crates/ledger-mcp/handbook/self-host.md b/crates/ledger-mcp/handbook/self-host.md new file mode 100644 index 0000000..09a8b8d --- /dev/null +++ b/crates/ledger-mcp/handbook/self-host.md @@ -0,0 +1,29 @@ +# On a machine of the owner's own + +These books may live on a machine the owner controls, not on the computer +an agent runs on. The server there runs as a separate account, and the +agent has no account on that machine: `books.start` says +`isolation: "service_user (enforced)"`. Locally it says +`same_user (advisory)`, which is honest: hidden, not locked. + +What that means for you: + +- Reach the books only through these tools. The machine they live on is + not yours to inspect, and you have no way to reach it except through + the server. +- The owner moves the books there with a verified copy or a backup, and + the move is confirmed at the head the preview read. If `books.verify` + or `books.start` reports a head the owner does not expect, say so and + stop; do not touch the books until the owner has looked. +- Each client has its own token. Never ask the owner for one, never print + one, and never put one in a note, an entry, a receipt or a file in the + books. If a tool says the token was refused, ask the owner to make a + new one for that client. +- Everything else is the same as on the owner's computer: the receipt + rule, the preview before a post, the books' own settings and + conventions. The machine changes where the books are, not how they are + kept. + +If the owner asks how the machine is set up, point them at the deployment +notes that came with the app; do not invent steps for them, and do not +try to run any yourself. diff --git a/crates/ledger-mcp/src/handbook.rs b/crates/ledger-mcp/src/handbook.rs index 6f72777..beb69b2 100644 --- a/crates/ledger-mcp/src/handbook.rs +++ b/crates/ledger-mcp/src/handbook.rs @@ -86,6 +86,7 @@ pub const SECTIONS: &[(&str, &str)] = &[ ), ("viewer", include_str!("../handbook/viewer.md")), ("backups", include_str!("../handbook/backups.md")), + ("self-host", include_str!("../handbook/self-host.md")), ( "format-upgrade", include_str!("../handbook/format-upgrade.md"), diff --git a/crates/ledger-mcp/src/local.rs b/crates/ledger-mcp/src/local.rs index f276501..95dcf48 100644 --- a/crates/ledger-mcp/src/local.rs +++ b/crates/ledger-mcp/src/local.rs @@ -60,7 +60,7 @@ //! ``` use crate::admin::{self, Request, Response}; -use crate::server::{BooksServer, Served, SharedStore}; +use crate::server::{BooksServer, Isolation, Served, SharedStore}; use ledger_store::Store; use ledger_store::access::{Access, AccessLog, Outcome, Principal}; use ledger_store::auth::{Principals, Scope, Token}; @@ -187,6 +187,9 @@ struct Books { drill_first: Duration, /// Held through a restore: one at a time. restoring: Mutex<()>, + /// How these books are kept from agents (plan 3.8, P9a): what + /// `books.start` reports (`--isolation`). + isolation: Isolation, } impl Books { @@ -294,7 +297,7 @@ impl Books { } let waiting = Arc::new(Mutex::new(store.journal_format().upgrade())); let database = (store.database_file(), store.other_databases()); - let served = Served::open(store).map_err(|e| self.hide(e))?; + let served = Served::open_with(store, self.isolation).map_err(|e| self.hide(e))?; let config = StreamableHttpServerConfig::default() .with_json_response(true) .with_legacy_session_mode(true) @@ -1853,9 +1856,13 @@ fn bind(run: &Path, name: &str) -> std::io::Result { } fn print_usage(prog: &str) { - eprintln!("usage: {prog} [--root DIR]"); + eprintln!("usage: {prog} [--root DIR] [--isolation LEVEL] [--http [ADDR]]"); eprintln!(" The books server for the data root: started by the host, not by hand."); eprintln!(" It runs until its standard input ends, or SIGTERM, SIGINT or SIGHUP."); + eprintln!(" --isolation same_user | enforced: what books.start says of how these books"); + eprintln!(" are kept from agents (enforced on the owner's own machine: P9a)."); + eprintln!(" --http [ADDR]: also serve the MCP door on 127.0.0.1:7411 (or ADDR), for a"); + eprintln!(" TLS proxy or a tunnel in front of it. Off unless given."); } /// `wanderbooks server`: serve every book in the data root until told to @@ -1867,10 +1874,34 @@ pub fn run( let mut root: Option = None; let mut backup_quiet = BACKUP_QUIET; let mut drill_first = DRILL_FIRST; - let mut args = args.into_iter(); + let mut isolation = Isolation::SameUser; + let mut tunnel: Option = None; + let mut args = args.into_iter().peekable(); while let Some(arg) = args.next() { match arg.as_str() { "--root" => root = Some(PathBuf::from(args.next().ok_or("--root needs a folder")?)), + // How the books are kept from agents (plan 3.8, P9a): the + // system unit on the owner's own machine passes `enforced`; + // the server never guesses. + "--isolation" => { + let value = args + .next() + .ok_or("--isolation needs same_user or enforced")?; + isolation = Isolation::parse(&value)?; + } + // The tunnel's door (plan 3.9, P9a): the same MCP service on + // 127.0.0.1, for a TLS proxy or tunnel in front of it. Off + // unless asked for; the bearer token still guards every call. + "--http" => { + let addr = match args.peek() { + Some(v) if !v.starts_with('-') => args.next().unwrap_or_default(), + _ => "127.0.0.1:7411".to_string(), + }; + tunnel = + Some(addr.parse().map_err(|e| { + format!("--http takes an address like 127.0.0.1:7411: {e}") + })?); + } // How long the books stay unwritten before a backup (tests). "--backup-quiet" => { let ms: u64 = args @@ -1967,6 +1998,7 @@ pub fn run( backup_quiet, drill_first, restoring: Mutex::new(()), + isolation, }); { let loaded = books.load_principals(); @@ -1984,7 +2016,7 @@ pub fn run( eprintln!("wanderbooks server: told to close while opening: closed"); Ok(()) } else { - rt.block_on(serve(books.clone(), &run, &stops)) + rt.block_on(serve(books.clone(), &run, &stops, tunnel)) }; // No request is running now (serve waited for each); the stores go // before the lock, and the process exit would release both. @@ -2024,9 +2056,26 @@ async fn serve( books: Arc, run: &Path, stops: &Stops, + tunnel: Option, ) -> Result<(), Box> { let mcp_listener = OwnerOnly(bind(run, MCP_SOCKET)?); let mut admin_listener = OwnerOnly(bind(run, ADMIN_SOCKET)?); + // The tunnel's door (plan 3.9, P9a): the same service on 127.0.0.1, + // for the TLS proxy or tunnel in front of it. Bound before ready, so + // what starts beside the server finds it; a bind that fails is said + // and is not fatal (3.9: the books' own sockets still serve). + let tunnel_listener = match tunnel { + Some(addr) => match tokio::net::TcpListener::bind(addr).await { + Ok(listener) => Some(listener), + Err(e) => { + eprintln!( + "wanderbooks server: the tunnel's door at {addr} can't be bound ({e}); serving the local sockets only" + ); + None + } + }, + None => None, + }; let stop = stops.close.clone(); { use std::io::Write; @@ -2083,11 +2132,21 @@ async fn serve( // close is done: a client is never cut off mid-request by it. let shut = CancellationToken::new(); let graceful = shut.clone(); + let tunnel_app = app.clone(); let mut http = tokio::spawn(async move { axum::serve(mcp_listener, app) .with_graceful_shutdown(async move { graceful.cancelled().await }) .await }); + let tunnel_http = tunnel_listener.map(|listener| { + let app = tunnel_app; + let graceful = shut.clone(); + tokio::spawn(async move { + axum::serve(listener, app) + .with_graceful_shutdown(async move { graceful.cancelled().await }) + .await + }) + }); let admin_jobs = TaskTracker::new(); loop { tokio::select! { @@ -2172,6 +2231,14 @@ async fn serve( Ok(Err(e)) => eprintln!("wanderbooks server: {e}"), Err(_) => http.abort(), } + if let Some(mut tunnel) = tunnel_http { + match tokio::time::timeout(HTTP_GRACE, &mut tunnel).await { + Ok(Ok(Ok(()))) => {} + Ok(Ok(Err(e))) => eprintln!("wanderbooks server: {e}"), + Ok(Err(e)) => eprintln!("wanderbooks server: {e}"), + Err(_) => tunnel.abort(), + } + } eprintln!("wanderbooks server: closed"); Ok(()) } @@ -2599,6 +2666,7 @@ mod tests { backup_quiet: BACKUP_QUIET, drill_first: DRILL_FIRST, restoring: Mutex::new(()), + isolation: Isolation::SameUser, }); let loaded = books.load_principals(); *books.principals.write().unwrap() = loaded; diff --git a/crates/ledger-mcp/src/server.rs b/crates/ledger-mcp/src/server.rs index a1068e3..dce5b8b 100644 --- a/crates/ledger-mcp/src/server.rs +++ b/crates/ledger-mcp/src/server.rs @@ -74,6 +74,46 @@ pub struct BooksServer { exports: ledger_store::exports::Exports, /// The books' backups (backup.rs). backups: Arc, + /// How these books are kept from agents (plan 3.8, P9a): what + /// `books.start` reports. + isolation: Isolation, +} + +/// How the books are kept from agents (plan 3.8, 2.1; D-32, D-51). +/// +/// Locally the server runs as the human and the books are hidden, not +/// locked: `SameUser`. On a machine of the owner's the server runs as a +/// dedicated account and agents have no account there: `ServiceUser`. +/// The server never guesses which; `--isolation enforced` says so +/// (`serve --isolation enforced` under the system unit, P9a). +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Isolation { + /// Hidden from the agent in the owner's own account, not locked. + SameUser, + /// A separate account on another machine; the agent has none there. + ServiceUser, +} + +impl Isolation { + /// What `books.start` says (the strings plan 3.8 names). + pub fn label(self) -> &'static str { + match self { + Self::SameUser => "same_user (advisory)", + Self::ServiceUser => "service_user (enforced)", + } + } + + /// What `--isolation` takes: `same_user`, `advisory`, `service_user` + /// or `enforced`. + pub fn parse(text: &str) -> Result { + match text { + "same_user" | "advisory" => Ok(Self::SameUser), + "service_user" | "enforced" => Ok(Self::ServiceUser), + other => Err(format!( + "--isolation takes same_user or enforced, not {other:?}" + )), + } + } } /// One book as the books server serves it (local.rs), shared by every @@ -88,15 +128,23 @@ pub(crate) struct Served { pub(crate) uploads: ledger_store::uploads::Uploads, pub(crate) exports: ledger_store::exports::Exports, pub(crate) backups: Arc, + pub(crate) isolation: Isolation, } impl Served { + /// [`Self::open_with`] for books served the local way + /// ([`Isolation::SameUser`]): how the server's own tests open them. + #[cfg(test)] + pub(crate) fn open(store: Store) -> Result { + Self::open_with(store, Isolation::SameUser) + } + /// What serving the books `store` holds needs, opened beside them: /// the settings in force (the books' own), the access log, the uploads /// and exports, the viewer's side channel, and the backups. The books /// server opens every book this way (local.rs), the only way books are - /// served (plan D-44). - pub(crate) fn open(store: Store) -> Result { + /// served (plan D-44), saying how the server runs (plan 3.8, P9a). + pub(crate) fn open_with(store: Store, isolation: Isolation) -> Result { let mut live = Live::own(); live.sync(store.ledger().settings(), store.base_currency().code()); let access = Arc::new( @@ -130,6 +178,7 @@ impl Served { uploads, exports, backups, + isolation, }) } } @@ -498,6 +547,7 @@ impl BooksServer { uploads: served.uploads.clone(), exports: served.exports.clone(), backups: served.backups.clone(), + isolation: served.isolation, } } diff --git a/crates/ledger-mcp/src/tools/kept.rs b/crates/ledger-mcp/src/tools/kept.rs index 96c622e..7d4e859 100644 --- a/crates/ledger-mcp/src/tools/kept.rs +++ b/crates/ledger-mcp/src/tools/kept.rs @@ -266,9 +266,11 @@ pub struct StartOut { pub format_upgrade: Option, /// What to do next, in order. pub next_steps: Vec, - /// How the books are kept from agents: "same_user (advisory)", hidden - /// from them in the human's own account, not locked against them - /// (plan D-32): reach them only through these tools. + /// How the books are kept from agents: "same_user (advisory)" locally + /// (hidden from them in the human's own account, not locked against + /// them: plan D-32), or "service_user (enforced)" when the books run + /// on the owner's own machine as a dedicated account (plan 3.8, P9a). + /// Either way, reach them only through these tools. pub isolation: String, } @@ -921,7 +923,7 @@ impl BooksServer { how: "only the owner, with the owner passphrase: in the app (it asks when it opens the books) or `wanderbooks books upgrade ID` at a terminal; a verified backup with the keys is made first. No tool can.".to_string(), }), next_steps, - isolation: "same_user (advisory)".to_string(), + isolation: self.isolation.label().to_string(), }; // Fit (INVARIANT 4): whole lists if they fit; else summaries go, // then the lists shorten from the end, each saying how many more. diff --git a/crates/ledger-mcp/src/types.rs b/crates/ledger-mcp/src/types.rs index 0698e5c..7a4ef1b 100644 --- a/crates/ledger-mcp/src/types.rs +++ b/crates/ledger-mcp/src/types.rs @@ -4719,7 +4719,7 @@ pub struct ReconciliationsOut { /// Parameters for `books.handbook`. #[derive(Debug, Default, Deserialize, JsonSchema)] pub struct HandbookParams { - /// The section to read: contract, entries, drafts-and-reversals, evidence-and-receipts, attachments, files-in-out, reconciliation, fixed-assets, prepaid-and-recurring, revenue-and-payouts, projects, parties-and-tins, 1099, cpa-handoff, settings-and-conventions, viewer, backups, format-upgrade, rules-that-change, control-roles, tracing-figures, or "all" for the whole handbook in order. Omit for the index. + /// The section to read: contract, entries, drafts-and-reversals, evidence-and-receipts, attachments, files-in-out, reconciliation, fixed-assets, prepaid-and-recurring, revenue-and-payouts, projects, parties-and-tins, 1099, cpa-handoff, settings-and-conventions, viewer, backups, self-host, format-upgrade, rules-that-change, control-roles, tracing-figures, or "all" for the whole handbook in order. Omit for the index. #[serde(default)] pub section: Option, } diff --git a/deploy/README.md b/deploy/README.md new file mode 100644 index 0000000..008d14a --- /dev/null +++ b/deploy/README.md @@ -0,0 +1,258 @@ +# The books on a machine of your own + +Plan 8.2 P9a, section 3.8. This is the recipe for the enforced deployment: +the books live on a machine you control, in a folder only a dedicated +account can read, and agents reach them through the server with a token. +Locally the books are `same_user (advisory)` — hidden from the agent, not +locked against it; here they are behind an account and a machine boundary. + +Nothing here needs the owner's books: every step works on a copy. + +The pieces: + +- `wanderbooks-server.service` — the system unit (one account, one data + root, loopback only). +- `wanderbooks` — the server-only build of the one executable + (`cargo build --release -p ledger-cli --no-default-features`). +- Caddy (or Tailscale Funnel, or cloudflared) — TLS and the door from the + internet. +- `crates/ledger-cli/tests/two_user.rs` — the two-account checks the CI job + `.github/workflows/two-user.yml` runs, and which you can run on one + machine (step 9). + +## 1. The account and the folder + +```sh +sudo useradd --system --create-home --home-dir /var/lib/wanderbooks \ + --shell /usr/sbin/nologin wanderbooks +sudo install -d -o wanderbooks -g wanderbooks -m 0700 /srv/wanderbooks +sudo install -m 0755 target/release/wanderbooks /usr/local/bin/wanderbooks +sudo install -D -m 0644 deploy/wanderbooks-server.service \ + /etc/systemd/system/wanderbooks-server.service +``` + +The executable is the server-only build: no viewer, no window libraries. +On a Linux package install (P11) this is the same binary and the package +manager writes the file; until then, copy it as above. + +## 2. Make the data root, once + +```sh +sudo -u wanderbooks -H /usr/local/bin/wanderbooks setup --root /srv/wanderbooks +``` + +`setup` makes the folder private (0700), records it for that account and +asks for the owner passphrase when a terminal is attached. Self-host has +no `/h/` pages (D-28, D-51); the passphrase guards a format upgrade only. +Set it at a terminal if the machine has none: + +```sh +ssh -t you@server 'sudo -u wanderbooks -H /usr/local/bin/wanderbooks passphrase' +``` + +## 3. Bring the books over + +The transport is a backup clone or a copied data folder (D-31: there is no +`fetch`, and no separate archive verifier). On the machine the books are +on now: + +```sh +wanderbooks close # nothing writes while it is copied +wanderbooks books list # note the id, e.g. acme +cp -a "$XDG_DATA_HOME/wanderbooks/books/acme" /media/usb/acme +``` + +The copy must hold the journal, the stored files, the vault and its key; +`cp -a` of the book's folder holds all four. If you would rather move a +verified backup, make one (`wanderbooks books back-up acme`) and clone the +repository it wrote; both are the same books. + +Bring that folder to the server, start the service, and adopt it: + +```sh +sudo systemctl daemon-reload +sudo systemctl enable --now wanderbooks-server +sudo -u wanderbooks -H /usr/local/bin/wanderbooks books adopt /media/usb/acme \ + --root /srv/wanderbooks --id acme +``` + +The preview names the source's head (`Head: event N (short hash)`); with +`--yes` (or a `y` at the prompt) the move is confirmed against exactly +that head, so a source that changed after the preview is refused. The +original is left as it was; nothing is deleted. + +**The head check.** Ask the agent on the server for `books.verify` (or +`books.start`): both report the head. Compare it with `books.verify` on +the machine the books came from, taken before the move. They are the same +event count, sequence number and hash: the move loses nothing. (The copy +is written in this version's journal format; a source written by an older +version gains one format event at adoption, and then the counts differ by +exactly that event, which the preview says: "The copy is in this version's +journal format".) + +## 4. Keep the machine's hardening + +The unit says `--isolation enforced`, so `books.start` reports +`service_user (enforced)`: the server is told, it never guesses. The unit +also sets `ProtectSystem=strict`, `NoNewPrivileges=yes`, `ProtectHome=yes` +and `IPAddressDeny=any` with `IPAddressAllow=localhost`, and no seccomp or +Landlock lines (D-32, D-51). + +The loopback-only rule is deliberate: the server reaches nothing out. A +book whose backup target is a **remote** git repository needs that host +allowed, or its backups fail: + +```ini +# /etc/systemd/system/wanderbooks-server.service.d/backup.conf +[Service] +IPAddressAllow=203.0.113.10 +``` + +A backup to a folder on this machine needs nothing. The tunnel and Caddy +run as their own services and reach 127.0.0.1:7411, which the rule allows. + +## 5. A token for each client + +The token is a static bearer (`wb_` and 32 random bytes); the server keeps +only its SHA-256. Mint one per client, named for it: + +```sh +sudo -u wanderbooks -H /usr/local/bin/wanderbooks token create \ + --root /srv/wanderbooks --book acme --name laptop +sudo -u wanderbooks -H /usr/local/bin/wanderbooks token print \ + --root /srv/wanderbooks --book acme --name laptop +``` + +`token create` keeps the text in the server account's OS keyring and +`token print` shows it once; copy it to that client and nowhere else. +Revoke it with `token revoke --book acme --name laptop` when a client is +lost. + +A headless machine has no Secret Service by default, and `token create` +refuses to mint a token it cannot keep. Run the two commands inside a +session that has one, for example: + +```sh +sudo -u wanderbooks -H dbus-run-session -- sh -c ' + gnome-keyring-daemon --unlock --components=secrets >/dev/null + /usr/local/bin/wanderbooks token create --root /srv/wanderbooks --book acme --name laptop + /usr/local/bin/wanderbooks token print --root /srv/wanderbooks --book acme --name laptop +' +``` + +The keyring lives only for that session, which is enough: the text is +copied to the laptop, and the server checks the token's SHA-256 from the +books, not from the keyring. (Making a token without keeping it anywhere +is not a command; if your machine has a persistent Secret Service, use it.) + +## 6. TLS and the door + +The server listens on 127.0.0.1:7411 for the tunnel, and on its own Unix +sockets for anything running as the same account. Put TLS in front of the +loopback port. Two ways that need no open inbound port: + +**Tailscale Funnel** (a tailnet name, TLS included): + +```sh +sudo tailscale funnel --bg 7411 +``` + +The books are then at `https://..ts.net/b/acme/mcp`. + +**Caddy** (a domain you own, automatic certificates): + +```caddyfile +books.example.com { + reverse_proxy 127.0.0.1:7411 +} +``` + +with `cloudflared tunnel --url http://127.0.0.1:7411` instead of Caddy if +you use a Cloudflare tunnel. Either way the path is passed through +unchanged: the MCP endpoint is `https:///b/acme/mcp`. + +Nothing else listens. Do not forward 7411 on the router, and do not put +the Unix sockets behind the proxy: they check the caller's account (3.9) +and a proxy would hide it. + +## 7. Point the agent at it + +**Claude Code** (a client that sends headers): + +```sh +claude mcp add --transport http wanderbooks-acme https://books.example.com/b/acme/mcp \ + --header "Authorization: Bearer " +``` + +**Codex** (`~/.codex/config.toml`; the token stays in the environment, not +the file): + +```toml +[mcp_servers.wanderbooks-acme] +url = "https://books.example.com/b/acme/mcp" +bearer_token_env_var = "WANDERBOOKS_ACME_TOKEN" +``` + +```sh +export WANDERBOOKS_ACME_TOKEN='' +``` + +The plan's local rule is that no client configuration holds a token (the +bridge reads the keyring). For a remote machine there is no command yet +that keeps a remote token in the client's keyring, so this recipe puts it +in Claude's configuration (mode 0600) or in Codex's environment. That +gap is recorded in `plan/served.md` 8.2 P9a's As-built block. + +## 8. First start + +Start a session and ask for `books.start`. It answers +`isolation: "service_user (enforced)"`, and every tool works over the TLS +door with the token. Logs: + +```sh +journalctl -u wanderbooks-server -f +``` + +The server runs no jobs in the background: `serve` holds the books open +until systemd stops it. On a stop it finishes the call in progress, makes +a last backup and closes (3.9), so `TimeoutStopSec=600` is generous rather +than needed. + +Backups still run on their own, from the books' settings, as on the +desktop (3.7, D-30, D-31); this machine runs git itself. + +## 9. Prove the boundary (what CI runs) + +`.github/workflows/two-user.yml` runs the same job on every change: + +1. makes two accounts, `wanderbooks` (server) and the runner (agent); +2. makes books with this version in a desktop root, copies them, and + adopts the copy into `/srv/wanderbooks`; +3. serves them as `wanderbooks` with `--isolation enforced --http + 127.0.0.1:17411`, and mints a token; +4. runs `crates/ledger-cli/tests/two_user.rs` as the runner with + `WB_TWO_USER=1` and the deployment's settings. The test proves the + runner cannot open `/srv/wanderbooks` or its journal, then reads, + writes, uploads, files, exports and verifies over the HTTP door (in a + second book, `tunnel`, so the moved books' head is never advanced), and + compares the moved books' head with the source's. + +`agent_uid_cannot_read_tenant` and `self_host_adopt_same_head` skip +themselves when `WB_TWO_USER` is unset, so an ordinary `cargo test` on a +laptop stays green; `isolation_label_is_enforced_under_the_unit` needs no +second account and always runs. + +On one machine without a second account, run the client side in a user +namespace (a different user ID than the server's): + +```sh +WB_TWO_USER=1 WB_TENANT_ROOT=... WB_SERVER_URL=http://127.0.0.1:17411 \ +WB_BOOK=acme WB_TOKEN=... WB_WRITE_BOOK=tunnel WB_WRITE_TOKEN=... \ +WB_SOURCE_DIR=... \ +unshare --user --map-auto --setuid 0 --setgid 0 -- target/debug/deps/two_user- --nocapture +``` + +The two books and their tokens are made by the job (or by hand as in +steps 2 to 5): `acme` is the moved books, and `tunnel` is empty, so the +write checks never move `acme`'s head. `WB_SOURCE_DIR` is the journal the +books were moved from, readable by the agent account. diff --git a/deploy/wanderbooks-server.service b/deploy/wanderbooks-server.service new file mode 100644 index 0000000..8a95a6d --- /dev/null +++ b/deploy/wanderbooks-server.service @@ -0,0 +1,60 @@ +# The books on a machine of the owner's own (plan 8.2 P9a, section 3.8). +# +# One server, one account, one door. It runs `wanderbooks serve` as the +# dedicated account `wanderbooks`, over a data root only that account can +# read, and says `service_user (enforced)` in `books.start` (the flag; the +# server never guesses). Agents have no account on this machine and reach +# the books through the TLS door on 127.0.0.1:7411 (a tunnel or Caddy in +# front of it), with a static bearer token. +# +# No seccomp or Landlock lines: D-32 (no self-confinement) and D-51 (P9a +# is the recipe and the boundary between accounts, nothing more). What is +# here is the unit's own hardening of the account and the filesystem. +# +# Install: /usr/local/bin/wanderbooks is the server-only build +# (`cargo build --release -p ledger-cli --no-default-features`), and the +# root has been made once by `wanderbooks setup --root /srv/wanderbooks` +# as the wanderbooks account. See README.md in this folder for the steps. + +[Unit] +Description=wanderbooks: the books server +After=network-online.target +Wants=network-online.target + +[Service] +Type=simple +User=wanderbooks +Group=wanderbooks +DynamicUser=no +# The account's own home (the keyring, when one runs; the recorded root). +StateDirectory=wanderbooks +Environment=HOME=/var/lib/wanderbooks +WorkingDirectory=/ +# The books' data root, and the tunnel's door. `--isolation enforced` is +# what `books.start` reports; `--http` serves the same MCP service on +# 127.0.0.1 for Caddy or a tunnel. Drop `--http` if this machine has no +# TLS door. +ExecStart=/usr/local/bin/wanderbooks serve --root /srv/wanderbooks --isolation enforced --http 127.0.0.1:7411 +# `serve` hears SIGTERM and lets go: the server finishes the call in +# progress, makes a last backup and closes. A long call is never cut off. +KillSignal=SIGTERM +TimeoutStopSec=600 +Restart=on-failure +RestartSec=2s + +# The account and the filesystem (plan 3.8, 3.9). +NoNewPrivileges=yes +ProtectSystem=strict +ReadWritePaths=/srv/wanderbooks +ProtectHome=yes +PrivateTmp=yes +PrivateDevices=yes +# No network but loopback: the tunnel reaches 127.0.0.1:7411, and the +# server reaches nothing out. A book whose backup target is a remote git +# repository needs that host allowed here (`IPAddressAllow=` its address) +# or its backups fail; see README.md. +IPAddressDeny=any +IPAddressAllow=localhost + +[Install] +WantedBy=multi-user.target diff --git a/docs/architecture.md b/docs/architecture.md index da798ff..3b33ce6 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -220,3 +220,35 @@ Neither bookspec crate depends on `ledger-core`; the dependency points inward from the adapter. `just check` runs the bookspec workspace's format, clippy and tests beside the wanderbooks workspace's, so an example cannot rot (P13 step 1). + +## Self-host: the same server, another account (P9a, D-51) + +The books can live on a machine the owner controls (plan 3.8). It is the +same `ledger-mcp` server, the same store and the same tools; only the +process boundary changes. `wanderbooks serve` runs as the dedicated +account `wanderbooks` under `deploy/wanderbooks-server.service`, over a +data root only that account can read (`/srv/wanderbooks`, 0700). Agents +have no account on that machine, so the isolation is enforced by the +operating system rather than advisory: the server is told +`--isolation enforced` (the flag, never a guess) and `books.start` +reports `service_user (enforced)` (`server::Isolation`). + +The door is the same HTTP service on a second listener: +`--http 127.0.0.1:7411` binds the MCP routes beside the Unix sockets +(3.9), and Caddy, Tailscale Funnel or cloudflared puts TLS and the public +name in front of it. The bearer token guards every call exactly as on the +socket; the tunnel only forwards. The socket's peer-credential check +(`OwnerOnly`) is for local callers and does not apply to the loopback +door, which is why the bearer and the loopback-only bind matter there. +The flag travels `serve` → `ledger-host`'s host → `server` +(`Opener::starting_with`, `start::start_host`, `host::Args`), so a host +already running with other flags keeps them; on a self-host machine +`serve` is the unit's only holder. + +The unit sets `IPAddressDeny=any` and `IPAddressAllow=localhost`: the +server reaches nothing out. A book whose backup target is a remote git +repository therefore needs that host allowed in a drop-in (D-31 backups +still run on the server). The recipe, the head check for the move and the +two-account CI job are in `deploy/README.md`, `deploy/wanderbooks-server.service` +and `.github/workflows/two-user.yml`; the checks are +`crates/ledger-cli/tests/two_user.rs`. diff --git a/docs/mcp-interface.md b/docs/mcp-interface.md index ebb9d31..d3734d5 100644 --- a/docs/mcp-interface.md +++ b/docs/mcp-interface.md @@ -115,9 +115,12 @@ settings-and-conventions section says how to use them. (below); its next step comes first, before anything else, at every start until it is done: `books.handbook {section: "format-upgrade"}`, "Tell the human first: …". - `isolation` is `same_user (advisory)` (D-32): the books are hidden from - agents in the owner's own account, not locked against them. Lists cut - to fit say so in `truncated` ("conventions: 4 more (books.conventions)"). + `isolation` is `same_user (advisory)` locally (D-32): the books are + hidden from agents in the owner's own account, not locked against them. + On the owner's own machine, where the server runs as a separate account + and agents have none, the unit passes `--isolation enforced` and it is + `service_user (enforced)`; the server never guesses (P9a, 3.8). Lists + cut to fit say so in `truncated` ("conventions: 4 more (books.conventions)"). `projects` lists the active and paused projects by name, at most 20 ("projects: N more (books.projects)" past that). `next_steps`, in order: the format upgrade's (when waiting); the @@ -326,8 +329,8 @@ The long guidance is the **handbook**, compiled in from contract, entries, drafts-and-reversals, evidence-and-receipts, attachments, files-in-out, reconciliation, fixed-assets, prepaid-and-recurring, revenue-and-payouts, parties-and-tins, 1099, - cpa-handoff, settings-and-conventions, viewer, backups, format-upgrade, - rules-that-change, control-roles, tracing-figures. + cpa-handoff, settings-and-conventions, viewer, backups, self-host, + format-upgrade, rules-that-change, control-roles, tracing-figures. - The same as resources: `books://handbook` (the index, JSON) and `books://handbook/{section}` (Markdown, `text/markdown`), one listed per section. diff --git a/docs/parity.md b/docs/parity.md index 665b723..e27be0b 100644 --- a/docs/parity.md +++ b/docs/parity.md @@ -91,3 +91,16 @@ trait), the runner (`bookspec/runner`: pass, fail or not claimed per rule), and the rule index from B-1 to B-45 (`bookspec/spec/rules-index.md`). Every existing bookspec example still runs on every `cargo test`, through both the loader's own checks and the `Engine` trait. + +### P9a self-host deployment (2026-10-26) + +P9a is a deployment, not a semantic feature: the same engine, store, MCP +surface and reports on a machine of the owner's own. No row above changed +status and the floor stands. What it adds is the enforced isolation the +catalog's parity claims rest on in practice: a dedicated account, a data +root only that account can read, `books.start` saying +`service_user (enforced)` (`--isolation`), and the same bearer-guarded MCP +door on loopback for Caddy or a tunnel. `crates/ledger-cli/tests/two_user.rs` +proves the account boundary and the move's head in CI +(`.github/workflows/two-user.yml`); the recipe is `deploy/README.md`. +The cutover rule in `docs/capability-parity.md` is unchanged. diff --git a/docs/testing.md b/docs/testing.md index 840fee7..5716b52 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -2970,3 +2970,75 @@ negative-zero test; `filter(|a| a.tool.is_some()).take(12)` and intact-blob set (a damaged blob reported per attachment) and `Store:: disk_check`'s snapshot load and `Unreadable` arm; the viewer wrapper's `Target` arms. + +## P9a: self-host, the move, and the static bearer over a tunnel (2026-10-26) + +Plan 8.2 P9a, D-51. The books on a machine of the owner's own: the same +server as a dedicated account (`deploy/wanderbooks-server.service`), a +data root only that account can read, `books.start` saying +`service_user (enforced)` when the unit passes `--isolation enforced`, the +move by `books adopt` with its head check, and the same MCP service on a +loopback door (`--http`) for Caddy or a tunnel. + +- **ledger-mcp:** `server::Isolation` (`label`, `parse`), carried from + `--isolation` through `Served::open_with` into `BooksServer`, and read by + `books.start`'s `start_out` (`tools/kept.rs`); `--http` binds a second + `axum::serve` listener over the same router and the same door, before the + ready line, and closes with the same graceful shutdown (a failed bind is + said and is not fatal, 3.9). `common/mod.rs`'s harness is unchanged; the + local flag test starts its own server (`two_user.rs::FlaggedServer`) + because `common::Server` takes no extra flags. +- **ledger-host:** `serve --isolation/--http` → `Opener::starting_with` → + `start::start_host` → `host::Args` → the server's command line. A host + already running keeps the flags it was started with (documented in + `deploy/README.md` and the opener's comment). +- **`crates/ledger-cli/tests/two_user.rs`:** + `isolation_label_is_enforced_under_the_unit` (a server with no flag says + `same_user (advisory)`; with `--isolation enforced --http` it says + `service_user (enforced)` over the Unix socket and over the loopback + door, and both report the same head; the unit file itself is read to + require the `--isolation enforced`, `--http` and `IPAddress*` lines the + test exercises); + `agent_uid_cannot_read_tenant` and `self_host_adopt_same_head` (two + accounts, run by `.github/workflows/two-user.yml`; they skip with a line + when `WB_TWO_USER` is unset, so a laptop's suite stays green). The first + proves `read_dir` and a journal `open` are refused with + `PermissionDenied`, then reads (`books.chart`), writes + (`books.add_account`), uploads (`books.begin_upload` + the `u/` route), + exports (`books.export` + the `x/` route, checked against its SHA-256) + and verifies (`books.verify {blobs:true}`) with the bearer over HTTP, in + a second book so the moved books' head is never advanced; the second + reads the source clone's journal, computes its head from the last line's + canonical bytes, and requires the moved books' `books.start` and + `books.verify` to report the same seq, hash and event count. +- **The job:** `.github/workflows/two-user.yml` makes the accounts, adopts + the repository's `p2-served` fixture into a desktop root at this version, + copies the book as the transport (D-31), adopts it into + `/srv/wanderbooks` behind `--isolation enforced --http`, mints a token + with the `file_keyring` test knob (a runner has no Secret Service), and + runs the test as the runner. `deploy/README.md` is the same recipe by + hand; the job does not install the systemd unit (it drives `serve` with + the same flags), so the unit's lines are checked by the test's file read + and by review, not executed as a unit. +- **Not covered:** the recipe walked on a clean VM by someone other than + its author (Done when 2), a real TLS tunnel (Tailscale Funnel or Caddy) + and a real Secret Service on a headless server. Those are the owner's + and the EM's checks; the As-built block says so. +- **Docs:** `docs/architecture.md` "Self-host", `docs/mcp-interface.md` + (`books.start`'s two `isolation` values), `docs/parity.md` (no semantic + row moved; the floor stands), the handbook's `self-host` section, and + `deploy/README.md`. + +**For P12 (mutation targets, none run, D-33):** `Isolation::parse`'s +`same_user`/`advisory` and `service_user`/`enforced` arms (a wrong arm +would report the advisory truth on the enforced machine); +`Isolation::label`'s two strings; `start_out`'s `self.isolation.label()` +(it replaced a literal, and a mutant that restores a literal must fail the +two-door test); the `--http` arm's `args.peek()` guard and the default +`127.0.0.1:7411` (binding the wrong interface, or consuming the next flag, +must fail); `tunnel.is_some()` and the bind failure path (ready must still +come after the try, and a failed bind must not be fatal); `open_with`'s +`isolation` into `Served` and `BooksServer::new`'s copy (a dropped field +must fail `isolation_label_is_enforced_under_the_unit`); the host's +`server_flags` extension and `parse_args`' passthrough of `--isolation` +and its optional `--http` value.