P2 slice 9: put the old copies aside after an adoption

`wanderbooks books put-aside ID DATA-DIR [--vault DIR --vault-key FILE]
[PATH...]` moves the old copies of adopted books (their data folder, their
TIN vault and its key, their settings file, the git history around them,
old exports) whole into numbered folders of their own in the root's
legacy/, a list of where each came from written and synced first. Nothing
is deleted. Refused unless the books hold everything the old copy does,
byte for byte (journal, stored files, vault records, key; what adoption
left behind and records the books sealed anew are listed and go aside
too); while a wanderbooks process holds it open for writing; when its
head moved since the preview; for links, paths named twice, inside the
root or missing, a vault or key given as a plain path, and paths on
another disk. Renames never replace anything, and a move cut short puts
back what it moved. Three review rounds of data safety.

The plan records the cutover's answers (Q-3: move the old copies aside;
the backup repository and folder).

Also: `connect`'s "the books fernhill' agent token" reads properly, and
the restart tests wait for a copied program to be runnable (a child forked
by another test while the copy was written made it "text file busy").

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3gXMVt7XJPi6ens1or9Dj
This commit is contained in:
swaitsandClaude Opus 5.5 committed 2026-10-20 12:50:43 -06:00
1 parent c53a01e163
commit cbb7d54463
7 files changed
+1044 -7

No files matched your search

+1 -1
View File
@@ -16,7 +16,7 @@ fn print_usage() {
eprintln!("usage: wanderbooks setup [--root PATH] [--home DIR]");
eprintln!(" wanderbooks gui [BOOK | --root PATH --book ID | --data-dir PATH ...]");
eprintln!(
" wanderbooks books list | create ID | adopt DATA-DIR --id ID | restore BACKUP | back-up ID"
" wanderbooks books list | create ID | adopt DATA-DIR --id ID | restore BACKUP | back-up ID | put-aside ID DATA-DIR [PATH...]"
);
eprintln!(" wanderbooks connect claude|codex --book ID");
eprintln!(
+96 -3
View File
@@ -28,6 +28,30 @@ struct Home {
exe: PathBuf,
}
/// Wait until a program just copied can be run. A child another test
/// thread forked while the copy was being written holds the copy open for
/// writing until it starts its own program, and running the copy before
/// then fails with "text file busy".
fn runnable(program: &Path) {
let deadline = Instant::now() + Duration::from_secs(10);
loop {
match Command::new(program)
.arg("--help")
.stdin(Stdio::null())
.stdout(Stdio::null())
.stderr(Stdio::null())
.status()
{
Err(e) if e.kind() == std::io::ErrorKind::ExecutableFileBusy => {
assert!(Instant::now() < deadline, "the copy stayed busy");
std::thread::sleep(Duration::from_millis(20));
}
Err(e) => panic!("the copy can't be run: {e}"),
Ok(_) => return,
}
}
}
impl Home {
fn new(tag: &str) -> Self {
let dir = tempfile::Builder::new()
@@ -53,6 +77,7 @@ impl Home {
std::fs::create_dir(&bin).unwrap();
let copy = bin.join("wanderbooks");
std::fs::copy(env!("CARGO_BIN_EXE_wanderbooks"), &copy).unwrap();
runnable(&copy);
self.exe = copy;
self
}
@@ -61,6 +86,7 @@ impl Home {
fn replace_program(&self) {
let next = self.exe.with_extension("new");
std::fs::copy(env!("CARGO_BIN_EXE_wanderbooks"), &next).unwrap();
runnable(&next);
std::fs::rename(&next, &self.exe).unwrap();
}
@@ -1914,6 +1940,70 @@ fn the_books_command_takes_its_flags_and_asks_before_adopting() {
assert!(hash.chars().all(|c| c.is_ascii_hexdigit()), "{head}");
}
#[test]
fn adopted_books_old_copies_are_put_aside_from_the_command_line() {
let home = Home::new("aside");
let old = tempfile::Builder::new()
.prefix("wb-held-aside-")
.tempdir()
.unwrap();
let data = old.path().join("books-data");
drop(ledger_store::Store::create(&data, ledger_core::Currency::USD).unwrap());
let toml = old.path().join("wanderbooks.toml");
std::fs::write(&toml, "[company]\nname = \"Old Co\"\n").unwrap();
let (d, t) = (data.to_str().unwrap(), toml.to_str().unwrap());
let out = home.books(&[
"adopt",
d,
"--id",
"old",
"--toml",
t,
"--no-vault",
"--yes",
]);
assert_eq!(out.status.code(), Some(0), "{out:?}");
let err = |o: &std::process::Output| String::from_utf8_lossy(&o.stderr).to_string();
let said = |o: &std::process::Output| String::from_utf8_lossy(&o.stdout).to_string();
// Not at a terminal and no --yes: what would go aside, and nothing moved.
let out = home.books(&["put-aside", "old", d, t]);
assert_eq!(out.status.code(), Some(1), "{out:?}");
assert!(err(&out).contains("not put aside"), "{}", err(&out));
assert!(
said(&out).contains("These go aside, each whole"),
"{}",
said(&out)
);
assert!(said(&out).contains("byte for byte"), "{}", said(&out));
assert!(said(&out).contains("Nothing is deleted"), "{}", said(&out));
assert!(data.is_dir() && toml.is_file());
// A vault comes with its key; books of no such name hold nothing.
let out = home.books(&["put-aside", "old", d, "--vault", t, "--yes"]);
assert_eq!(out.status.code(), Some(1), "{out:?}");
assert!(err(&out).contains("together"), "{}", err(&out));
let out = home.books(&["put-aside", "nobody", d, "--yes"]);
assert_eq!(out.status.code(), Some(1), "{out:?}");
assert!(data.is_dir());
// With --yes: moved, whole, into the legacy folder; the books go on.
let out = home.books(&["put-aside", "old", d, t, "--yes"]);
assert_eq!(out.status.code(), Some(0), "{out:?}");
assert!(said(&out).contains("Put aside: 2 paths"), "{}", said(&out));
assert!(!data.exists() && !toml.exists());
let legacy: Vec<_> = std::fs::read_dir(home.root().join("legacy"))
.unwrap()
.map(|e| e.unwrap().path())
.collect();
assert_eq!(legacy.len(), 1, "{legacy:?}");
assert!(legacy[0].join("1/books-data/journal.jsonl").is_file());
assert!(legacy[0].join("put-aside.txt").is_file());
assert_eq!(
std::fs::read_to_string(legacy[0].join("2/wanderbooks.toml")).unwrap(),
"[company]\nname = \"Old Co\"\n"
);
let listed = said(&home.books(&["list"]));
assert!(listed.contains("old"), "{listed}");
}
/// A program that starts the bridge with its stderr closed (some launchers
/// do) still gets a working bridge, its descriptor 2 on /dev/null: Rust's
/// runtime opens it before main, so no socket the bridge opens can take
@@ -2494,7 +2584,10 @@ fn an_agent_is_connected_and_reaches_the_books_by_what_it_was_given() {
let out = connect("claude", &bin);
assert!(out.status.success(), "{out:?}");
let said = String::from_utf8_lossy(&out.stdout);
assert!(said.contains("Made the books beta' agent token"), "{said}");
assert!(
said.contains("Made the agent token of the books beta,"),
"{said}"
);
assert!(
said.contains("Claude Code reaches the books beta"),
"{said}"
@@ -2525,7 +2618,7 @@ fn an_agent_is_connected_and_reaches_the_books_by_what_it_was_given() {
let out = connect("claude", &bin);
assert!(out.status.success(), "{out:?}");
assert!(
!String::from_utf8_lossy(&out.stdout).contains("Made the books"),
!String::from_utf8_lossy(&out.stdout).contains("Made the agent token"),
"{out:?}"
);
// Codex: its own list, no scope.
@@ -2677,7 +2770,7 @@ fn connect_makes_anew_a_token_the_keyring_lost_and_says_when_an_agent_refuses()
let out = connect(&bin);
assert!(out.status.success(), "{out:?}");
assert!(
String::from_utf8_lossy(&out.stdout).contains("Made the books acme' agent token"),
String::from_utf8_lossy(&out.stdout).contains("Made the agent token of the books acme,"),
"{out:?}"
);
// The agent's own command refuses: said, with what it said.
+115 -1
View File
@@ -398,6 +398,9 @@ fn books_usage(prog: &str) {
);
eprintln!(" {prog} restore BACKUP [--id ID] [--replace] [--yes]");
eprintln!(" {prog} back-up ID [--new-disk]");
eprintln!(
" {prog} put-aside ID DATA-DIR [--vault DIR --vault-key FILE] [PATH...] [--yes]"
);
eprintln!(" list the books, and whether each is open");
eprintln!(" create start new, empty books (base currency USD unless given)");
eprintln!(" adopt copy existing books in, verified; the original is left as it was");
@@ -407,6 +410,9 @@ fn books_usage(prog: &str) {
eprintln!(
" restore bring back a backup (a git repository's address, or BOOK.git in the backup folder), proven whole first: as new books, or replacing books (kept in legacy/)"
);
eprintln!(
" put-aside move the old copies of adopted books (their data folder, their vault and its key, and other paths given: settings file, git history) whole into the legacy folder, once the books hold everything they do, byte for byte; nothing is deleted"
);
}
/// `wanderbooks books list|create|adopt|restore`.
@@ -488,6 +494,17 @@ pub fn books(
);
}
["restore", source] => return restore(&common, source, id, replace, yes),
["put-aside", book, source, also @ ..] => {
let also: Vec<PathBuf> = also.iter().map(PathBuf::from).collect();
let old = ledger_store::root::OldCopies {
source: Path::new(source),
vault_dir: vault.as_deref(),
vault_key: vault_key.as_deref(),
also: &also,
expect_head: None,
};
return put_aside(&common, book, old, yes);
}
["back-up", book] => {
return match back_up_now(&common, book, new_disk) {
Ok(lines) => {
@@ -805,7 +822,7 @@ pub fn connect_agent(common: &Common, agent: &str, book: &str) -> Result<Vec<Str
.collect();
make_token(common, &book, crate::tokens::DEFAULT_NAME, scopes, true)?;
lines.push(format!(
"Made the books {book}' agent token, kept in this computer's keyring."
"Made the agent token of the books {book}, kept in this computer's keyring."
));
}
// The agent's own list: the books' entry made anew.
@@ -1336,6 +1353,103 @@ fn adopt(
}
}
/// `wanderbooks books put-aside` (INVARIANT 8 of the root): what goes
/// aside and what was checked, asked for (or --yes), then moved, only if
/// the old copy is still as the preview read it.
fn put_aside(
common: &Common,
book: &str,
old: ledger_store::root::OldCopies<'_>,
yes: bool,
) -> Result<(), Box<dyn std::error::Error>> {
let fail = |why: String| -> Result<(), Box<dyn std::error::Error>> {
eprintln!("error: {why}");
Err(exit(1))
};
if old.vault_dir.is_some() != old.vault_key.is_some() {
return fail(
"give the vault's folder and its key together, with --vault DIR --vault-key FILE"
.into(),
);
}
let root = match ledger_store::root::Root::find(common.root.as_deref()) {
Ok(root) => root,
Err(e) => return fail(e.to_string()),
};
let id = match ledger_store::root::BookId::parse(book) {
Ok(id) => id,
Err(e) => return fail(e.to_string()),
};
let preview = match root.put_aside(&id, &old, false) {
Ok(p) => p,
Err(e) => return fail(e.to_string()),
};
let mut held = format!(
"its journal through event {} ({}) and its {} stored files",
preview.head.0,
short_hash(&preview.head.1),
preview.blob_files
);
if preview.key_checked {
held.push_str(&format!(
", its {} vault records and the vault's key",
preview.vault_files
));
}
let alike = if preview.vault_changed.is_empty() {
"byte for byte"
} else {
"byte for byte, or newer"
};
println!("The books \"{id}\" hold everything the old copy does, {alike}: {held}.");
println!(
"These go aside, each whole in a numbered folder of its own, into the legacy folder as {}:",
preview.name
);
for (k, item) in preview.items.iter().enumerate() {
println!(" {} {}", k + 1, item.display());
}
if !preview.vault_changed.is_empty() {
println!(
"The books have sealed these vault records anew since (theirs is newer; the old ones go aside too):"
);
for rel in &preview.vault_changed {
println!(" {}", rel.display());
}
}
if !preview.left_behind.is_empty() {
println!(
"These files beside the stored files were never part of the books (left behind by adoption; they go aside too):"
);
for rel in &preview.left_behind {
println!(" {}", rel.display());
}
}
println!("Nothing is deleted; a list of where each came from is kept with them.");
if !confirmed("Put them aside?", yes).map_err(|why| {
eprintln!("error: not put aside: {why}");
exit(1)
})? {
println!("Nothing was put aside.");
return Ok(());
}
let old = ledger_store::root::OldCopies {
expect_head: Some(&preview.head),
..old
};
match root.put_aside(&id, &old, true) {
Ok(done) => {
println!(
"Put aside: {} paths, in the legacy folder as {}.",
done.items.len(),
done.name
);
Ok(())
}
Err(e) => fail(e.to_string()),
}
}
/// What an adoption did, a line each.
fn adopted_lines(a: &admin::Adopted) -> Vec<String> {
let mut extra = String::new();
+477
View File
@@ -36,6 +36,14 @@
//! is (INVARIANT 5), and its head must be the one the preview read;
//! it never overwrites: books it replaces are moved whole into
//! `legacy/`, only when asked, and only while nothing holds them.
//! 8. Putting old copies aside (after an adoption) deletes nothing: each
//! path is moved whole into a numbered folder of its own in `legacy/`,
//! a list of where each came from written there first; only when the
//! adopted books hold everything the old copy holds, byte for byte
//! (journal, stored files, vault records, key); only while no
//! wanderbooks process holds it open for writing; only on the root's
//! own disk; and a move cut short puts back what it moved, never over
//! anything.
//! ```
use crate::error::StoreError;
@@ -1127,6 +1135,475 @@ impl Root {
})()
.map_err(|e| self.hide(e))
}
/// Put the old copies of the books `id` aside (INVARIANT 8): the
/// data folder they were adopted from, their TIN vault and its key,
/// and other paths that hold them too (their settings file, the git
/// history around them, old exports), each moved whole into a numbered
/// folder of its own in one folder of `legacy/`, nothing deleted, with
/// a list of where each came from written there first. Refused unless
/// the books `id` hold everything the old copy does, byte for byte: its
/// journal (its head is in theirs), every stored file, every vault
/// record and the vault's key; while a wanderbooks process holds the
/// old copy open for writing; when its head isn't the one a preview
/// read (`expect_head`); for a path that isn't there, is a link, is
/// inside the root or holds it, or is inside another one given; and
/// for a path on another disk than the root (moving it there would be
/// a copy). Without `confirm` it moves and writes nothing: what would
/// move, and where. A move cut short puts back what it moved, never
/// over anything.
pub fn put_aside(
&self,
id: &BookId,
old: &OldCopies<'_>,
confirm: bool,
) -> Result<PutAside, RootError> {
(|| {
let root = std::fs::canonicalize(&self.path)?;
let book = self.book_dir(id);
if !book.join(JOURNAL_FILE).is_file() {
return Err(RootError::Adopt(format!(
"there are no books called {:?}",
id.as_str()
)));
}
let given: Vec<&Path> = std::iter::once(old.source)
.chain(old.vault_dir)
.chain(old.vault_key)
.chain(old.also.iter().map(PathBuf::as_path))
.collect();
let mut items = Vec::new();
for given in given {
let given = std::path::absolute(given)?;
let shown = given
.file_name()
.map(|n| n.to_string_lossy().into_owned())
.unwrap_or_else(|| given.display().to_string());
let (Some(parent), Some(name)) = (given.parent(), given.file_name()) else {
return Err(RootError::Adopt(format!("{shown} can't be put aside")));
};
let path = std::fs::canonicalize(parent)
.map_err(|e| RootError::Adopt(format!("{shown}: {}", e.kind())))?
.join(name);
let meta = std::fs::symlink_metadata(&path)
.map_err(|e| RootError::Adopt(format!("{shown}: {}", e.kind())))?;
// A link would go aside as a link, its data left where it
// is: name what it points to instead.
if meta.file_type().is_symlink() {
return Err(RootError::Adopt(format!(
"{shown} is a link: name what it points to"
)));
}
// Named by its last part only: a path into the root never
// shows where the root is (INVARIANT 6).
if path.starts_with(&root) || root.starts_with(&path) {
return Err(RootError::Adopt(format!(
"{shown} is inside {THE_ROOT} (or holds it): only what is outside it is \
put aside"
)));
}
#[cfg(unix)]
{
use std::os::unix::fs::MetadataExt;
if meta.dev() != std::fs::metadata(&root)?.dev() {
return Err(RootError::Adopt(format!(
"{} is on another disk than {THE_ROOT}: moving it there would be a \
copy, so nothing is put aside",
path.display()
)));
}
}
// A vault or its key among the other paths would go aside
// unchecked: each comes by its own name, and is checked.
let k = items.len();
let checked = (k == 1 && old.vault_dir.is_some())
|| (k == 1 + usize::from(old.vault_dir.is_some()) && old.vault_key.is_some());
let beside_vault = items
.first()
.and_then(|s: &PathBuf| s.parent())
.is_some_and(|dir| path == dir.join(VAULT_DIR));
if k > 0 && !checked && (beside_vault || name == "vault.key") {
return Err(RootError::Adopt(format!(
"{shown} looks like the TIN vault or its key: give it with --vault DIR \
--vault-key FILE, so it is checked against the books before it goes"
)));
}
items.push(path);
}
for (i, a) in items.iter().enumerate() {
for (j, b) in items.iter().enumerate() {
if i != j && a.starts_with(b) {
return Err(RootError::Adopt(format!(
"{} is {} or inside it: name each only once",
a.display(),
b.display()
)));
}
}
}
let source = &items[0];
if !source.join(JOURNAL_FILE).is_file() {
return Err(RootError::Adopt(format!(
"there are no books at {}: it has no {JOURNAL_FILE}",
source.display()
)));
}
// Held for writing: nothing is moved from under it. A preview
// only looks; the move holds the lock to the end.
let _lock = if confirm {
Some(lock_source(source)?.0)
} else {
if let Ok(lock) = std::fs::File::open(source.join(crate::store::LOCK_FILE))
&& let Err(std::fs::TryLockError::WouldBlock) = lock.try_lock_shared()
{
return Err(RootError::from(StoreError::Locked(
"another wanderbooks process has those books open for writing (an \
agent session, or its MCP server); close it first"
.into(),
)));
}
None
};
let (_, lines) = read_source_journal(source)?;
let head = head_of(&lines);
if let Some(expected) = old.expect_head
&& *expected != head
{
return Err(RootError::Adopt(format!(
"the old copy changed since it was looked at (now through event {}): look \
again",
head.0
)));
}
let lacks = |what: String| {
RootError::Adopt(format!(
"the books {:?} don't hold everything the old copy does ({what}): adopt \
it first, or name the books that hold it",
id.as_str()
))
};
if !journal_holds(&book, &head)? {
return Err(lacks(format!("its journal through event {}", head.0)));
}
let (blob_files, left_behind) = blobs_held(
&source.join(crate::store::BLOBS_DIR),
&book.join(crate::store::BLOBS_DIR),
)?
.map_err(|rel| lacks(format!("the stored file {}", rel.display())))?;
let (vault_files, vault_changed) = match old.vault_dir {
Some(_) => vault_held(&items[1], &book.join(VAULT_DIR))?.map_err(|rel| {
lacks(format!(
"the vault record {} (never adopted, or purged since)",
rel.display()
))
})?,
None => (0, Vec::new()),
};
if old.vault_key.is_some() {
let key = &items[1 + usize::from(old.vault_dir.is_some())];
let theirs = std::fs::read(book.join(KEYS_DIR).join(LEGACY_KEY_FILE)).ok();
if theirs.as_deref() != Some(&std::fs::read(key)?[..]) {
return Err(lacks("the vault's key".into()));
}
}
let legacy = root.join(LEGACY_DIR);
let stamp = chrono::Utc::now().format("%Y%m%d-%H%M%S");
let mut name = format!("{id}-old-{stamp}");
let mut n = 1;
while std::fs::symlink_metadata(legacy.join(&name)).is_ok() {
n += 1;
name = format!("{id}-old-{stamp}-{n}");
}
let report = PutAside {
head,
items: items.clone(),
blob_files,
left_behind,
vault_files,
vault_changed,
key_checked: old.vault_key.is_some(),
name: name.clone(),
moved: false,
};
if !confirm {
return Ok(report);
}
self.private_subdir(LEGACY_DIR, false)?;
let into = legacy.join(&name);
make_private_dir(&into)?;
// Where each came from, written down before anything moves, so
// a move cut short by a crash can be finished by hand.
let mut list = format!(
"wanderbooks put-aside: the old copies of the books \"{id}\", which hold \
everything they did (through event {}, {}).\nStarted {}.\n",
report.head.0,
report.head.1,
chrono::Utc::now().to_rfc3339()
);
for (k, item) in items.iter().enumerate() {
list.push_str(&format!("{} {}\n", k + 1, item.display()));
}
let listed = write_private_file(&into.join(PUT_ASIDE_LIST), list.as_bytes())
.and_then(|()| sync_dir(&into))
.and_then(|()| sync_dir(&legacy));
if let Err(e) = listed {
// Nothing has moved: no list of moves that never happened.
let _ = std::fs::remove_file(into.join(PUT_ASIDE_LIST));
let _ = std::fs::remove_dir(&into);
return Err(RootError::Adopt(format!(
"the list of where each came from couldn't be written ({e}): nothing was \
put aside"
)));
}
// Each in a numbered folder of its own: no two can meet, and a
// git history never sits beside a vault it was kept apart from.
let mut moved: Vec<(&PathBuf, PathBuf)> = Vec::new();
for (k, item) in items.iter().enumerate() {
let own = into.join((k + 1).to_string());
let dest = own.join(item.file_name().unwrap_or_default());
let done = make_private_dir(&own)
.map_err(|e| std::io::Error::other(e.to_string()))
.and_then(|()| rename_new(item, &dest));
if let Err(e) = done {
// Back where they were, the last moved first, never
// over anything that came since.
let mut stuck = Vec::new();
for (from, at) in moved.iter().rev() {
if rename_new(at, from).is_err() {
stuck.push(from.display().to_string());
}
}
if stuck.is_empty() {
// The list goes only once every numbered folder is
// gone, empty: else it stays to say what is where.
let emptied = (1..=items.len()).all(|k| {
match std::fs::remove_dir(into.join(k.to_string())) {
Ok(()) => true,
Err(e) => e.kind() == std::io::ErrorKind::NotFound,
}
});
if emptied {
let _ = std::fs::remove_file(into.join(PUT_ASIDE_LIST));
let _ = std::fs::remove_dir(&into);
}
return Err(RootError::Adopt(format!(
"{} couldn't be moved ({}): nothing was put aside",
item.display(),
e.kind()
)));
}
return Err(RootError::Adopt(format!(
"{} couldn't be moved ({}), and {} couldn't be put back: they are in \
the legacy folder as {name}, whole, with its {PUT_ASIDE_LIST} saying \
where each came from",
item.display(),
e.kind(),
stuck.join(", ")
)));
}
moved.push((item, dest));
}
// Every path has moved: the list saying so is a courtesy, and
// its failing (a full disk) doesn't undo or deny the move.
let _ = (|| -> std::io::Result<()> {
use std::io::Write;
let mut list = std::fs::OpenOptions::new()
.append(true)
.open(into.join(PUT_ASIDE_LIST))?;
writeln!(
list,
"Finished {}: each path above is in the folder of its number.",
chrono::Utc::now().to_rfc3339()
)?;
list.sync_all()
})();
let _ = sync_dir(&into);
let _ = sync_dir(&legacy);
for item in &items {
if let Some(parent) = item.parent() {
let _ = sync_dir(parent);
}
}
Ok(PutAside {
moved: true,
..report
})
})()
.map_err(|e| self.hide(e))
}
}
/// The list [`Root::put_aside`] leaves in the folder it fills: where each
/// numbered folder's path came from.
pub const PUT_ASIDE_LIST: &str = "put-aside.txt";
/// Rename `from` to `to`, never over something already at `to`.
fn rename_new(from: &Path, to: &Path) -> std::io::Result<()> {
#[cfg(target_os = "linux")]
{
use std::os::unix::ffi::OsStrExt;
let f = std::ffi::CString::new(from.as_os_str().as_bytes())?;
let t = std::ffi::CString::new(to.as_os_str().as_bytes())?;
// SAFETY: two NUL-terminated paths that live past the call.
let rc = unsafe {
libc::renameat2(
libc::AT_FDCWD,
f.as_ptr(),
libc::AT_FDCWD,
t.as_ptr(),
libc::RENAME_NOREPLACE,
)
};
if rc == 0 {
return Ok(());
}
let e = std::io::Error::last_os_error();
// A file system that can't refuse to replace: looked at, then moved.
if e.raw_os_error() != Some(libc::EINVAL) {
return Err(e);
}
}
if std::fs::symlink_metadata(to).is_ok() {
return Err(std::io::Error::from(std::io::ErrorKind::AlreadyExists));
}
std::fs::rename(from, to)
}
/// Whether the stored files under `old` (a books' `blobs/`) are all in
/// `new` at the same place with the same bytes: Ok((count, left behind))
/// if so, the left behind being what adoption leaves (hidden files, and
/// unfinished `<sha>.tmp` writes: never part of the books), else Ok(Err(the
/// first that isn't)). A folder that isn't there holds nothing.
fn blobs_held(old: &Path, new: &Path) -> Result<Result<(usize, Vec<PathBuf>), PathBuf>, RootError> {
let is_hex = |s: &str, n: usize| s.len() == n && s.bytes().all(|b| b.is_ascii_hexdigit());
let mut count = 0;
let mut left = Vec::new();
let mut stack = vec![PathBuf::new()];
while let Some(rel) = stack.pop() {
let entries = match std::fs::read_dir(old.join(&rel)) {
Ok(entries) => entries,
Err(e) if e.kind() == std::io::ErrorKind::NotFound && rel.as_os_str().is_empty() => {
return Ok(Ok((0, left)));
}
Err(e) => return Err(e.into()),
};
for entry in entries {
let entry = entry?;
let name = entry.file_name().to_string_lossy().into_owned();
let rel = rel.join(&name);
let kind = entry.file_type()?;
if kind.is_dir() {
stack.push(rel);
} else if kind.is_file()
&& (name.starts_with('.')
|| name.strip_suffix(".tmp").is_some_and(|sha| is_hex(sha, 64)))
{
left.push(rel);
} else if std::fs::read(new.join(&rel)).ok() == Some(std::fs::read(old.join(&rel))?) {
count += 1;
} else {
return Ok(Err(rel));
}
}
}
left.sort();
Ok(Ok((count, left)))
}
/// Whether the TIN vault `old` is held by the books' vault `new`: every
/// record there (a record the books sealed anew since is theirs, newer,
/// and listed), and the registry of files with full TINs only grown since.
/// Ok((count, changed)) if so, else Ok(Err(the first that isn't)).
fn vault_held(old: &Path, new: &Path) -> Result<Result<(usize, Vec<PathBuf>), PathBuf>, RootError> {
let mut count = 0;
let mut changed = Vec::new();
let mut names: Vec<_> = std::fs::read_dir(old)?
.map(|e| e.map(|e| e.file_name()))
.collect::<Result<_, _>>()?;
names.sort();
for name in names {
let rel = PathBuf::from(&name);
if !std::fs::symlink_metadata(old.join(&name))?.is_file() {
return Ok(Err(rel));
}
let mine = std::fs::read(old.join(&name))?;
let Ok(theirs) = std::fs::read(new.join(&name)) else {
return Ok(Err(rel));
};
if name == TIN_REGISTRY_FILE {
if !theirs.starts_with(&mine) {
return Ok(Err(rel));
}
} else if theirs != mine {
changed.push(rel);
}
count += 1;
}
Ok(Ok((count, changed)))
}
/// The vault's append-only list of files known to hold full TINs (kept by
/// the books server; ledger-mcp `tin_files`).
const TIN_REGISTRY_FILE: &str = "tin-files.sha256";
/// Whether the journal of the books in `book` holds the event `head`
/// names, `(seq, hash)`: its line of that seq hashes to that. Only whole
/// lines are read, so a line still being written is no matter.
fn journal_holds(book: &Path, head: &(u64, String)) -> Result<bool, RootError> {
if head.0 == 0 {
return Ok(true);
}
let raw = std::fs::read(book.join(JOURNAL_FILE))?;
let Some(end) = raw.iter().rposition(|&b| b == b'\n') else {
return Ok(false);
};
let (lines, _format) = crate::journal::verify_journal_and_format(&raw[..=end])?;
Ok(lines
.iter()
.find(|l| l.seq == head.0)
.is_some_and(|l| head_of(std::slice::from_ref(l)) == *head))
}
/// The old copies of books, for [`Root::put_aside`].
#[derive(Clone, Copy, Debug)]
pub struct OldCopies<'a> {
/// The data folder they were adopted from.
pub source: &'a Path,
/// Their TIN vault: each record must be the books' own, byte for byte.
pub vault_dir: Option<&'a Path>,
/// The vault's key: it must be the books' legacy key, byte for byte.
pub vault_key: Option<&'a Path>,
/// Other paths that hold them (settings file, git history, exports),
/// put aside as they are.
pub also: &'a [PathBuf],
/// The head a preview read: if it moved since, nothing is put aside.
pub expect_head: Option<&'a (u64, String)>,
}
/// What [`Root::put_aside`] would move, or moved.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct PutAside {
/// The old copy's head, `(seq, hash)`: the books hold it too.
pub head: (u64, String),
/// The paths that go aside, in order: the data folder, the vault, its
/// key, then the others. Each goes into the folder of its number.
pub items: Vec<PathBuf>,
/// The old copy's stored files, each found in the books, byte for byte.
pub blob_files: usize,
/// Files in its stored files' folder that were never part of the books
/// (hidden files, unfinished writes): left behind by adoption, going
/// aside with the rest.
pub left_behind: Vec<PathBuf>,
/// The vault's records, each found in the books.
pub vault_files: usize,
/// Records the books have sealed anew since (theirs is newer).
pub vault_changed: Vec<PathBuf>,
/// Whether the vault's key was found to be the books' own.
pub key_checked: bool,
/// The folder of their own in `legacy/` they go to.
pub name: String,
/// Whether they were moved (false: a preview, nothing moved).
pub moved: bool,
}
/// The source's journal, whole and verified: its bytes and lines.
+334 -1
View File
@@ -6,7 +6,7 @@ use chrono::NaiveDate;
use ledger_core::settings::{Change, Source};
use ledger_core::{AccountType, Currency, Money, Side};
use ledger_store::root::{
AdoptOptions, BookId, LOCATION_FILE, MARKER_FILE, Root, RootError, default_root,
AdoptOptions, BookId, LOCATION_FILE, MARKER_FILE, OldCopies, Root, RootError, default_root,
platform_data_dir,
};
use ledger_store::{JOURNAL_FILE, JournalLine, Snapshot, Store, StoreError};
@@ -1348,3 +1348,336 @@ fn a_restore_never_moves_books_something_holds() {
.collect();
assert!(left.is_empty(), "{left:?}");
}
/// Old books beside their vault, key, settings file and git history, as an
/// old books dir kept them, adopted into a new root as "acme" (with their
/// vault and key unless `vault` is false): the home, the root, and the old
/// dir.
fn adopted_old_dir(tag: &str, vault: bool) -> (tempfile::TempDir, Root, tempfile::TempDir) {
let home = tmp(tag);
let (root, _) = Root::setup(home.path(), None, None).unwrap();
let old = tmp(&format!("{tag}-old"));
let data = old.path().join("books-data");
let head = source_books(&data, true).head();
std::fs::create_dir(old.path().join("vault")).unwrap();
std::fs::write(old.path().join("vault/tin-7.bin"), b"sealed record").unwrap();
std::fs::write(
old.path().join("vault.key"),
b"0123456789abcdef0123456789abcdef",
)
.unwrap();
std::fs::write(old.path().join("wanderbooks.toml"), b"[viewer]\n").unwrap();
std::fs::create_dir_all(old.path().join(".git/objects")).unwrap();
std::fs::write(old.path().join(".git/objects/ab"), b"history").unwrap();
let (vault_dir, vault_key) = (old.path().join("vault"), old.path().join("vault.key"));
root.adopt(
&data,
&id("acme"),
&AdoptOptions {
settings: None,
vault_dir: vault.then_some(vault_dir),
vault_key: vault.then_some(vault_key),
expect_head: Some(head),
},
)
.unwrap();
(home, root, old)
}
/// The paths of an old dir from [`adopted_old_dir`] beside its data folder.
struct OldPaths {
data: PathBuf,
vault: PathBuf,
key: PathBuf,
also: Vec<PathBuf>,
}
fn old_paths(old: &Path) -> OldPaths {
OldPaths {
data: old.join("books-data"),
vault: old.join("vault"),
key: old.join("vault.key"),
also: vec![old.join("wanderbooks.toml"), old.join(".git")],
}
}
fn copies<'a>(p: &'a OldPaths, expect_head: Option<&'a (u64, String)>) -> OldCopies<'a> {
OldCopies {
source: &p.data,
vault_dir: Some(&p.vault),
vault_key: Some(&p.key),
also: &p.also,
expect_head,
}
}
#[test]
fn old_copies_go_aside_whole_once_the_books_hold_them() {
let (_home, root, old) = adopted_old_dir("aside", true);
let p = old_paths(old.path());
let before = snapshot_files(old.path());
let book_before = snapshot_files(&root.book_dir(&id("acme")));
// A preview: what would move and where, what was checked, and nothing
// moved or written.
let preview = root
.put_aside(&id("acme"), &copies(&p, None), false)
.unwrap();
assert!(!preview.moved);
assert!(preview.name.starts_with("acme-old-"), "{}", preview.name);
assert_eq!(preview.items.len(), 5);
assert_eq!(
(preview.blob_files, preview.vault_files, preview.key_checked),
(2, 1, true)
);
assert_eq!(snapshot_files(old.path()), before);
assert!(!root.path().join("legacy").join(&preview.name).exists());
let done = root
.put_aside(&id("acme"), &copies(&p, Some(&preview.head)), true)
.unwrap();
assert!(done.moved);
assert_eq!(done.head, preview.head);
// Gone from the old folder; every byte of it in a private folder of its
// own in legacy/, each path in the numbered folder of its own.
assert_eq!(std::fs::read_dir(old.path()).unwrap().count(), 0);
let into = root.path().join("legacy").join(&done.name);
assert_eq!(mode(&into), 0o700);
let order = [
"books-data",
"vault",
"vault.key",
"wanderbooks.toml",
".git",
];
let moved: Vec<(PathBuf, Vec<u8>)> = snapshot_files(&into)
.into_iter()
.filter(|(f, _)| f.file_name().unwrap() != "put-aside.txt")
.map(|(f, b)| {
let rel = f.strip_prefix(&into).unwrap();
let mut parts = rel.components();
let k: usize = parts
.next()
.unwrap()
.as_os_str()
.to_str()
.unwrap()
.parse()
.unwrap();
let rest = parts.as_path();
assert!(rest.starts_with(order[k - 1]), "{}", rel.display());
(old.path().join(rest), b)
})
.collect();
let mut moved = moved;
moved.sort();
assert_eq!(moved, before);
// Where each came from, written down, and the move said finished.
let list = std::fs::read_to_string(into.join("put-aside.txt")).unwrap();
assert_eq!(mode(&into.join("put-aside.txt")), 0o600);
for (k, item) in done.items.iter().enumerate() {
assert!(
list.contains(&format!("{} {}\n", k + 1, item.display())),
"{list}"
);
}
assert!(list.contains("Finished"), "{list}");
// The adopted books are untouched.
assert_eq!(snapshot_files(&root.book_dir(&id("acme"))), book_before);
Store::open(&root.book_dir(&id("acme")))
.unwrap()
.verify()
.unwrap();
// Nothing is left to put aside a second time.
assert!(matches!(
refused(root.put_aside(&id("acme"), &copies(&p, None), true)),
RootError::Adopt(_)
));
}
#[test]
fn old_copies_stay_unless_the_books_hold_everything_they_do() {
let empty_legacy = |root: &Root| {
assert_eq!(
std::fs::read_dir(root.path().join("legacy"))
.unwrap()
.count(),
0
);
};
// Written after the adoption: the books no longer hold its journal.
let (_home, root, old) = adopted_old_dir("aside-newer", true);
let p = old_paths(old.path());
{
let mut s = Store::open(&p.data).unwrap();
s.add_account("6100", "Postage", AccountType::Expense, None, false)
.unwrap();
}
let before = snapshot_files(old.path());
for confirm in [false, true] {
let e = refused(root.put_aside(&id("acme"), &copies(&p, None), confirm));
assert!(e.to_string().contains("its journal through event"), "{e}");
// Nor do books of another name.
let e = refused(root.put_aside(&id("other"), &copies(&p, None), confirm));
assert!(e.to_string().contains("no books called"), "{e}");
}
assert_eq!(snapshot_files(old.path()), before);
empty_legacy(&root);
// Adopted without the vault: the vault and its key aren't the books'.
let (_home, root, old) = adopted_old_dir("aside-novault", false);
let p = old_paths(old.path());
let e = refused(root.put_aside(&id("acme"), &copies(&p, None), true));
assert!(e.to_string().contains("the vault record tin-7.bin"), "{e}");
let key_only = OldCopies {
vault_dir: None,
..copies(&p, None)
};
let e = refused(root.put_aside(&id("acme"), &key_only, true));
assert!(e.to_string().contains("the vault's key"), "{e}");
// A stored file the books never got.
let (_home, root, old) = adopted_old_dir("aside-blob", true);
let p = old_paths(old.path());
let blobs = p.data.join("blobs");
std::fs::write(blobs.join("stray.bin"), b"a file the books never named").unwrap();
let e = refused(root.put_aside(&id("acme"), &copies(&p, None), true));
assert!(e.to_string().contains("the stored file stray.bin"), "{e}");
empty_legacy(&root);
// The old copy moved on since the preview.
let (_home, root, old) = adopted_old_dir("aside-head", true);
let p = old_paths(old.path());
let stale = (1, "0".repeat(64));
let e = refused(root.put_aside(&id("acme"), &copies(&p, Some(&stale)), true));
assert!(
e.to_string().contains("changed since it was looked at"),
"{e}"
);
empty_legacy(&root);
}
#[test]
fn old_copies_held_open_linked_named_twice_inside_the_root_or_missing_stay() {
let (_home, root, old) = adopted_old_dir("aside-refused", true);
let p = old_paths(old.path());
let before = snapshot_files(old.path());
{
// Held open for writing.
let _held = Store::open(&p.data).unwrap();
for confirm in [false, true] {
let e = refused(root.put_aside(&id("acme"), &copies(&p, None), confirm));
assert!(
matches!(e, RootError::Store(_, StoreError::Locked(_))),
"{e}"
);
}
}
let link = old.path().join("link-to-git");
std::os::unix::fs::symlink(old.path().join(".git"), &link).unwrap();
let inside_data = vec![p.data.join(JOURNAL_FILE)];
let twice = vec![p.also[0].clone(), p.also[0].clone()];
let in_root = vec![root.path().join("books")];
let missing = vec![old.path().join("no-such-thing")];
let linked = vec![link.clone()];
for bad in [inside_data, twice, in_root, missing, linked] {
for confirm in [false, true] {
let c = OldCopies {
also: &bad,
..copies(&p, None)
};
let e = refused(root.put_aside(&id("acme"), &c, confirm));
assert!(matches!(e, RootError::Adopt(_)), "{bad:?}: {e}");
// No error names the root.
assert!(
!e.to_string().contains(&*root.path().to_string_lossy()),
"{e}"
);
}
}
std::fs::remove_file(&link).unwrap();
assert_eq!(snapshot_files(old.path()), before);
}
#[test]
fn a_put_aside_cut_short_puts_back_what_it_moved() {
let (_home, root, old) = adopted_old_dir("aside-cut", true);
let mut p = old_paths(old.path());
// The last path's folder lets nothing out: the others have moved by
// then, and go back.
let shut = old.path().join("shut");
std::fs::create_dir(&shut).unwrap();
std::fs::write(shut.join("notes.txt"), b"notes").unwrap();
p.also.push(shut.join("notes.txt"));
std::fs::set_permissions(&shut, std::fs::Permissions::from_mode(0o500)).unwrap();
let before = snapshot_files(old.path());
let e = refused(root.put_aside(&id("acme"), &copies(&p, None), true));
std::fs::set_permissions(&shut, std::fs::Permissions::from_mode(0o700)).unwrap();
assert!(e.to_string().contains("nothing was put aside"), "{e}");
assert_eq!(snapshot_files(old.path()), before);
let legacy = root.path().join("legacy");
assert_eq!(std::fs::read_dir(&legacy).unwrap().count(), 0);
}
#[test]
fn what_adoption_left_behind_and_what_the_books_sealed_anew_still_go_aside() {
let (_home, root, old) = adopted_old_dir("aside-left", true);
let p = old_paths(old.path());
// Never part of the books, so adoption left them: a hidden file and an
// unfinished write.
let blobs = p.data.join("blobs");
let shard = std::fs::read_dir(&blobs)
.unwrap()
.map(|e| e.unwrap().path())
.find(|d| d.is_dir())
.unwrap();
std::fs::write(blobs.join(".DS_Store"), b"finder").unwrap();
std::fs::write(shard.join(format!("{}.tmp", "a".repeat(64))), b"half").unwrap();
// The books sealed a record anew since, and grew the registry of files
// with full TINs.
let vault = root.book_dir(&id("acme")).join("vault");
std::fs::write(vault.join("tin-7.bin"), b"sealed anew").unwrap();
std::fs::write(p.vault.join("tin-files.sha256"), b"aa\n").unwrap();
std::fs::write(vault.join("tin-files.sha256"), b"aa\nbb\n").unwrap();
let preview = root
.put_aside(&id("acme"), &copies(&p, None), false)
.unwrap();
assert_eq!(preview.left_behind.len(), 2, "{:?}", preview.left_behind);
assert_eq!(preview.vault_changed, vec![PathBuf::from("tin-7.bin")]);
assert_eq!(preview.vault_files, 2);
let done = root
.put_aside(&id("acme"), &copies(&p, Some(&preview.head)), true)
.unwrap();
let into = root.path().join("legacy").join(&done.name);
assert!(into.join("1/books-data/blobs/.DS_Store").is_file());
assert_eq!(
std::fs::read(into.join("2/vault/tin-7.bin")).unwrap(),
b"sealed record"
);
}
#[test]
fn a_registry_rewritten_or_a_vault_given_unchecked_is_refused() {
let (_home, root, old) = adopted_old_dir("aside-registry", true);
let p = old_paths(old.path());
let vault = root.book_dir(&id("acme")).join("vault");
std::fs::write(p.vault.join("tin-files.sha256"), b"aa\n").unwrap();
std::fs::write(vault.join("tin-files.sha256"), b"zz\n").unwrap();
let e = refused(root.put_aside(&id("acme"), &copies(&p, None), true));
assert!(e.to_string().contains("tin-files.sha256"), "{e}");
// The vault or its key among the other paths: never moved unchecked.
for plain in [p.vault.clone(), p.key.clone()] {
let also = vec![plain];
let c = OldCopies {
source: &p.data,
vault_dir: None,
vault_key: None,
also: &also,
expect_head: None,
};
let e = refused(root.put_aside(&id("acme"), &c, false));
assert!(
e.to_string().contains("--vault DIR --vault-key FILE"),
"{e}"
);
}
assert!(p.vault.is_dir() && p.key.is_file());
}
+18
View File
@@ -534,6 +534,24 @@ the keyring lost and saying when an agent refuses, the two lease
intents), `one_executable.rs` (the scaffold gone, and saying so),
`spec_kept.rs` (`isolation` is `same_user (advisory)`). Two review rounds.
**Slice 9, the cutover's put-aside.** `wanderbooks books put-aside ID
DATA-DIR [--vault DIR --vault-key FILE] [PATH...]` (root INVARIANT 8)
moves the old copies of adopted books whole into numbered folders of
their own in `legacy/`, a list of where each came from written first.
`ledger-store/tests/root.rs` (seven: a preview moves nothing; every byte
moved and the list written; refused while the books lack its journal, a
stored file, a vault record or the key, or when its head moved since the
preview; refused while held open, for a link, a path named twice, inside
the root or missing, and for a vault or key given as a plain path;
what adoption left behind and records the books sealed anew go aside
too; a registry rewritten is refused; a move cut short puts back what it
moved), `ledger-cli/tests/open_while_held.rs` (from the command line).
Three review rounds of data safety alone (12 findings, then 5, then 3
minor; all fixed but one: a vault record the old journal never named, a
crash between sealing and journaling, would be listed as the books'
newer copy; nothing is lost, the old vault going aside whole; hardening
for P7).
**Tests that mutation found, before the pause.** Pass 1 on slices 3 to 6
(each crate's own tests, the viewer left out): `ledger-store` 385 mutants
(44 missed), `ledger-package` 137 (38), `ledger-mcp` 497 (247),
+3 -1
View File
@@ -1334,7 +1334,9 @@ These are the only open owner questions. Ask them just before the move, one at a
**Q-3. After the move, should the old copies of the books leave the project folder?** The project folder still holds the old journal and receipts (`books-data/`), the old sealed tax ID numbers (`vault/`), and the folder's git history, which contains the journal and receipts too. **Recommended: move them into the private folder's legacy area.** That is what "hidden away" means; otherwise an agent working in the project folder can still read the old copies. Nothing is deleted.
**Crumbs to clear at the cutover** (owner, 2026-10-19: "migrate the first company books into the new scheme, data directory, clear out the crumbs"). After the adoption verifies, nothing of the old arrangement may be left where an agent or a later setup trips on it; every item is moved per Q-3 (into the root's `legacy/`, nothing deleted) or removed if it holds no books data:
*Answered 2026-10-20 (the owner), one question at a time:* **Q-3: move them aside** (into the root's `legacy/`, nothing deleted). **Q-1's addresses:** the git remote is a new private repository on his server, `ssh://git@git.swaits.com:2222/swaits/fernhill-books-backup.git` (made private, with his go-ahead), and the folder is `~/Sync/wanderbooks-backups` (his Syncthing folder: a local disk, copied to his other devices; only this machine writes it).
**Crumbs to clear at the cutover** (owner, 2026-10-19: "migrate the first company books into the new scheme, data directory, clear out the crumbs"). After the adoption verifies, nothing of the old arrangement may be left where an agent or a later setup trips on it; every item is moved per Q-3 (into the root's `legacy/`, nothing deleted, by `wanderbooks books put-aside ID DATA-DIR [PATH...]`, added at the cutover: only once the adopted books hold everything the old copy holds, only while nothing holds it open, only on the root's own disk, and a move cut short puts back what it moved) or removed if it holds no books data:
- `books-data/` (journal, blobs, database, lock, `gui.log`) and `vault/`, and the project folder's git history holding them (Q-3);
- the v1 vault key outside the books, `~/.config/wanderbooks/vault.key` (adoption copies it to the book's `keys/legacy.key`; the old one goes to `legacy/` with the vault);
- `wanderbooks.toml` (its settings are in the books since P1; only `[viewer] open_at_start` remains, which the app's own preferences replace);