Chronos/DBWaitlist

Changelog / Chronos DB

Chronos DB 0.4.2

Joins, merges and memory, from running BranchBench (Columbia's benchmark of agent workflows on branching databases): joins on several columns and joins too big for memory, merges where both sides changed a table's columns, a merge that counts its conflicts, `DIFF ... SUMMARY` and `UNDO DIFF`, and a memory limit for every query

Beta26 changesDownloads

Shipped with Chronos Gateway 0.1.1, Chronos Gateway 0.1.0.

Joins, merges and memory, from running BranchBench (Columbia's benchmark of agent workflows on branching databases): joins on several columns and joins too big for memory, merges where both sides changed a table's columns, a merge that counts its conflicts, DIFF ... SUMMARY and UNDO DIFF, and a memory limit for every query.

Changed from 0.4.1: every statement has a memory limit. A statement past CHRONOS_QUERY_MEM (default a quarter of the machine's memory; 0 for none, as before) fails alone with 53200, and statements together are held to twice that, where a runaway join used to take the server down. See the entry below.

  • Joins on several columns too big for memory hash on all of them too. A hash join whose other side outgrows CHRONOS_WORK_MEM goes to disk, split into files by the join key; that split, and each file's hash table, used the first equality alone. On TPC-C's (w_id, d_id, ...) keys that put a warehouse's rows in one file and paired each with all the others there: CH Q3 and Q18 took 17 minutes at five warehouses (16 MB of work memory: at one), where they now take 0.6 s. Found by BranchBench.
  • Joins take the equalities their ON clauses imply. customer c JOIN new_order n ON n.w = c.w AND n.d = c.d JOIN orders o ON o.w = n.w AND o.d = n.d AND o.id = n.o AND o.c = c.id (TPC-C's CH Q3) joined customer to new_order first, as written: every new order of a district met every customer of it, and the rows held grew past 12 GB at one warehouse (at five, the server was killed for memory). Equal columns now carry across (o.w = n.w and n.w = c.w give o.w = c.w, between columns of one type and collation), and the next table joined is the one linked by its whole key, then by the most columns: orders comes second. Q3 at one warehouse: 0.35 s and 850 MB. Found by BranchBench.
  • Joins on several columns hash on all of them. A join on (w_id, d_id, o_id) built its hash table on the first equality alone and checked the others row by row, so where that column has few values (one warehouse) every row probed the whole warehouse: TPC-C's order_line JOIN orders at one warehouse took over two minutes, orders LEFT JOIN new_order 4 s, customer JOIN orders 27 s (Postgres: 74 ms, 3 ms, 11 ms). An in-memory hash join now keys on every a.x = b.y of the same kind; the same three take 56 ms, 14 ms and 15 ms. A join bigger than CHRONOS_WORK_MEM (on disk) still hashes on one column. Found by BranchBench, whose TPC-C consistency checks are such joins.
  • MCP tells agents to settle conflicts by column first. The instructions used to say "decide which side wins and call merge again with resolve set to ours or theirs", and agents did, overwriting main's changes to columns they never touched. They now say that merging the same way again fails the same way, and give the safest order: merge_preview, then by_columns=true (with resolve to pick a side only for the columns both changed), then rows for single rows, and resolve alone, which takes whole rows, last. A conflict from merge says the same, also when the call already had by_columns, rows or the like (before, only a plain merge got the hint), and for a conflict with an attached Postgres. The resolve and by_columns descriptions and the TypeScript and Python clients' merge docs say what each does to the other side's columns.
  • The log ships to cloud storage as it is written. With --remote, a database's log used to stay on local disk until the next checkpoint (every 64 MB of log), so losing the machine lost everything since. It now goes to the bucket in chunks (wal-<segment>-<offset> objects) within wal_shipping_delay (1 s) of being written, and a --takeover from an empty folder on another machine replays it. ALTER SYSTEM SET wal_shipping = sync makes a commit to main or a merge return only once its log is in the bucket, so a takeover loses no acknowledged write; off ships only at checkpoints, as before. A takeover fences the old writer at the end of the shipped log: its next upload fails and it ships nothing more. Checkpoints delete the shipped log the bucket no longer needs. Also new: gs://bucket/prefix remotes (Google Cloud Storage). See operations and the design.
  • USE WORLD 'name@when' switches a session to a world as it was. The docs said use world 'main@2026-09-20 10:00' reads the past for the rest of the session, as connecting to the database main@... does, but USE WORLD (and SWITCH WORLD) only took a name or an ID and failed with branch "main@..." does not exist. It now takes 'name@when' too: read only (writes fail with 25006), 22023 for a time outside the history window, 3D000 for a world that doesn't exist. Found by BranchBench, whose commit refs read a branch at a commit.
  • DIFF sees a column's new size. ALTER TABLE t ALTER COLUMN v TYPE varchar(10) on a varchar(5) (or numeric(10,2) to numeric(12,2), integer to smallint, text to varchar(20)) showed in DIFF as an alter table with no column named in columns, and DIFF ... AS SQL left the change out, so running it elsewhere kept the old size. Both compare the whole type now, size included: v: varchar(5) -> varchar(10), and alter table t alter column v type varchar(10);. Found by BranchBench, whose schema diffs read columns.
  • Attach: rows Postgres triggers write to other tables come into main with the merge. A trigger on leads that keeps accounts.lead_count, writes an audit row, or deletes a lead's notes (and a foreign key's ON DELETE CASCADE) left those rows as main had them until the next refresh, and a world that changed one of them meanwhile got a conflict. The write-back now counts what its transaction wrote per attached table (deferred triggers included), and the tables written besides the merge's own rows are refreshed alone right after it; after a crash, recovery does that refresh.
  • REINDEX DATABASE rebuilds the database's indexes. It reported success and rebuilt nothing (it was read as REINDEX SYSTEM). It now rebuilds every index of the tables and materialized views you own, moves them with REINDEX (TABLESPACE t) DATABASE, and, as in Postgres, can't run inside a transaction block. REINDEX SYSTEM still rebuilds only the catalogs'.
  • Temporary tables follow temp_tablespaces. A temporary table or index made without TABLESPACE went where default_tablespace said, so a CREATE INDEX on one failed with 42501 when that named a tablespace you can't create in. It now goes to the first tablespace of temp_tablespaces you may create in, else the database's default, as in Postgres. CREATE TEMP TABLE ... TABLESPACE t works (it said tablespace "t" does not exist), and DROP TABLESPACE refuses one that holds the session's temporary tables or indexes.
  • BY COLUMNS USING OURS (or THEIRS) settles a row both sides changed column by column. The side named wins only the columns both changed; a column only the other side changed keeps its value. It used to take the named side's whole row, putting back the fork's value in columns only the other side changed (an email main changed went back to what it was). The same goes for USING OURS|THEIRS into a table set to merge_by_columns. USING OURS without BY COLUMNS still takes whole rows. Under a merge policy, only a column ours actually overwrites counts as an overwrite.
  • MERGE ... DRY RUN says what each side changed, by column. A new columns column (HTTP: columns) gives {here, there, both} for each row both sides changed, and a conflict's detail says what USING OURS / THEIRS would discard against BY COLUMNS with that side (both changed age; only here: name; only the parent: email. USING OURS overwrites the parent's age, email, BY COLUMNS USING OURS only age; ...). Settled rows explain themselves the same way, instead of both sides changed this row.
  • SHOW MERGE OUTCOMES counts a merge's conflict with Postgres as a conflict, not a stale read. In an attached folder, a merge refused because Postgres changed its rows was counted under stale_reads, and so were refusals because Postgres was busy (a lock timeout) or a world's scope changed meanwhile. Postgres conflicts count as conflicts now, and the others aren't counted.
  • Attach: a refresh takes columns added in Postgres. A migration that adds a column (ALTER TABLE leads ADD COLUMN status text NOT NULL DEFAULT 'new') no longer stops a table's refresh and every merge into it until you attach again: the next chronos refresh (or serve --refresh-every, or HTTP refresh) adds the column to main's table with Postgres's type and default, then main takes its values, and says so (added). Merges into the table are refused only until that refresh. A world forked before the column existed leaves Postgres's values in it alone for rows it changes, even settled USING OURS. Dropped, renamed or retyped columns still need a new attach.
  • Attach: a tenant's read of a big table no longer holds every merge. A world that checks its reads and ran select ... where org_id = 42 and status = 'open' on an attached table of more than 100,000 rows was always held for a person ("read leads more broadly than can be checked against Postgres"), as the check read the table whole from Postgres. The check now asks Postgres only for the rows the scan's conditions can match, at the fork and now, so the app writing other tenants' rows doesn't hold it, and a matching row changed or added there still does. A scan through an index keeps its conditions as a scan without one does, rather than the index entries it read, inside Chronos too: a row its other conditions rule out (status = 'won') no longer holds it when it changes.
  • Attach: a row a world inserts after Postgres added columns gets Postgres's defaults. A world forked before a column was added in Postgres sent it as the default Chronos worked out when the column came over (or NULL, for one added bare): now() was that moment's, random() NULL. The write-back now leaves the columns the world never had out of its INSERT, so Postgres fills them as for its own rows, and main takes the row back as Postgres has it.
  • Hourly limits count per tenant for agents held to one. rows_per_hour, deletes_per_hour and effects_per_hour counted each agent's merges on their own, so a product that makes an agent per run (CREATE AGENT run_123 WITH (tenant = 'org_id = 42', valid_for = '1 hour'), as the agent gateway does) started the hour again with every run. An agent made WITH (tenant = ...) now shares one count with every agent made for that tenant; a merge over it waits for a person, saying merges for tenant "org_id" = 42 deleted 100 in the last hour. Agents with no tenant are counted alone, as before.
  • Every query has a memory limit. Only an agent with max_memory_mb had one, so a runaway join by anyone else held rows until the operating system killed the server and every session with it (BranchBench's S4 took one to 55 GB). Now any statement past CHRONOS_QUERY_MEM (default a quarter of the machine's memory, 0 for none) fails alone with 53200, and statements together are held to twice that. A hash join's probe checks the limit, and statement_timeout and cancels, as it makes rows, on every core: a 120 s timeout used to land at 160 s or later. A join that held 51 GB within 13 s now fails in a second with the server up. An agent's count(*) over a join bigger than its max_memory_mb is now refused too: it holds the joined rows while counting.
  • Attach: merges keep their connection to Postgres. Every merge into main of an attached folder connected to Postgres anew, with a TLS handshake and a SCRAM login: 4 to 5 of a small merge's 13 or so round trips, so a 1-row merge from Mumbai to Neon in Singapore (70 ms away) took 0.9 to 1 s. The folder now keeps one connection between merges, and refreshes, dry runs, CHECK READS and the ids reserved from Postgres's sequences use it too. Each use still starts its own transaction with its settings SET LOCAL, and the connection is kept only when no transaction is left open. One the server closed meanwhile (a restart, a pooler, pg_terminate_backend) fails at the merge's first statement, before anything is written, and the merge connects again once; a connection lost after that is the merge's error, as before. A new CHRONOS_SOURCE_URL (or CHRONOS_SOURCE_FILE) is used from the next merge on, and the connection closes after 5 minutes unused, so a Neon or Supabase database can still scale to zero.
  • COPY FREEZE is refused once the COPY has begun, as Postgres refuses it. A COPY ... FROM STDIN FREEZE into a table the transaction didn't create or truncate (or a partitioned one) was refused before the CopyInResponse; Postgres refuses it after, so the client sends its data, which is dropped. With the refusal first, psql read a script's data lines after the COPY as statements. Found by Postgres's copy2 regression file, which the nightly regress run now runs with psql 16.14, the version of its files.
  • Both sides adding columns to one table merge. A world and its parent that each changed a table's columns (one adds tier, the other loyalty) were refused with both sides changed table t itself, whatever the columns: merge one side, redo the other. They now merge column by column: each side's new and dropped columns, the world's after the parent's, every row read through the merged table (the other side's new columns null or their default), in a merge, a rebase and a dry run. The same column changed by both, or two columns of one name, is still a conflict. Found by BranchBench's S4, where a dev world moves its column out of the way of the spine's (add, copy, drop) and its rebase was still refused.
  • A hash join holds only the columns it reads of its other side. That side's rows kept a value's room for every column of their table, the ones nothing reads after the table's own conditions included (as nulls), so a wide table went to disk (CHRONOS_WORK_MEM) long before its rows needed to. CH-benCHmark's Q3 and Q18 read 4 of order_line's 10 columns: at five warehouses its 1.5 million rows went to disk, and the queries took 3.0 s and 5.3 s; held narrow they stay in memory and take 0.7 s and 1.4 s, as with CHRONOS_WORK_MEM=4GB before. A join that does go to disk writes the narrow rows too. Found by BranchBench, where these two queries took 73% of S4's TPC-C spine.
  • A merge says how many rows both sides changed, and how it settled them. MERGE WORLD and REBASE WORLD that settle rows both sides changed (USING, BY COLUMNS, RESOLVE) send one NOTICE with the counts: 3 rows both sides changed: 1 combined by columns, 0 settled as x has them, 1 as main has them, 1 as RESOLVE gave them. A client that counts conflicts used to run a DRY RUN first and then merge again; the count now comes with the merge. Found by BranchBench, whose merges count conflicts.
  • DIFF ... SUMMARY counts a diff without reading it. DIFF WORLD a [TO b] SUMMARY gives one row per table and kind of change (table, change, rows), and the changes to tables and other objects as DIFF lists them, without turning any row into JSON. A client that wanted only the counts of a 10,000-row diff used to read every row (0.5 s). Found by BranchBench, whose diff counts rows.
  • UNDO DIFF a TO b reverts a change in one statement. In the session's world it puts back what turned a into b (two worlds, or one world at two moments, as DIFF WORLD a TO b lists it), indexes rebuilt and constraints checked as in a merge. A row changed since is named in a 40001 and nothing is undone, as for UNDO AGENT; SKIP CHANGED leaves those, and EXCEPT TABLES (log) leaves tables alone. A revert used to be DIFF ... AS SQL, its statements then run one by one: 2 s for 10,000 rows, and a row changed since was overwritten without a word. Found by BranchBench, whose revert undoes a commit.
  • LangGraph keeps its state in a world, and pytest gets one per test. clients/python/chronos_langgraph.py has ChronosSaver, a LangGraph checkpointer, and ChronosStore, its long-term memory store, both on a branch: builder.compile(checkpointer=ChronosSaver(world), store=ChronosStore(world)). A run's checkpoints sit beside the rows it wrote, so the diff a reviewer reads shows both, world.merge() lands them together and world.discard() drops both, and forking the world forks the thread. LangGraph's own checkpointer conformance suite passes (66 tests; prune and delete_for_runs aren't offered). The store's search(query=...) is Chronos's full-text search, typos allowed; it has no vector index. clients/python/chronos_pytest.py adds chronos_world, a fork of main per test, discarded after it (pytest -p chronos_pytest). Both run in the drivers build (cloudbuild/drivers.yaml). LangGraph's persistence keys threads by thread_id with nothing per run; this was audit item A5.

The macOS builds need macOS 13 or later, as for 0.2.0.

Chronos DB 0.4.2 · Chronos