Scheduled consolidation: the dream cycle¶

particles memory consolidate runs the cross-session memory maintenance passes in one verb, in a fixed order, under the existing cost caps, so "memory that tends itself" becomes a crontab line instead of four verbs an operator has to remember to run.

The pass list, in order:

Extract catch-up (LLM): extract PENDING snapshots, least-tried first and then oldest first, capped at consolidation.max_pending_entries per run. A capped run discloses the remainder and how long it has waited (12 remain, oldest waiting since 2026-07-19; the next run continues). See When the backlog outgrows the cap.
Reconcile (LLM, capped): the cross-entry document-supersession sweep. Each candidate pair costs one replacement-signal probe, spent highest-similarity-first under consolidation.max_reconcile_probes; a truncated run discloses "probed X of Y candidate pairs". Skipped (disclosed) on degraded runs.

Pass 2c, re-anchor (LLM, capped): when an update retires a claim about the user's circumstances ("the user lives in Delhi, in Lajpat Nagar"), a claim from the same passage may have relied on it ("Sandeep's Curry House is a ten-minute walk from the user's flat"). After the move that claim still reads as current and is false. The pass asks one probe per retirement which such claims depended on the old state, has a second reading on llm.verification confirm each verdict and the restatement it wrote, and replaces the claim with a dated restatement ("… was a ten-minute walk from the flat in Lajpat Nagar, Delhi, where the user lived, as of 2026-09-04"). The original is kept, SUPERSEDED with reason SUPERSEDED_BY_REANCHOR. A restatement never changes another belief: one that would contradict a standing claim, or whose checks could not run, is not written, and the claim becomes a stale_basis card in the curation queue instead (affirm keeps it, supersede or retract replaces it). The pass resumes from where the last run stopped, at most consolidation.reanchor.max_retirements_per_run retirements a night (default 20); particles reconcile --dependents runs it by hand. The report's line reads, for example, re-anchor 2 update(s) examined, 1 dependent claim(s) restated. Skipped (disclosed) on degraded runs. 3. Census (LLM, capped + scoped, weekly): the audit's contradiction probe + duplicate scan, capped at audit.max_contradiction_probes and scoped to what changed since the previous census (see Delta scope). It runs on its own cadence, not every night (see The census cadence). As in particles audit, a flagged pair counts only once a second reading on llm.verification confirms it (audit.verify_contradictions, at most audit.max_contradiction_verifications readings). The probes go out as one batch; the second readings follow one at a time, at full price. The report's contradictions line shows both counts, for example contradictions 3 [first pass flagged 22 claim pairs, a second reading confirmed 4]. The headline counts disagreements: confirmed pairs that share a claim count once.

Every answer the probe and the second reading give is kept in the probe-verdict ledger, keyed by both claims' wording and the prompt's version. A pair already answered no is not asked again while neither claim nor the prompt has changed, and it does not count against the cap. The report discloses the saving, for example contradiction probe: 812 pair(s) skipped as previously cleared. The same ledger serves the update sweep (particles reconcile --updates), whose report gains a previously_cleared count.

Pass 3b, disclosure (zero-LLM): a contradiction the second reading confirmed between claims from two different sources becomes one open INCONSISTENCY record per disagreement. The session-start digest then flags every claim in it, so the agent's next session is warned before it repeats either one. Nothing else changes: no claim's status or confidence moves, and the record is resolved in particles review like any other. The pass also closes a record by itself when a side is no longer stated by any current source (for example, the note that was wrong is corrected), and replaces one whose group of claims changed; each close writes an INCONSISTENCY_CLOSED event. At most consolidation.contradiction_disclosure.max_per_run new records open per run (default 10), and the rest wait for the next run. The report's line reads, for example, contradiction disclosure: opened 1 inconsistency (2 claims) for the agent (cap 10); 0 waiting. Claims from one note are never opened this way, and neither is a pair whose claims no single project sees. particles audit reports what is already disclosed and opens nothing. 4. Curation-queue refresh: the queue, computed from the same card collection the census already paid for; the report ends with the morning's worklist. On a night without a census it ranks the last census's stored collection instead, with every card snoozed, resolved or retired since then removed. 5. Utility mining (LLM, bounded): the pass over harvested session transcripts. Only beliefs a session was shown are candidates, and one is credited only when an LLM judge rules that the session's actions applied it. A literal token match nominates a candidate; it no longer credits one by itself, so a structural-only run credits nothing. The judge spends one shared per-run budget (utility.mining.max_behavioural_calls, 150 by default) across all sessions, and exhaustion is disclosed ("use-judge budget exhausted after N of M sessions"). The budget is spent in session order, and every session's judge requests go out together as one batch, so the pass waits on one batch turnaround however many sessions it mines. 6. Projection re-render (zero-LLM): the MEMORY.md render-splice cycle, via the same harvest-then-render tail the SessionEnd hook uses. It covers only the memory directories whose installed Claude Code hooks name the store being consolidated. A run against any other store (a scratch database, a test) harvests and renders none of them, and the run record says why. 7. Closure measure (zero-LLM, read-only): the share of ACTIVE beliefs that carry the contested badge, counted over the whole store on every run, and the lifecycle transitions since the previous run split into autonomous and gesture. See The closure measure. 8. Record + report: one CONSOLIDATION_RUN operator event per run, readable afterwards through Auditing.

Each pass has a page of its own if you want to run it by hand first, or to understand what it is doing overnight: pass 1 and 2 are the deposit → extract → supersede loop that mutable local sources depends on, pass 3's contradiction and duplicate findings are the same ones lint and review and co-evidential curation surface interactively, and pass 6 re-renders the MEMORY.md region described in User guide → Claude Code.

Scheduling (launchd / cron)¶

On a host, cadence comes from the operating system's scheduler, and --if-due makes over-scheduling harmless. There is no --install-schedule verb (deliberately: writing a LaunchAgent plist is a system-level footprint the installer never touches), so you install the job by hand, once.

Containers schedule themselves

An opt-in resident daemon mode is added (particles engine serve … --daemon) that runs this cycle in-process on a timer, because a container has neither launchd nor cron. It is a rider on the external-scheduler contract, not a replacement: everything on this page stands for CLI-only use and for hosts that run no daemon, and daemon mode is off unless you ask for it. See Running in a container.

A scheduled job inherits nothing: bake in absolute paths

This is the one way to get a job that looks healthy and does nothing. A LaunchAgent runs with no working directory (effectively /), no shell profile, and a minimal environment. A bare particles memory consolidate therefore finds no config.yaml (so none of your named stores or consolidation tuning exists), resolves a relative storage.database_url against the wrong directory (so it opens an empty store rather than yours), and sees no ANTHROPIC_API_KEY (so every semantic pass degrades to structural-only). With no log paths set, the disclosures that would tell you all three go nowhere.

The same trap bit the session hooks, which silently dropped harvests until v1.70.2 baked absolute env pins into their command strings. Every path and variable below is absolute for that reason; substitute your own, and do not shorten them to relative forms.

DATABASE_URL pins only the default store. It overrides storage.database_url, and that is the default handle's DSN. Any other handle resolves from storage.stores[<handle>] in config.yaml and ignores DATABASE_URL completely (particles/db.py). In practice:

running against default (the example below): set both PARTICLES_CONFIG and DATABASE_URL;
running against a named store (--store memory, --store research): PARTICLES_CONFIG is the load-bearing pin, because the DSN comes from the config file; DATABASE_URL does nothing for that handle, and setting it alongside a named store is the misleading combination to avoid.

PARTICLES_BLOB_DIR matters as much as the database URL. The corpus stores raw source bytes as content-addressed blobs, and storage.blob_dir defaults to the relative ./corpus_blobs, so a process started from a different directory writes blobs somewhere else while happily sharing the same database. The rows then reference content that is not where this process is looking, and extraction fails with Blob not found for hash …. Observed in practice: deposits made from several git worktrees left one database pointing at blobs scattered across five corpus_blobs/ directories, and blobs written inside a worktree that was later deleted were lost outright. Pin it absolutely here and set storage.blob_dir to an absolute path in config.yaml.

To check a store's blobs are all reachable:

# any blob directories other than the one you intend?
find "$(dirname /Users/you/src/myproject/particles.db)" -name corpus_blobs -type d

macOS (launchd)¶

Write ~/Library/LaunchAgents/dev.particles.consolidate.plist. Two things to substitute:

every /Users/you/src/myproject path, with your project's absolute path;
the store handle: the value after --store in ProgramArguments below. A store handle names a database: default is the implicit store (most setups have only this one), and any other handle must be declared under storage.stores in your config.yaml (see config.yaml.sample).

Do not be thrown by memory appearing twice in a full command line like particles memory consolidate --if-due --store memory: the first is the CLI command group, the second would be a store that happens to be named memory. The example below uses the handle default to keep the two distinct, and because of the pin rule in the next paragraph.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
  "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>
  <string>dev.particles.consolidate</string>

  <!-- Absolute path to the venv console script; launchd has no PATH. -->
  <key>ProgramArguments</key>
  <array>
    <string>/Users/you/src/myproject/.venv/bin/particles</string>
    <string>memory</string>       <!-- the CLI command group: `particles memory …` -->
    <string>consolidate</string>
    <string>--if-due</string>
    <string>--store</string>
    <string>default</string>      <!-- the STORE HANDLE (see below) -->
  </array>

  <!-- Without these the job runs against compiled-in defaults and an
       empty store. PARTICLES_CONFIG finds your operator config; an
       absolute DATABASE_URL is required whenever the configured DSN is
       not already absolute; PARTICLES_BLOB_DIR pins the content-addressed
       blob store, whose compiled default `./corpus_blobs` is RELATIVE and
       therefore follows the process's working directory. -->
  <key>EnvironmentVariables</key>
  <dict>
    <key>PARTICLES_CONFIG</key>
    <string>/Users/you/src/myproject/config.yaml</string>
    <key>DATABASE_URL</key>
    <string>sqlite+aiosqlite:////Users/you/src/myproject/particles.db</string>
    <key>PARTICLES_BLOB_DIR</key>
    <string>/Users/you/src/myproject/corpus_blobs</string>
  </dict>

  <!-- Belt and braces: also gives relative paths a sane base. -->
  <key>WorkingDirectory</key>
  <string>/Users/you/src/myproject</string>

  <key>StartCalendarInterval</key>
  <dict>
    <key>Hour</key><integer>3</integer>
    <key>Minute</key><integer>30</integer>
  </dict>

  <!-- Cron mails you on failure; launchd does not. These files ARE your
       failure notification; the exit-code table below is only useful if
       someone reads it. -->
  <key>StandardOutPath</key>
  <string>/Users/you/Library/Logs/particles-consolidate.log</string>
  <key>StandardErrorPath</key>
  <string>/Users/you/Library/Logs/particles-consolidate.err</string>
</dict>
</plist>

The API key¶

The semantic passes read ANTHROPIC_API_KEY from the process environment. There is no secrets file and no PARTICLES_SECRETS variable (particles/secrets.py is an internal module that calls os.environ.get("ANTHROPIC_API_KEY"); it is deliberately absent from config.yaml, because secrets never live in config), and launchd does not source shell files: EnvironmentVariables is a literal key→value dict, so pointing it at ~/.zshenv or ~/.zprofile sets a useless string and the job runs key-less. Three honest options:

1. Key in the plist (simplest; the plist becomes a secret: chmod 600, never commit it):

  <key>EnvironmentVariables</key>
  <dict>
    <key>PARTICLES_CONFIG</key>
    <string>/Users/you/src/myproject/config.yaml</string>
    <key>DATABASE_URL</key>
    <string>sqlite+aiosqlite:////Users/you/src/myproject/particles.db</string>
    <key>ANTHROPIC_API_KEY</key>
    <string>sk-ant-…</string>
  </dict>

2. Wrapper script (keeps the key out of the plist; the script is what sources a file). Point ProgramArguments at the wrapper instead of at particles:

#!/bin/sh
# ~/bin/particles-consolidate.sh (chmod 700)
. "$HOME/.particles-env"          # a chmod 600 file: export ANTHROPIC_API_KEY=sk-ant-…
export PARTICLES_CONFIG=/Users/you/src/myproject/config.yaml
export DATABASE_URL=sqlite+aiosqlite:////Users/you/src/myproject/particles.db
export PARTICLES_BLOB_DIR=/Users/you/src/myproject/corpus_blobs
exec /Users/you/src/myproject/.venv/bin/particles memory consolidate --if-due --store default

Verified to work from a stripped environment (env -i), which is what launchd supplies. Note . (POSIX source), not source: the script runs under /bin/sh, not zsh.

3. No key at all. A key-less run is honest, not broken: it completes, writes its run record, and discloses the downgrade rather than reporting a clean bill. In the log you will see

  semantic passes skipped: no API key (structural-only run)

and the structural passes (curation refresh, projection re-render) still do their work. Choose this if you would rather not put a key on disk; see Degradation.

Install, verify, and remove:

# install (modern launchd; bootstrap replaces the deprecated `load -w`)
launchctl bootstrap gui/$UID ~/Library/LaunchAgents/dev.particles.consolidate.plist

# confirm it is registered
launchctl print gui/$UID/dev.particles.consolidate | head -20

# force one run now instead of waiting for 03:30: the real smoke test
launchctl kickstart -p gui/$UID/dev.particles.consolidate

# uninstall
launchctl bootout gui/$UID/dev.particles.consolidate

After the kickstart, verify it hit your store rather than an empty one:

# The run must appear in the store the job actually targeted. `events list`
# reads the *default* store and has no --store flag. For the `default` handle
# below, pointing DATABASE_URL at the same file the plist pins is exactly the
# check you want. (If your job targets a NAMED store, set PARTICLES_CONFIG here
# instead and give that store's DSN; DATABASE_URL cannot reach it.)
DATABASE_URL=sqlite+aiosqlite:////Users/you/src/myproject/particles.db \
  particles events list --type CONSOLIDATION_RUN --limit 1

# ...and the log should not be reporting an empty or missing store
tail -20 ~/Library/Logs/particles-consolidate.err

If events list shows no new record, the job ran against the wrong store. Re-check PARTICLES_CONFIG and DATABASE_URL in the plist before trusting the schedule.

Linux / BSD (cron)¶

30 3 * * *  cd /path/to/project && PARTICLES_CONFIG=/path/to/project/config.yaml .venv/bin/particles memory consolidate --if-due --store default

cd covers the working-directory half of the trap, and cron mails you on a non-zero exit (see the exit-code table below), so cron needs less scaffolding than launchd, but set PARTICLES_CONFIG explicitly anyway, since cron's environment is also minimal.

--if-due reads the verb's own last successful CONSOLIDATION_RUN event (an interactive particles audit writes the same event type but does not count; a disclosed structural-only run does, so a key-less setup retries next interval instead of hot-looping) and exits 0 without running (one log line, no run record) when that run is younger than consolidation.min_interval_hours (default 20: daily scheduling with headroom for clock drift). A laptop that was asleep at 03:30 can therefore safely retry hourly; two overlapping schedules collapse to one run per interval.

Exit codes (cron observability)¶

Code	Meaning
`0`	Success, including disclosed structural-only runs and `--if-due` / lock skips
`1`	One or more passes failed (the run record is still written; stderr names them)
`2`	The cycle could not start (unusable store, config error)

Cron's mail-on-nonzero fires exactly on the two conditions an operator must see.

Reading the logs, and knowing when a run finished¶

The log files append; nothing rotates them. launchd opens StandardOutPath / StandardErrorPath in append mode, so every run stacks below the last one; cat will show you the oldest run first and get less useful over time. Use tail:

tail -50 ~/Library/Logs/particles-consolidate.log   # the most recent run
tail -f  ~/Library/Logs/particles-consolidate.log   # watch one finish live

Is it still running? launchd gives no completion notification, and a first run on a populated store can take a long while (extraction backlog + the census). Three checks, most direct first:

# alive? (silent = finished)
pgrep -f "memory consolidate"

# state, and the exit code of the last COMPLETED run
launchctl print gui/$UID/dev.particles.consolidate | grep -E "state|last exit code"

# the authoritative record, written even on partial failure
DATABASE_URL=sqlite+aiosqlite:////Users/you/src/myproject/particles.db \
  particles events list --type CONSOLIDATION_RUN --limit 2

last exit code reports the previous run while a new one is in flight, so read it together with state (running vs not running); map the value with the table above.

Rotating the logs¶

Nothing rotates these files for you. newsyslog.d is the system mechanism but needs root to install and validate, so the self-contained option (and the one that composes with the wrapper script above) is to rotate at the start of the wrapper, when no run is in flight. That ordering matters: the job is periodic rather than long-lived, so between runs no process holds the file open and a rename-based rotation is safe.

#!/bin/sh
# ~/bin/particles-consolidate.sh: rotate first, then run.
rotate_log() {                      # rotate_log <path> <max_bytes> <generations>
  f=$1; max_bytes=$2; keep=$3
  [ -f "$f" ] || return 0
  [ "$(stat -f%z "$f")" -gt "$max_bytes" ] || return 0
  i=$keep
  while [ "$i" -gt 1 ]; do
    prev=$((i - 1))
    [ -f "$f.$prev.gz" ] && mv "$f.$prev.gz" "$f.$i.gz"
    i=$prev
  done
  gzip -c "$f" > "$f.1.gz" && : > "$f"   # truncate in place, keep the inode
}

LOGDIR="$HOME/Library/Logs"
rotate_log "$LOGDIR/particles-consolidate.log" 524288 7   # 512 KB, 7 generations
rotate_log "$LOGDIR/particles-consolidate.err" 524288 7

. "$HOME/.particles-env"
export PARTICLES_CONFIG=/Users/you/src/myproject/config.yaml
export DATABASE_URL=sqlite+aiosqlite:////Users/you/src/myproject/particles.db
export PARTICLES_BLOB_DIR=/Users/you/src/myproject/corpus_blobs
exec /Users/you/src/myproject/.venv/bin/particles memory consolidate --if-due --store default

Verified behaviour: generations shift .1.gz → .2.gz → …, the count is capped at keep, .1.gz holds the most recent content, and the live file is truncated to zero rather than unlinked (: >), so anything still holding the descriptor keeps writing to the same inode.

Flags¶

particles memory consolidate
  --store HANDLE          # default: the default store
  --if-due                # cadence guard (see above)
  --structural-only       # skip all LLM passes (disclosed, never silent)
  --scope delta|store     # semantic-pass scope (default delta)
  --output FILE           # also write the run report as Markdown
  --format markdown|json  # terminal format (default markdown)
  --history               # print each past run's closure measure; runs nothing
  --verbose / --debug

There is no confirmation prompt and no --yes: an autonomous verb cannot prompt, so cost is bounded by construction (the existing caps plus delta scope) and confirmation is replaced with disclosure.

The closure measure¶

A store nobody curates still works: ingest runs the trust ladder, query returns both sides of a disagreement under the contested badge, and the curation queue stays bounded. A contradiction is only disclosed until something closes it, though, so the share of the store under a badge can creep upward until the badge stops meaning anything. Every run measures two numbers so that drift is visible:

Contested fraction: ACTIVE beliefs carrying the contested badge, over all ACTIVE beliefs. The count is store-wide on every run, whatever the census cadence or delta scope.
Autonomous share: of the lifecycle transitions since the previous run, the share no explicit write caused. Three kinds count. A belief leaving ACTIVE is a gesture when a supersede, retract, source retract or review event names it, and autonomous otherwise (the trust ladder, update supersession, duplicate merge, re-anchor, staleness). An INCONSISTENCY record closed by a review is a gesture, and one the disclosure pass closed is autonomous. An abstraction is a gesture when the accept gesture asserted it, and autonomous when the abstraction pass promoted it on its own. Gestures include an agent's own supersede and retract writes, and the run record's gesture_by_actor keeps them apart from the operator's.

The report prints both after the headline counts:

  contested        37 of 1204 active beliefs (3.1%) [inconsistency 35, stance 2]; previous run 3.0%
  transitions      12 since the previous run: 12 autonomous, 0 by gesture (100.0% autonomous)

Each run's window opens where the previous run's closed, so consecutive runs tile the event log. The run record carries the counts behind both numbers under its closure key, including the autonomous retirements by status reason. particles memory consolidate --history prints the series, one row per run, oldest first, and --format json gives the full records. The measure changes nothing in the store and moves no closure default.

Delta scope¶

By default the semantic passes probe only the particles created or modified since the watermark, plus particles from corpus entries deposited since then. The watermark is the previous eligible run's started_at (not its completion, so nothing written mid-cycle ever falls between two runs' windows; overlap re-probes are idempotent), and the scope is computed after pass 1, so the particles extraction just minted are censused in the same run. Watermark-eligible means: a successful, non-degraded run by the consolidation verb itself; a structural-only night or an interactive particles audit never advances the watermark. Nightly cost therefore scales with the day's delta, not the store. The first run (no prior eligible record) and --scope store run store-wide, still capped, with the "probed X of Y candidate pairs" disclosure. The below-cap tail of pre-existing pairs is never reached by delta-scoped runs. Repeated --scope store runs do reach it, because a pair already cleared is skipped without spending the cap, so each run probes further down the similarity order. particles lint remains the exhaustive instrument in one run.

The census's own window opens at the last census's started_at, not the last run's, so a belief written on a night the census skipped is probed by the next one.

The census cadence¶

The census is the cycle's largest LLM cost and the only report-only pass: its findings become curation cards, and nothing it finds changes what retrieval returns. On the measured LongMemEval cycle (2026-09-20) it made 30,000 of the cycle's 36,627 calls. It therefore runs every consolidation.census.interval_hours (default 168, weekly) while every other pass runs nightly. A run up to an hour early still counts as due, so a scheduler's drift does not slip it a night.

A night that skips it says so in the report, in place of the counts:

  census skipped: last ran 2026-09-28 03:30 UTC, next due 2026-10-05 03:30 UTC (consolidation.census.interval_hours = 168)
  as of that census: 4 contradictions, 16 duplicate pairs, 7 stale

The counts are the last census's, not that night's, and the run reports no deltas. The next census reports its deltas against the last census. Between censuses the curation queue is served from the last census's stored collection, and particles curate does not mark it stale until it is a day older than the interval.

The first run on a store runs the census, as does any run when no census is on record.
--scope store runs it whatever the interval: it is the deliberate whole-store census.
Only a full census counts. A structural-only or key-less night that runs the census probes nothing, so it does not reset the interval, and an interactive particles audit does not either.
consolidation.census.enabled: false never runs it from the cycle; the queue keeps serving whatever collection is stored.
interval_hours: 0 runs it on every cycle, the behaviour before the setting existed.

particles audit is unaffected: it runs its own census on first contact.

Degradation: structural-only is disclosed, never silent¶

With no API key, an open circuit breaker, --structural-only, or consolidation.semantic: false, the LLM-free passes (curation refresh, projection render) still run in full; extraction and the probe-bearing reconcile sweep are skipped (each disclosed), the census runs structural finders + REPORT-mode duplicates only, and utility mining runs the literal tier only. Every skip is disclosed in the report and recorded on the run record (semantic_degraded + reason). A degraded run's contradiction line reads "not probed this run", never "0", and a degraded run never advances the delta watermark, so everything it did not probe stays in scope for the next full run.

The run record¶

Each run writes one CONSOLIDATION_RUN operator event with a versioned payload (format: 1): per-pass status/durations, per-pass LLM call counts, the machine-readable census (probe counts, duplicate totals, pending backlog, utility events), degradation disclosures, provider/model per purpose, and started_at / completed_at (started_at is the next run's delta watermark; see Delta scope). "Is the cycle actually running?" is one command:

particles events list --type CONSOLIDATION_RUN

particles audit records the same event shape (actor: audit) and the report's headline lines carry "+2 since the last census" deltas against the most recent prior run of either kind that ran a census, but an audit event neither advances the consolidation watermark nor satisfies --if-due (the audit runs none of the cross-session passes, so it cannot stand in for a consolidation run).

Concurrency and failure¶

One cycle at a time. A consolidate.lock file in the integration state directory (claude_code.state_dir) serializes cycles. The running cycle holds a kernel advisory lock on it, so a cycle that is still running is never reclaimed, however long it takes, and a crashed cycle frees the lock the moment its process exits. A second caller starts nothing and exits 0 with a message naming the holder, for example consolidation already running (pid 4242 on mbp.local, started 2026-09-29 10:30 UTC, pass extract since 10:31 UTC) — skipped. Once the holder has run longer than consolidation.lock_timeout_minutes (default 120), the caller also writes warning: consolidation lock held for 8h 19m; if it is hung, stop pid 4242 to stderr, still with exit 0. A hung cycle is never stopped for you: check the pid and stop it if it is not making progress. On a filesystem without advisory locks, and against a lock left by a pre-change version, the lock falls back to the old rule (a dead pid, or older than lock_timeout_minutes, is reclaimed). Upgrade every venv or image that runs cycles against one state directory at the same time.
Interactive sessions interleave safely. The lockfile serializes cycles, not writes: each pass takes the cross-process write lock per transaction exactly as its verb always has, and every pass is idempotent.
Failure mid-pass: continue and report. A failing pass is caught, recorded on the run record (failed(<error>)), and the cycle continues; the zero-LLM projection render still runs, so a flaky network night never leaves MEMORY.md staler than it had to be.

Configuration¶

consolidation:
  min_interval_hours: 20    # --if-due threshold
  extract_pending: true     # pass 1 on/off
  max_pending_entries: 20   # pass 1 per-run cap
  extract_batching: true    # pass 1 pooled half-price batching
  semantic: true            # LLM passes on scheduled runs
  max_reconcile_probes: 50  # pass 2 per-run probe cap (highest-similarity-first)
  lock_timeout_minutes: 120 # warn that a holder may be hung
  lock_heartbeat_seconds: 60        # holder heartbeat interval
  lock_heartbeat_stale_minutes: 10  # another host's holder is stopped after this
  contradiction_disclosure:
    enabled: true           # pass 3b on/off
    max_per_run: 10         # new records opened per run; the rest wait
  reanchor:
    enabled: true                     # pass 2c on/off
    max_retirements_per_run: 20       # update retirements examined per run
    max_candidates_per_retirement: 8  # claims sent in one probe call
    max_restatements_per_run: 20      # second readings, and so writes, per run
  batch_wait:
    budget_seconds: 3600        # total batch waiting per run
    min_remaining_seconds: 300  # below this, remaining sets run sequentially
  census:
    enabled: true               # pass 3 on/off from the cycle
    interval_hours: 168         # weekly; 0 = every run

Detection thresholds deliberately stay where they live (audit.*, lint.*, utility.mining.*, links_suggest.*): consolidation composes the finders, it does not re-tune them.

Half-price probes: batch completion¶

Nobody is waiting for a 03:30 run, so its two largest probe populations, the contradiction probe (capped at audit.max_contradiction_probes) and the utility use judge (utility.mining.max_behavioural_calls), are submitted to the Anthropic Message Batches API instead of one call each: the contradiction probe as one job, and the matcher's requests for every session the pass mines pooled into one more. All token usage in a batch is billed at 50%. At today's caps that is roughly 1200 of the cycle's ~1250 nightly probe calls.

The trade is latency for price: a batch usually completes within an hour and may take up to 24, which is why only a caller that has declared itself latency-tolerant takes it. particles lint and the first-run particles audit always run the sequential per-probe loop and keep their answer-in-seconds, no matter what is configured here.

llm:
  batch:
    enabled: true                # false ⇒ everything sequential, as before 0218
    min_requests: 4              # below this, a batch is not worth the round trip
    max_requests_per_batch: 1000 # a larger set is chunked into successive batches
    poll_interval_seconds: 30    # how often processing_status is checked
    max_wait_seconds: 3600       # per-batch wall-clock ceiling

Failure behaviour is the existing probe-unavailable degradation, at three levels. A batch the provider refuses outright falls back to sequential calls (you lose the discount, not the pass). A batch still processing after max_wait_seconds is cancelled and its probes report unavailable, so a stuck job cannot hold the cycle open until the API's own 24-hour expiry. An individual request that errored or expired inside an otherwise healthy batch comes back as one unanswered probe, counted in the run's disclosure, never silently dropped.

One pass is not batched: the reconcile sweep (50 probes/night, whose loop skips candidates already demoted earlier in the same loop). It is still billed per call.

To turn the whole thing off and get a cycle that finishes fast at full price, set llm.batch.enabled: false.

Half-price extraction: the pooled extract pass¶

The extract pass is the cycle's dominant token consumer (the Claude Code session harvests are large transcripts), and since it rides the same Message Batches discount: the capped pending set runs as concurrent per-snapshot tasks whose chunk requests merge into one nightly batch job through a completion pool, so the whole set (single-chunk documents included) clears llm.batch.min_requests together. Wall clock for the pass becomes roughly one batch turnaround instead of 10–20 sequential multi-minute calls.

What does not change: interactive particles extract and reindex stay sequential; per-snapshot failure isolation is the existing machinery (an unanswered request resets its snapshot to PENDING for the next night; an account-level failure stops the pass with one disclosed error); and at most one snapshot per corpus entry runs per night, so carry-forward keeps seeing the previous snapshot's persisted chunk hashes. Vision-flagged PDF pages and image sources are multimodal and still run sequentially at full price.

consolidation.extract_batching: false restores the serial per-snapshot loop exactly; llm.batch.enabled: false keeps the pooling but degrades dispatch to sequential full-price calls.

A budget for the whole run's batch waiting¶

llm.batch.max_wait_seconds bounds one batch. A run submits several, one pass after another (the extract batch, the census probes, then the utility matcher's batch), and on a slow-batch night each can wait out its full ceiling. On 2026-09-29 nine batches did, and the scheduled run was still going ten hours after it started.

consolidation.batch_wait.budget_seconds bounds the sum. Each batch waits at most the balance left, and once less than min_remaining_seconds remains, the rest of the night's sets run as ordinary sequential calls at full price. The extract batch comes first, so it always has the whole budget and keeps the half-price path; what moves to full price on a slow night is the tail of the census and utility work: short probes that, on the store this was measured on, bill under half a dollar a night at full price. Nothing is skipped, and the results are the same either way.

A spent budget is disclosed in three places: a clause on each affected pass in the run record, a batch_wait object in the CONSOLIDATION_RUN payload that names the pass during which the budget ran out, and one line in the report:

  batch-wait budget: waited 3512s of 3600s, spent during extract; 1 batch(es) cut short (12 request(s) unavailable), 2 set(s) sequential at full price (31 request(s)) (consolidation.batch_wait.budget_seconds)

A batch cut short by the budget reports its requests unavailable, as an expiry at max_wait_seconds does: its snapshots return to PENDING and its probes go unanswered this night. Such a night is not marked degraded. The budget covers waiting only; sequential work and the zero-LLM passes are not charged to it.

When the backlog outgrows the cap¶

Pass 1 extracts at most consolidation.max_pending_entries corpus entries a night. Every Claude Code session end deposits a new transcript snapshot and every edited memory file a new generation, so on a busy week the backlog can grow faster than the cap drains it. Nothing fails when that happens; new material simply waits. The report line and the run record are where it shows:

  pending          extracted 13, 2 left for retry (296 remain, oldest waiting since 2026-07-19; the next run continues)

Four behaviours keep one bad snapshot from holding the queue:

Least-tried first. Each snapshot counts how many times extraction has claimed it. A snapshot whose every LLM call failed (a reply cut at the output budget, a batch still running at llm.batch.max_wait_seconds) is handed back PENDING and waits behind every snapshot not yet tried, where before it was first in line again the next night.
The cap counts entries. One snapshot per corpus entry runs per pooled night, and the cap is filled with distinct entries, so a transcript with six pending snapshots takes one slot rather than six.
Stranded claims are released. A snapshot left IN_PROGRESS by a killed run for longer than extraction.stale_in_progress_minutes is returned to PENDING at the start of the pass.
Nothing is counted twice. extracted counts only snapshots that reached COMPLETE. A snapshot handed back PENDING is reported as left for retry, and one that completed with no belief written, carried forward or matched is reported as with no beliefs (and listed by particles lint as EMPTY_COMPLETE_SNAPSHOT). The run record carries the same counts as pending_extracted, pending_retry, pending_empty, pending_failed, pending_reset_stale and pending_oldest_at.

A COMPLETE snapshot is not always an extracted one. The refresh pass records an unchanged local file as a REVISIT snapshot with status COMPLETE, which means "no fetch work is owed", even when the RESPONSE snapshot it points to is still PENDING. To ask whether a source's content was extracted, read the status of its RESPONSE snapshots.

Catching up by hand. When the backlog you care about is small and the rest can wait, extract it directly. --tag scopes --all-pending to the corpus entries that carry a tag, and the Claude Code harness tags every memory file it harvests memory-file:

particles extract --all-pending --tag memory-file

This runs sequentially at full price (no batch discount), releases stranded claims first, extracts only the newest generation of each edited file, and exits 1 if any snapshot failed or was left PENDING. Price it before you run it. particles audit ~/.claude/projects/<key>/memory --estimate prices every file in the memory directory at the same per-call rates without depositing or calling anything, which bounds this backfill from above.

Local-model structured output¶

The cycle's JSON-shaped LLM calls (extraction, the contradiction probe, the behavioural utility matcher) now pass their reply schemas through the completion port. When a purpose is routed to a named OpenAI-compatible provider (e.g. the compiled-in local entry), the entry's structured_output: auto (the default) sends OpenAI-style response_format: {type: json_schema, strict: true} so a local model cannot silently drop particles behind unparseable JSON; an endpoint that rejects the parameter gets one retry without it (a logged downgrade back to tolerant-parser reliability). Set structured_output: off to disable. The Anthropic provider ignores the schema in v1. Note: v1 ships the semantic passes on the configured (Anthropic) provider; routing semantic_lint to a local model is one config edit, pending the probe-quality measurement named in the consolidation design.