> ## Documentation Index
> Fetch the complete documentation index at: https://docs.nyumbazetu.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Data retention
> What ages out of the database, after how long, and how to read a retention sweep before anything is armed.
**Administration → Data Retention → `/data-retention`**
Some tables grow forever. Delivery logs, webhook events, poll failures, meter readings and audit
streams accumulate at a rate set by the size of the portfolio, not by anything a user does. Data
retention is the engine that ages those rows out on a policy, so storage stays bounded and queries
stay fast.
It is deliberately hard to arm. Two surfaces:
| Surface | What it is |
| ---------------------- | ------------------------------------------------------------------------------------ |
| **Retention Policies** | What ages out of which table, after how long, and whether the policy is armed. |
| **Retention Runs** | Every sweep, with what it removed — or in shadow mode, what it *would* have removed. |
## The safety model
An armed retention policy deletes production rows in bulk and the rows do not come back. Arming
requires **three** separate switches to line up:
1. `RETENTION_SWEEPER_ENABLED` — the environment-level master switch for the sweeper.
2. `RETENTION_ARMED` — the environment-level arm switch.
3. The per-policy `is_armed` column.
All three must be on before a single row is removed. Any one of them off means the sweep runs in
**shadow mode**: it computes exactly what it would delete, records the counts, and deletes nothing.
There is no arm/disarm button in the app, and this is intentional. Policies are authored and armed by
migration or explicit SQL, where the change leaves a reviewable trail. A browser toggle would collapse
a three-switch safety model into one click.
## Reading a policy
| Field | Meaning |
| ---------------------------- | ---------------------------------------------------------------------------------------- |
| **Policy key** | The policy's stable identifier. |
| **MySQL table** | The table it ages. |
| **Mode** | Shadow or armed — the effective mode, after all three switches are resolved. |
| **Evaluated by the sweeper** | Whether the sweeper considers this policy at all. |
| **Strategy** | `Delete rows` removes whole rows; `Blank columns` keeps the row and nulls named columns. |
| **Keep (days)** | Rows older than this are in scope. |
| **Aged on column** | The date column the age is measured from. |
| **Columns blanked** | For the blank-columns strategy, which columns are cleared. |
| **Extra filter** | An additional predicate narrowing what is in scope. |
| **Rows per statement** | Chunk size — how many rows one statement touches. |
| **Max statements per sweep** | Ceiling on work per run, so a sweep cannot monopolise the database. |
| **Swept / Result** | When the policy last ran and how it ended. |
**Blank columns** exists for tables where the row itself is still needed — a record of the event —
but the payload inside it is not. It shrinks the table without breaking references.
## Reading a run
Each sweep produces a run record per policy.
| Field | Meaning |
| -------------------------- | --------------------------------------------------------------------------------------- |
| **Mode** | Shadow or armed for *this run*. |
| **Result** | Succeeded, failed, or refused. |
| **Rows in scope** | What matched the policy. In shadow mode this is what *would* have been removed. |
| **Rows removed** | What was actually removed. Always `0` in shadow mode. |
| **Chunks executed** | How many statements ran. |
| **Backlog remaining** | The chunk ceiling was hit before the backlog cleared — more remains for the next sweep. |
| **Cutoff** | The computed date; everything older was in scope. |
| **Error / refusal reason** | Why a run failed or refused to proceed. |
A **refused** run is not a failure. It means a guard fired — the policy was misconfigured, the scope
was implausibly large, or a switch was off — and the engine declined rather than guessing.
## Bringing a policy into service
The policy is created disarmed. Nothing is deleted.
Check **rows in scope** against what you expect. A count far larger than anticipated usually
means the wrong date column or a missing filter.
Confirm the cutoff lands where the keep-days value implies.
Check nothing reports off it, and that any regulatory retention period is longer than the
keep-days value, not shorter.
By migration, with the shadow evidence in the change description.
Expect a backlog over several sweeps rather than one large deletion.
Deleting rows does not immediately return disk to the operating system — the space is reused by the
table before the file shrinks. Table size on disk will plateau before it falls.
## Common problems
| Symptom | Cause |
| ------------------------------------------ | ---------------------------------------------------------------------------------- |
| Runs succeed but remove nothing | Shadow mode. Check all three switches. |
| Rows in scope is zero | Wrong date column, or the extra filter excludes everything. |
| Backlog never clears | Chunk ceiling too low for the table's growth rate. Raise max statements per sweep. |
| Run refused | Read the refusal reason — a guard fired on purpose. |
| Table size unchanged after a large removal | Space is reclaimed internally first; the file shrinks later. |
| A policy is not listed | It is not enabled, so the sweeper never evaluates it. |
## Related
The sweeper's cadence and run history.
What the audit trail retains, and for how long.
Detecting data problems before they age out.
Platform-wide operational streams.