-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Description
Description
When a subgraph with history_blocks (pruning) enabled is rewound using graphman rewind, the revert operation fails to properly re-open entity versions for frequently-updated entities. After the rewind, these entities have no open (current) version, causing all subsequent handlers to fail with deterministic unexpected null errors.
The rewind passes the existing guard from PR #5423 (earliest_block + reorg_threshold check), yet the resulting entity state is corrupted.
Environment
- graph-node version: 0.41.1 (
graphman 0.41.1 (2025-11-07)) - Chain: BSC (BNB Smart Chain)
- Subgraph: PancakeSwap V2 (
QmTXPMrFyoukdHmWjPp5aJER9NaX4hCsFKEodeqRNzQWCq)
Subgraph configuration
| Parameter | Value |
|---|---|
history_blocks |
500 |
earliest_block_number |
27,671,263 |
start_block |
6,809,736 |
Pruning state (before rewind)
first_block: 6,809,736
final_block: 27,671,513
latest_block: 27,671,763
history_blocks: 500
started_at: 2025-08-09 10:17:18
finished_at: 2025-08-09 12:06:20
Steps to reproduce
- Deploy a subgraph with
history_blocks: 500(pruning enabled) - Let it sync past at least one pruning cycle
- Execute
graphman rewindto a block within the valid range (>=earliest_block + reorg_threshold):- Rewind block: 27,671,513
- Rewind hash:
0xd1ff8512669957082b8a3600e474e69f8145fef68b55dccc07acbb281f3c7faa
- The subgraph immediately fails at the next block (27,671,514)
Observed behavior
After the rewind, frequently-updated entities have no open (current) version. All their block_range upper bounds are closed:
bundle entity (updated almost every block — price oracle)
SELECT count(*) as total,
count(*) FILTER (WHERE upper(block_range) IS NULL) as open
FROM sgd3776.bundle; total | open
-------+------
148 | 0 ← NO OPEN VERSION
Latest versions:
from_block | to_block
------------+----------
27671507 | 27671511 ← last version, CLOSED
27671506 | 27671507
27671505 | 27671506
The version [27671511, X) that was created when bundle was updated at block 27,671,511 is missing. It should have survived the revert (its lower(block_range) = 27671511 <= 27671513) and been re-opened.
pancake_factory entity (updated every block)
SELECT count(*) as total,
count(*) FILTER (WHERE upper(block_range) IS NULL) as open
FROM sgd3776.pancake_factory; total | open
-------+------
247 | 0 ← NO OPEN VERSION
Latest version: [27671512, 27671513) — closed, not re-opened.
pair entity (created once, rarely updated) — NOT affected
total | open
---------+---------
1303686 | 1300088 ← has open versions ✓
token entity (created once, rarely updated) — NOT affected
total | open
---------+---------
1239937 | 1235940 ← has open versions ✓
Expected behavior
After graphman rewind to block 27,671,513, the revert operation should:
DELETEentity versions withlower(block_range) > 27671513- Re-open entity versions covering block 27,671,513 by setting
upper(block_range) = NULL
For the bundle entity, the version [27671511, ...) (which covered block 27,671,513) should have been re-opened to [27671511, ∞). Instead, it was deleted or never restored.
Root cause hypothesis
The revert logic appears to incorrectly handle entity versions in pruned subgraphs. The interaction between the pruning state and the revert SQL causes versions that should be preserved and re-opened to be deleted instead. This affects entities that are frequently updated (where the version covering the rewind block has lower < rewind_block and upper > rewind_block), while rarely-updated entities (where the open version has lower << rewind_block) are unaffected.
Fatal error
transaction 7572757063fcbf868165139f9cd660b169b235458b141820178926668885ecf4:
error while executing at wasm backtrace:
0: 0x581e - <unknown>!mappings/core/handleSync:
Mapping aborted at mappings/core.ts, line 182, column 17,
with message: unexpected null
in handler `handleSync` at block #27671514
Multiple transactions at block 27,671,514 fail with the same pattern in both handleSync (line 182) and handleSwap (line 368) — all due to null entity loads for entities that should exist.
Impact
- The subgraph is unrecoverable without a full re-index from start block
- The deterministic error means it will never self-heal
- Any pruned subgraph rewound within the valid range is at risk
Related issues
- Add more checks for rewinds for subgraphs that have ongoing pruning enabled #5422 (closed): Added guard to prevent rewind past
earliest_block + reorg_threshold— our rewind passes this guard - [Bug] Unable to sync QoS subgraph #5912 (open): User stuck with "No rows affected" error — different symptom but same area (rewind × pruning interaction)